Sure, PCs from when 98 was out should all have USB ports. Just make sure the drive supports it, things are not always guaranteed to work in such an old OS, especially larger drives. If the PC does not have USB2 though expect very slow file copy speeds.
If you do not have Windows 98 drivers for you external drive then you need a generic driver the best one to use is Maximus Decim Native USB ver.3.3 (3.5 is available)
If you do not have Windows 98 drivers for you external drive then you need a generic driver the best one to use is Maximus Decim Native USB ver.3.3 (3.5 is available)
The generic USB mass storage driver normally works. If it does not, you may be able to find Windows 98 drivers for your external hdd on the manufacturer's website.
The first driver set includes USB 2.0 host controller drivers for various chipsets.
Be aware that your old hardware may not have a USB 2.0 controller, in which case you will be limited to transfer rates of about 1MB/s, whereas USB 2.0 will typically achieve 20MB/s. Look for an EHCI (enhanced) host controller in Device Manager, or use Microsoft's UVCView utility to detect your USB hardware:
If the USB 2.0 hardware isn't there, you can install a cheap PCI USB 2.0 card. However, I would avoid those with a VIA chipset. These have lots of issues, as discussed at the following URL.