Tangent first: First thing to do is visit Windows Update. Then visit DirectX .com to see if there isn't a newer version which supports Windows 98. Then visit nvidia.com and search for the newest graphics driver for that card. Then get rid of the graphics card because it still will be too slow...there's only so much you can do with 8 or maybe 16 MB of RAM. That's also a rather high amount of RAM for Windows 98 - not the most stable operating system. I would at least upgrade to Windows 98 SE (I don't condone piracy, but Microsoft doesn't seem to be enforcing activation on Windows 98 SE - there's a key which apparently a Microsoft person leaked for you to use. I suspect most people use it with Virtual PC). So then you'll want to get rid of Windows 98 as well, because the newest versions of your internet browser (certainly IE, certainly Firefox) don't support it. VLC Media Player doesn't support it after 0.8.6c (which was a couple years ago). Windows 98 isn't too stable and probably has more than its fair share of known security holes, so you won't want to do any internet banking, eBay, or much of anything else on it. If you
must use that PC, you probably should install Ubuntu just for the sake of newer software. I personally have a recently-set-up Windows 98 SE computer but I don't use it for these obvious reasons.
Okay, now to answer your question:
The computer may be 5 years old but the design of this card is over 10 years old. The TNT2 M64 is so old that its core predates the GeForce name (I think the specific model came out after the original Geforce 128...edit, GeForce 256...goofy name). It's a low-cost version of a 1999 card. Looked at Wikipedia a bit; apparently it outperforms the TNT...a 1998 card...but not the TNT2. The M64 comes from it having only a 64-bit bus, as opposed to an 128-bit one. You can read more on it
here.
I used one of these for a number of years - it came in a OEM PC, probably a Presario (I'm not positively sure, but probably a Compaq 5461 sitting on the floor to my immediate right). That PC's CPU is an AMD K6-2, 450 or 500 MHz, 3D Now! technology (extra instruction set, supporting SIMD operations - helps graphical operations).
Poor performance is normal for this card.
I've never used an ATi card (though I've been sitting on one for nearly two years now...) so I can't compare. There were different models of Rage 128, too, I believe, but overall I think they were rather close in spec, being out about the same time. The one good point about the TNT2 M64 is that it has a better list of bullet-points than the other cards, such as supporting bigger textures, and some other stuff. I think even better color. But not speed, necessarily.
I vividly remember, back in the good old days, editing the .WAD with Day of Defeat's texture file in hopes of lessening the hit on the graphics card (as a whim, I also tried replacing weapon sounds with the spoken name of each gun...I learned quickly about how sounds are actually played...). I say "the good old days" because the game wasn't smart enough to notice the difference, no VAC ban, heh. I don't think it made much of a difference to FPS, unfortunately. Day of Defeat, of course, is an old Half-Life engine modification. Now, how much of that was due to the CPU of the system, and how much was due to the GPU. I also remember there being trouble (well, some would call it a feature) with transparent walls (and very poor frame rates) in Quake 3, too. I don't think I ever ran games far below 1024x resolution because, well, there wasn't much point (it's especially important in a game like DOD to have a high resolution so you can see and actually aim accurately at things in the distance).
Thankfully, you can get a much better card for almost nothing.
First thing for you to do is compare what's in the old computer with the new one. I'll take your word that the old computer's components are mostly inferior to the new one; however, if only the motherboard is at fault, you may still have a graphics card in there, and it sounds like it might be better than what's in this system. There may also be other components worth taking out, like a sound card or a LAN card...though I doubt it. But it's still worth a look. The old hard drive may be worth saving as a special backup drive at least. Overall, though, it sounds like it's all ready to be parted out for donation, and the chassis / motherboard recycled.
What you upgrade with depends on how what kind of bus you have available - AGP or just plain PCI. How much wattage your power supply provides has an impact as well, but since there are lots of low-cost, low-heat, low-power options available that are also very cheap by now and also far more powerful than the TNT2, you have lots of options. AGP is too slow for modern purposes, but it was considerably more useful for graphics than standard 32-bit PCI ports. You can get fairly recent cards in both formats - I believe I've seen plain PCI cards from 2005 (about the time the new PCI-E standard appeared) - and AGP cards survived a bit longer still.
If you are serious about putting an AGP card into this old system for games up to maybe five years old (any newer and I think they would be terribly slow or at least require very low resolution and many features turned off), here's Tom's Hardware's recommendations:
Best AGP cards for the money - January 2009. I think that's about the time they got rid of the AGP category entirely. PCI cards are a different matter, and you'll have to go back a few years further. Also, if you want to improve performance of an old computer, see if you can't find a good cheap IDE hard drive - there were some on Newegg somewhat recently, and they probably have much more space than your old drive, and might be very noticeable faster + quieter (not to mention more reliable) than the old one.
There is one other thing that comes to mind - you say the computer has 768MB of memory, which is either an odd number of memory modules (3 256MB modules) or unmatched ones (i.e. 256MB put in by the manufacturer for a basic configuration, plus a 512MB module). I don't know how much of an impact it makes, but it is possible in some cases to get performance problems from different specced RAM modules (though usually this is limited to a faster RAM stick simply clocking down to the speed of the slower one, no big deal). Probably isn't any problem, and it's best not to mess with it unless you're seeing troubles outside games.
If you want my advice, save up a few dollars to get an old computer with a dual-core or 64-bit CPU, and a motherboard with a PCI-E bus, and SATA hard drive connections. The motherboard will have to support all three elements, and all new hardware in the immediate future is being based off these parts. You get better bang for the buck than in trying to hunt down parts for obsolete systems, SATA is easier to manage than huge thick IDE cables, and you can do this all while still not using a lot of power. If anything dies in a system like this, for the next five years at least you will only have to replace one part, as opposed to having to replace the whole system including many perfectly working parts. For me, the winning bit is that you don't have to wait forever and a day for tasks to complete. Though I realize by advocating getting a newer system, I'm suggesting just the reverse of this, but AGP and that other stuff is SO OLD.
There is a good old saying - "if it ain't broke..." - but sometimes when you see a problem coming over the horizon it can be worth taking a hit now to prevent being stuck without your essential tools. For me, there's little as aggravating as having to wait around for your computer to do things, which was the case on old systems (such as those with AGP graphics cards). Just getting a dual-core CPU has made life immensely easier.