As much as we would like to, we cannot let the nVidia / FutureMark thing rest.
With the help of ATI, Scott from The Tech Report discovered that if you rename 3DMark03.exe to something different, overall 3DMark2003 scores with nVidia graphics cards in play with the new DET FX drivers are DECREASED. Basically nVidia's DET FX drivers tell 3DMark2003.exe to run lower image quality than usual, thus resulting in a higher score. Even with the new 3.3.0 patch from FutureMark it makes no difference and I doubt FutureMark will be able to fix the problem so the onus falls back on nVidia to make the next move.
If you're old like me, you may remember the day when ATI was caught optimizing its Radeon 8500 drivers for Quake III Arena timedemos. The trick was simple: ATI's drivers looked for an executable named "quake3.exe" and turned down texture quality when "quake3.exe" started. Kyle Bennett at the cold, HardOCP renamed "quake3.exe" to "quack3.exe" and ran some benchmarks. ATI was busted.I have defended nVidia for the past two weeks during the on-going fiasco but it seems I have been wrong - the benchmarks speak for themselves, I am now hoping nVidia change their ways from here on in for the better of the enthusiast computer user.
In a funny twist of fate, I got a tip earlier this week about NVIDIA's Detonator FX drivers. The allegation: if you rename 3DMark03.exe to something else and run the benchmark with anisotropic filtering enabled in the drivers, test scores drop. In other words, NVIDIA appears to be using the same lame technique ATI did way back when: keying on the program's filename in order to trigger benchmark "optimizations." In this case, those optimizations appear to be a lower quality form of texture filtering than the anisotropic filtering method selected in the driver control panel. Many review sites like us benchmark cards with anisotropic filtering and edge antialiasing turned on, so these things do matter.
Keep in mind, now, that this goes beyond the eight specific Detonator FX "cheats" FutureMark identified in its audit report. This is in addition to all of those things, and this one works with the latest release version of 3DMark, build 330. It appears NVIDIA has taken yet another measure, which FutureMark didn't catch in its audit, in order to boost scores in 3DMark03.
You probably won't be surprised to learn that my tip came from ATI. The ATI folks say they were studying the Detonator FX driver's new filtering routines, to see what they could learn, when they discovered this quirk. (Some have claimed NVIDIA was the original source for the Quake/Quack debacle, but I have no first-hand knowledge about that.) Regardless, we felt compelled to test the Detonator FX drivers to see if the allegations were true. So we gathered up a GeForce FX 5800 Ultra, a GeForce FX 5600, and a GeForce FX 5200 Ultra for a little benchmarking session. We brought along our typing fingers, so we could rename "3DMark03.exe" to "3DMurk03.exe" and see what happened. Our results follow.
Everyone is entitled to take advantage of something which is waiting to be taken advantage of, but when it deceives people unfairly it should be frowned upon deeply.
More information @ The Tech Report