Doom 3 is finally here and we are tipping many gamers around the world are dieing to benchmark their system with one of the most anticipated games to hit computer screens ever. We've posted the definitive guide to benchmarking with DOOM 3 which includes information on exactly how to benchmark the game and the issues to look out for. It's benchmark time!
IntroductionOn the 3rd of August 2004, ID Software unleashed Doom 3 to gamers all over the globe. Shop stores all around the world replaced the empty promotional boxes with actual copies and online stores shipped all the pre-orders to gamers who have been eagerly waiting many years for the next version of the cult-type game series which is Doom.Doom 3 is here and is certain to become a major success with all genres of gamers because of its cult following and ultra realistic graphics and cinematic-like gaming style which as others have suggested makes you feel like big bad Bruce Willis in the next big Hollywood shooting hero movie. We as gamers have never seen this type of realism in games and it's really just a start for what manufacturers like ATI and nVidia are hoping to help the game developers create for all of us in years ahead, now that the hardware required is available to render such impressive graphics. It's certainly a fun and refreshing time to be in the computing industry - definitely a breath of fresh air.After playing single player Doom 3 for only an hour I was automatically compelled to figure out exactly how to benchmark the game on my hardware - and I am tipping most people reading TweakTown would have felt the same way. I was genuinely interested in benchmarking all of the latest graphics cards we have in our labs from our generous suppliers. In this line of work, when you are benchmarking or seeing products being benchmarked almost 7 days a week for years on end, it can become difficult to appreciate the process.
On the day we first got our copy of Doom 3, it brought me back in time a few years to the very first Quake 3: Arena days when we first started the little website called TweakTown when we had a very strong interest in the benchmarking process and what hardware offered the best frame rate performance for our ultra l33t 3D gaming experience. Doom 3 offers enthusiasts not only a great gaming experience but the chance tweak their systems to increase frame rates in the game which can often go well into the single digit range even with high-end systems and the latest graphics cards. Our focus here today is not on comparing the latest graphics cards in Doom 3 (many sites such as AnandTech, AMDZone and VR Zone have already covered it in good detail) but to provide the definitive online resource on exactly how to benchmark Doom 3 and the various issues involved in doing so in a coherent and reliable manner. We are sure many of you are dieing to see how your hardware stacks up against your best friends PC and in the following pages you will learn exactly how to benchmark Doom 3, the issues involved, what to use and finishing it all off comparing an ATI Radeon X800XT PE with ATI's current official 4.7 drivers along with their latest beta 4.9 Catalyst drivers which offer better OpenGL performance which is said to help increase performance of X800 graphics cards in the game.Last Updated: Valid as of 5th of August 2004
Doom 3 Benchmarking Guide - Quick FPS Lesson
Quick FPS Lesson - Let's quickly recapWhat exactly is FPS?I think it is important we all recap on FPS no matter if you are a new gamer this week with the launch of Doom 3 or a gamer from way back who shoot baddies back on the Commodore 64. FPS (or Frames per Second) refers to the amount of times a frame is rendered (drawn or displayed) on the screen during game play. Generally, the higher the FPS or frame rate, the better. We will be referring to the term "FPS" and "frame rate" a lot in this guide so it is important you have a clear understanding of the terminology.High frame rates are a good thing since the higher the frame rate, the more smooth the game play will feel (and the bigger man you will be, kidding!) - Though over a certain amount of FPS, the human eye can technically not notice any difference. Although, if you have a game which has extreme varying high and low frame rates, the game will not feel as smooth flowing since the frame rates are not consistent - for some people this creates a sort of motion sickness type effect in the most extreme of cases. The solution is to enable a feature called vertical synchronization or v-sync as it is commonly known.When you enable v-sync (you can enable it for specific games in the game options or you can force enable it globally in your driver control panel for all games) the maximum frame rate the game will render at is whatever your monitor refreshes at. So, I have a Sony 21" G520 CRT monitor and if I had my resolution set at 1024 x 768 with a refresh Hz rates of 100Hz, the game will not exceed 100 FPS. Many people are either for or against using v-sync - you will need to decide what is best for you with your specific computer system and gaming requirements, trial and error basically.
Most websites who review hardware will disable v-sync globally as a rule of thumb so frame rate performance is not limited to the monitors refresh rate. In all of our benchmarks in this guide, v-sync will be turned off so we can get an accurate guide on pure performance numbers. In Doom 3 you can either disable v-sync in your driver control panel in Windows or disable it in the advanced graphics options in the game (as you can see in the screenshot above), however, by default v-sync is turned off in Doom 3.
Doom 3 Benchmarking Guide - How to benchmark Doom 3
How to benchmark Doom 3 with timedemosNow you have an understanding of frame rates and the importance behind them, we are ready to learn how to benchmark Doom 3 on your system with the built-in timedemo feature.- All about timedemosA timedemo is a way of comparing frame rates from point A to point B in any given game which includes a timedemo feature. Some games come with pre-recorded timedemos from the game developer which you can use (Doom 3 comes with a timedemo called demo1.demo which is quite good) or you can record your own to compare performance from one system to another.The method of creating your own timedemo rather than using a popular publicly available one can be considered a more reliable method as in the past some companies have been known to optimize their drivers for certain timedemos which some consider cheating.The timedemos job is to measure the amount of time it takes to render a certain amount of frames - usually at least 3000 frames in total to give a fair idea of system performance - preferably more though. The quicker the timedemo completes, the quicker the system, as it was able to render the most amounts of frames per second. As a rule of thumb, the frames per second in a timedemo are averaged out over the entire demo to give an overall idea of performance.When creating a timedemo, the idea is to create a demo which covers all aspects of game play - running around, jumping, crouching, ducking, shooting bad guys and so on - simulate what you would do, as if you were actually playing the game. This is the best way to create a realistic timedemo which will provide accurate results - for example, it would be no good walking around in a circle inactive as a game like Doom 3 is not usually like this in reality when you are playing through the levels. You usually engage in zombie shooting which is quite intensive and puts stress on the entire system, mostly the CPU, memory and graphics card.- Creating a timedemoCreating a timedemo is quite easy in Doom 3 and is basically the same as creating one in Quake 3 and some other games by ID Software - even the console looks just about the same. Before we get to the timedemo recording, we need to modify the DoomConfig.cfg file to make for easier usage a little down the track.First of all you will need to open the DoomConfig.cfg file in notepad or something similar, the file is located in your /base/ root folder - for example: C:\Doom 3\base\DoomConfig.cfg
Once you have the DoomConfig.cfg file open, you will need to add the following two commands:bind "," "recorddemo"bind "." "stoprecording"These commands tell Doom 3 to begin recording a timedemo when "," key is pushed and to stop recording when the "." key is pushed - Trust me, this method is much easier than pulling down and console and typing in the commands each time. As far as I know, there is no limit to the size of the timedemo you create but be aware a 3 minute timedemo creates a 150MB or thereabout file in your Doom 3 demos folder. Make sure you have enough hard drive space if you want to create super long timedemos.If for whatever reason you want to type in the commands manually, you'll need to pull down the console to do so by pressing CTRL + ALT + ~. If this is too much work for you, you can avoid this by adding the following command to the DoomConfig.cfg file:seta com_allowConsole 1This command tells Doom 3 to allow you to access the console by just pressing ~ instead of the rest of the keys like by default.- IMPORTANT: You'll find a complete set of Doom 3 console commands and cheats in our Gaming forum, here.Now we are ready to create the actual timedemo. Load up Doom 3 and fire up a single player level - any will do. When you feel you are at a good point to begin recording your timedemo, press the "," key once only - the timedemo is now recording. Move around actively and shot bad guys and after period of about 2 or 3 minutes (or more if you like) and then press the "." key to stop the timedemo from recording. Now to check that the demo actually was recorded, pull down the console and you should see a message saying "stopped recoding demos/demo001.demo" - your timedemo is now complete and ready to be used, as you can see in the screenshot below:
Okay, now we are ready to execute the timedemo and measure your system performance. For the most accurate results, I suggest you close Doom 3 after each timedemo test as it clears the system memory from previous textures and so forth. Once you have a fresh load of Doom 3 ready, pull down the console at the start screen (you don't have to be in the game) and type "timedemo demo002" or whatever your demo is called, as you can see in the screenshot below:
Once you press enter the timedemo will load, depending on the system of your system and most importantly hard disk drive, it will vary on the time it takes to begin running. Once the timedemo is finished you will be given the average FPS score along with some other details, as you can see in the screenshot below:
As you can see from the screenshot above, our custom timedemo rendered a total of 5456 frames in 44.2 seconds which gave us an average FPS score of 123.6 frames per second. If you want to see the frame rate as the timedemo runs, enable the Doom 3 FPS counter by pulling down the console and typing "com_showfps 1" - If you want to disable the FPS counter in the future, just type in "com_showfps 0" (without the speech marks each time).It is very important to keep in mind that when you are running a timedemo your system is actually doing a little less work than it would if you were actually playing the game. When you record a timedemo it also records the work the CPU and graphics card would normally do in actual game play - such as AI. This means when you run a timedemo your system (read: CPU) is not being as stressed as much as it would but luckily this doesn't affect the scores too much - it's just good to keep in mind. We also noted that when you run timedemos, the animated screens in the game are not present (as you can see in the screenshot below).
That covers the basics of benchmarking Doom 3, the following pages will discuss the in-depth issues of benchmarking Doom 3 such as with FRAPS, different image quality setting effects, capped internal tick and more.
In-depth methods of benchmarking Doom 3You've now learnt how to create a timedemo in Doom 3 and measure the performance of your system - that was the easy part. From now on, things get more technical, so read carefully. There are a few issues with Doom 3 benchmarking that we must address then we will talk more about the in-depth benchmarking methods available to us.- 60 FPS internal lockWhile reading through initial online Doom 3 benchmarks before the game was released, it became obvious that the game included an "internal tick" or FPS lock of 60 FPS which was failed to be mentioned. This means that when you play Doom 3, the most FPS you'll ever obtain is 60 FPS no matter what hardware you are using.As far as we can gather from our talks with ATI and nVidia over the past few days (ID did not respond to us), the 60 FPS lock is in place to create a smoother gaming experience by having a better chance at producing more consistent frame rates. John Carmack, the head-honcho developer at ID Software suggests another reason:"The game tic simulation, including player movement, runs at 60hz, so if it rendered any faster, it would just be rendering identical frames. A fixed tic rate removes issues like Quake 3 had, where some jumps could only be made at certain framerates. In Doom, the same player inputs will produce the same motions, no matter what the framerate is."Some are calling it an "anti-cheat device" around the forums but we aren't here to decide the good and bad reasons for including the lock - the fact is, it's there and at this stage it cannot be removed.From a real-world standpoint, it is important people are aware of the 60 FPS lock when it comes to benchmarking with Doom 3. If we cannot play Doom 3 above 60 FPS, why should any score over this be considered important? That's one of the questions which plagued our minds for the past few days. Even with the 60 FPS lock, we consider scores over this important as Doom 3 is a good graphics card benchmark (as you will soon see in our own benchmarks, CPU doesn't play much of a roll) and scores over 60 FPS will provide a good way of telling how your system will perform as you ramp up Image Quality settings such as AA and AF.The important thing to remember is that at this stage, until ID Software let us know how to disable the FPS lock or release a patch to do so (and that's IF they do either!), your maximum game play FPS in Doom 3 is 60 FPS. At the end of the day I would have preferred an option to disable the FPS lock as choice is always good but ID obviously decided otherwise.The good news is that when you initiate a timedemo in Doom 3, the FPS cap is non-existent and you are able to measure performance just like any uncapped game which means Doom 3 can be used as a true benchmark for review websites and gamers. Initially we thought that you wouldn't be able to but the simple idea of a timedemo is to time the demo and it does exactly just that.
In-depth methods of benchmarking Doom 3 Continued- Image Quality Settings ExplainedWhen it comes to actually benchmarking Doom 3 with your timedemo, you need to make a few decisions regarding image quality. In the system options section in the game you have the option of automatically detecting optimal video quality based on a scan of your system.Doom 3 lets you choose four preset video quality settings: Low Quality, Medium Quality, High Quality and Ultra Quality. Low Quality is for older systems with 64MB graphics cards. Medium Quality is for slightly newer systems with 128MB graphics cards. High Quality is our weapon of choice at the moment which is designed for current generation graphics cards (X800 and 6800 series) with 256MB of memory, 8X anisotropic filtering is also enabled by default. High Quality is the big fish designed for future graphics cards to be released over the following months with 512MB of memory onboard and compresses none of the textures providing the ultimate Doom 3 experience.To give you an idea, our AMD Athlon 64 3400+ on reference nForce3 250GB motherboard with 1GB of DDR-400 memory with relaxed timings and either GeForce 6800 Ultra or Radeon X800XT PE with Windows XP SP1 and DX9c suggested we should use High Quality at 1024 x 768. ID has obviously been cautious (probably over-cautious) in their recommendations as the system above very easily handles the suggested video settings and in fact we could run at 1600 x 1200 with 4X AA and 8x AF enabled which scored us an average just over 50 FPS which is playable indeed - the recommended video settings scored us around 130 FPS on average. We could even use Ultra Quality video settings but you do notice more game pauses than usual every now and again as massive 500MB + uncompressed textures are drawn.
From the findings above, we can tell that when you are trying to stress out Doom 3 with those who have a modern system, you will need to ramp up the image quality settings as high as possible - more so than what the built-in detection device suggests. We suggest at least a resolution of 1024 x 768 with 4X AA and 8X AF enabled in High Quality mode. Play around with the settings and work out what is best for your system. You can adjust the AA settings in the advanced options section of the system options between 2X, 4X, 8X and 16X or you can force on AA in the driver control panel of your video card in Windows.- IMPORTANT: For a full run down on Antialiasing (AA) and Anisotropic Filtering (AF) please read our "The Simple Antialiasing and Anisotropic Guide", here.If you really want to stress out your system you would choose 1600 x 1200 with 4X AA and 8X AF (or higher) with Ultra Quality. Apart from putting massive strain on all current systems, this should prove as a good method of stress testing your system and particularly your graphics card and stability when overclocking since Doom 3 is so advanced and uses many parts of your graphics cards which games in the past haven't needed to.Idealism would tell us the best thing is to be able to stress your system to the point that you stay under the 60 FPS lock all the time but this will become harder and harder as new and faster system components are released over the following months which let you pass that mark even with very high image quality settings enabled. Let's hope ID gives in soon and tells us how to unlock the 60 FPS lock ;)- Game Pausing with Massive TexturingAnother facet of Doom 3 with current day hardware (any) is the massive texturing the game includes. If you are running High Quality or Ultra Quality video settings (more so the latter) you will notice the game will pause now and again for almost one second. This is due to the game having to draw massive amounts of textures, some above 500MB. You will notice this when you're about to enter an area with complex textures or a bunch of zombies or even the moment before a cut scene (which are drawn in real time in Doom 3).During the first timedemo run of Doom 3 when textures are not in memory (e.g., when you first load up the game from your OS) it is always a good deal slower then the second and third (and so on) timedemo runs when the large textures are already in memory. The reason behind this is quite simple - even if the game pauses to load a texture, the timedemo continues counting and during the pause registers as 0 frames per second and consequently reduces your score, sometimes with a big hit. Our suggestion is to always run three times and then average out for the most accurate and realistic results.One question which has already popped up around forums is if the timedemo runs are consistent - forgetting runs #1 and #2, we've always been within one FPS on the third and forth runs which is great since we can put more faith in the benchmarks. More good news is textures clear out of memory each time you close Doom 3 so you won't have to worry about those monster-like textures sticking in the memory and slowing down your usual OS activities.It would be safe to assume that once 512MB graphics cards start hitting the market (we've been told toward the end of this year and Q1 2005) from ATI and nVidia and so on the pausing which occurs now and again during game play should be gone or at least very minimal.
Doom 3 Benchmarking Guide - Benchmarking with FRAPS
Benchmarking with FRAPS (more precision)We've covered the basics of benchmarking Doom 3 with the built-in timedemo feature and the issues to watch out for but we can go one set further and use a very handy and popular Australian made program called FRAPS (download it here) to measure system performance.FRAPS is a popular program used by ATI and nVidia and many gamers around the world to measure in detail the FPS of both DX and OpenGL games which don't have a timedemo feature. You can also capture screenshots and record a video of your game play to help improve your skills or just show your friends.In our case, we'll be using FRAPS to provide a more precise idea of benchmarking with Doom 3 - in particular: minimum, maximum and average FPS scores. The built-in Doom 3 timedemo feature only gives you an average FPS score at the end. FRAPS is able to provide a whole bunch of information on the timedemo run. It's important we know the minimum and maximum FPS scores because even if you have an average FPS of say 50, you don't want to have too many areas where the FPS drops into the single digit range as it will make the game less enjoyable.Before we get onto using FRAPS, it's important to note that you will still need to create a timedemo when using FRAPS. If not, you will be capped at 60 FPS and the results will not provide what we are looking for to the fullest extent.Once you have downloaded and installed FRAPS from their website, open the program and click on the FPS tab at the top - this is the section we are most interested in today. By default, you press F11 in game to let FRAPS know that you want to begin testing. Once you press F11 the testing will begin and once you are finished you need to hit F11 again to stop it. You'll also want to click on the "Save detailed benchmark statistics" tick box to enable minimum and maximum FPS scores, as you can see below:
You can enable the FPS overlay display counter which will display in a yellow font in any corner of the screen you choose. This is not as important as you can just enable the Doom 3 FPS counter by pulling down the console and typing "com_showfps 1" - If you want to disable the FPS counter, just type in "com_showfps 0", without the speech marks each time.Once the timedemo is finished, locate the folder where you installed FRAPS and open the benchmarks folder. This folder contains all of your detailed benchmark results.
Doom 3 Benchmarking Guide - Our own benchmarks
Our own benchmarksNow that we are feeling all empowered with the fact we know exactly how to benchmark Doom 3, now would be a good time to show you some of our own benchmarks we ran with our own custom timedemo. Before we get into the timedemo, here are a few screenshots from Mars City Underground which played host as our timedemo level.
Now you have an idea of what is in our timedemo, let's take a look at our Test System Setup.Test System SystemProcessor(s): AMD Athlon 64 3400+Motherboard(s): nVidia nForce3 250GB reference (Supplied by nVidia)Memory(s): 2x 512MB Mushkin DDR-500 (2.5 - 3 - 8 - 3 timings) (Supplied by Mushkin)Video Card(s): Gigabyte Radeon X800XT PE 256MB (Supplied by Gigabyte)Hard Disk(s): Western Digital Raptor 36GB 10,000 RPM (Supplied by Western Digital)Operating System Used: Windows XP Professional XP1Drivers Used: ATI Catalyst 4.7 and 4.9 BETA (Overdrive disabled) and DX9cOriginally we had good intentions of comparing the GeForce 6800 Ultra against the Radeon X800XT PE but just as we began to benchmark with Doom 3 our reference GeForce 6800 Ultra decided it didn't want to work properly. Despite efforts to get the graphics card working, we had no luck.Instead we'll be comparing ATI's current official Catalyst 4.7 drivers against the BETA 4.9 drivers which ATI released this week which promises better Doom 3 performance - ATI is providing an early version of CATALYST 4.9 to users interested in increasing performance of Doom 3. These drivers will be officially released in roughly 2 months time after the official 4.8 drivers become available for download this month.As suggested throughout this guide, we ran the timedemos three times at each resolution and image quality setting and then averaged the scores. We used FRAPS to gather our results as seen on page 6 of this guide. We choose to benchmark with HQ settings as high-end graphics card users will want the best available (Ultra Quality really needs 512MB graphics card for the best experience) IQ and not lower. As mentioned in this guide, High Quality video settings automatically enable 8X AF and then with 4X AA and 8X AF we forced these settings in driver control panel.Quick FSB Overclocking Performance InvestigationJust before we present you with our benchmark graphs, we want to make note of one important factor. We overclocked our AMD Athlon 64 3400+ processor 100MHz and 200MHz over default via FSB and at both overclocked settings our average FPS at 1024 x 768 0X AA / 8X AF did not increase over one frame per second.This tells us that Doom 3 is very graphics card dependent and because of this should soon prove to be a popular benchmark in graphics cards review from many review websites. We'll certainly be adding it to our graphics card benchmark suites straight away.Custom Timedemo Benchmark ResultsAlright, now we are ready to check out the benchmarks we took on our test system.
In our first test, you can see that our test system handles 800 x 600 with ease. At this resolution the lowest the frame rate hit was 26 FPS which is workable but thankfully the averages were very high and more than enough to provide a smooth game play experience.Here we can see that the new Catalyst 4.9 BETA drivers provide around a 13% increase in performance.
At 1024 x 768 again our test system more than handles the task. Here our lowest frame rate got down to 18 FPS which isn't all that pretty but again the average FPS was quite high and again provided a good gaming experience.Here we see almost an 18% increase in performance with the new ATI BETA drivers.
In our final test at 1600 x 1200 the average FPS is a little lower. Even with 4X AA and 8X AF image quality settings, we have enough average frames per second to provide a fairly solid gaming experience. We did get down into single digit frame rates for the minimum which was not pretty.As with the 800 x 600 resolution, we see around about a 13% increase in performance from the new drivers. Overall we have about a 15% increase in performance which is quite good and will help ATI take back a little ground against nVidia and their flagship 6800 Ultra which performs really well under Doom 3.
Doom 3 Benchmarking Guide - Final Thoughts
Final ThoughtsOver 5000 words later, we have approached the end of our definitive guide to benchmarking with Doom 3. With all the information provided in this benchmarking guide, you should be well and truly empowered enough to create your own timedemos in Doom 3 and then execute them and measure your system performance in detail.We are happy ID Software once again decided to include a built-in timedemo feature for gamers and review websites. We think Doom 3 will quickly become one of the key benchmark standards in graphics card reviews on the web as we proved in our testing where increases in CPU clock speeds of 100MHz - 200MHz make little difference in performance. For this very reason, we'll be sure to begin including Doom 3 benchmarks in all of our future graphics card reviews very soon.
It's very important you read this entire guide as it includes crucial bits of information throughout such as the massive texture pausing, various image quality settings and the internal 60 FPS lock. I've spent the last few days investigating and researching benchmarking with Doom 3 and now it's time to hold the benchmarking and actually get some Doom 3 game play in - don't you forget to do the same either!Finally if you wish to discuss this guide or any topics related to benchmarking with Doom 3 or just general issues relating to the game, please direct your comments to our gaming forums where we can answer your questions. Most questions sent via e-mail will not be answered.Happy benchmarking and gaming!
PRICING: You can find products similar to this one for sale below.
United States: Find other tech and computer products like this over at Amazon.com
United Kingdom: Find other tech and computer products like this over at Amazon.co.uk
Australia: Find other tech and computer products like this over at Amazon.com.au
Canada: Find other tech and computer products like this over at Amazon.ca
Deutschland: Finde andere Technik- und Computerprodukte wie dieses auf Amazon.de
Cameron founded TweakTown in 1999 after it originally started off as his personal homepage. Cameron was once, many years ago, the only person at TweakTown producing content, but nowadays, he spends his time ensuring TweakTown operates at its best in his senior management role.
We openly invite the companies who provide us with review samples / who are mentioned or discussed to express their opinion. If any company representative wishes to respond, we will publish the response here. Please contact us if you wish to respond.