Introduction
In the realm of benchmarking one name stands out as THE company to go to for your benchmark. That company is Futuremark. Since the first release of their 3DMark test they have been the ones to turn to when it comes to benchmarking not only your GPU, but in many cases your entire system.
But things have not always been pizza and beer for the gang at Futuremark. They have had their share of criticism levels at them. In Futuremark's Vantage they chose to go all DX10 and also to include the Ageia PhysX API. This was perfectly fine until NVIDIA bought the struggling company and enabled the API to run on their GPUs. This gave every NVIDIA GPU an unfair advantage in many people's eyes.
The fact that Futuremark put that in there before NVIDIA bought Ageia escaped most people. Now Futuremark is back at it with a new Graphics test; 3DMark 11. This test changes a few things and once again will surely get Futuremark in trouble with both of the big GPU houses.
What's New?
What's New
One of the first things that is new about 3DMark 11 is of course the use of DX11 (Tessellation, Compute Shaders and Multi-threading) for the testing suite. But there are other less obvious things that are new as well. For the first time 3DMark is shipping in multiple languages. You can install the new test in English, Finish, German and both Simplified and Traditional Chinese.
This opens up the market even more than it did before (especially in China). We also see the return to the Free edition of 3DMark with unlimited testing runs. This item alone will make a number of enthusiasts happy. The Advanced Edition will set you back $19.95 while the Professional Edition will run you a whopping $995!
Languages and money aside, there are other differences that are both cosmetic and functional. The flow of the application has changed (for the better if you ask me). It is a tabbed style (like most web browsers) with each tab containing different functions and providing more depth to the application.
The Basic tab is what you will get if you are only looking at the basic "free" edition of the benchmark. On this tab you have everything you need to run 3DMark 11; you just have very little control over what you are running.
The Advanced tab gets you quite a bit more control over the tests, the resolutions, anisotropic filtering and even a loop mode (which we use) to put the GPU in question under more stress.
The Professional mode adds something that we have been waiting to see for a very long time. This is a built in Image Quality test that allows you to do a direct frame by frame comparison. Unfortunately it does not have a movie mode; instead it captures one frame at a time and even when you set it up for a range of frames.
The good news is that you can take these frames and put them together using Windows Movie maker or any animation program. Of course, the down side to that is Windows Movie maker will reduce the quality of the still images a little as it converts them to video.
One of the last items that is new (or returned to) is the Demo Mode. This is simply a pair of rendered scenes that are fun to watch and can also help to "warm up" your GPU.
As with all versions of 3DMark Since 06, you can setup a personal account to upload, compare and share your scores with other users on Futuremark's website.
One item that I was not happy with was the removal of the detailed scores from the application. To get detailed scores you have to submit your results and then view them online. While I understand the move, I would have liked to see the detailed scores available in the Advanced and Professional editions.
The New Tests
As with every version of 3DMark, each test is designed to cover different aspects of gaming. The four Graphics tests are broken up into render types as we see listed below.
As you can see, the first test has no Tessellation (something that is big with DX11) but the other three have some when it comes to the rendering of items in the scene. Volumetric lighting is big in all of the tests, as well as post processing for lens effects (film grain etc). Depth of field also plays a big role in the test runs along with particle effects.
The Physics testing (not PhysX) is handled by Bullet this time instead of the Ageia/NVIDIA API. This is a welcomed surprise to AMD fans as it removes the seeming advantage that NV had in the past. The first test is all CPU based with the CPU handling rigid body testing physics at a fixed frame rate. The second test throws the GPU to handle volumetric lighting, and Direct Compute Physics.
To cover the details of the tests, test one puts you in an underwater scene. You see deep sea submersibles moving through the dark waters; their lights the only source of illumination.
In each scene they have included rusted surfaces that are barnacle encrusted to add additional shadows (and tessellation in the second test). These scenes are reminiscent of the movie "The Abyss". In fact, in the longer demo one of the undersea wrecks they find looks a lot like the deep sea drilling lab from that movie.
In test three and four we move to a jungle where we happen upon a deserted campsite. There is lots of vegetation moving in the wind as well as rounded surfaces (for that, Tessellation). The sun shining through the jungle offers a great opportunity for volumetric lighting while we can see some limited particle effects in the light.
The CPU Physics testing is a much like we saw in 3DMark Vantage. It is the same image group rendered multiple times and performing the same action (although each does react differently). The image groups are made up of a series of columns covered by stone slabs. A large stone ball is dropped onto them and they fall apart. This puts the CPU under the strain of calculating each impact and the trajectory of the falling items.
The combined test is something of a close-up with more detail. 3DMark has added in some vegetation, lighting and filtering while the CPU still has to contend with the falling balls of stone and what to do when they hit the columns.
Each of these tests is much shorter in length than in any of the previous versions of 3DMark. We have been told this is to get the results to you faster; this should make more than a few enthusiasts happy along with a few members of the press that use this test.
Performance
Performance Testing
Our GPU selection was a little light at the time of this writing, as was our CPU availability. However, we were able to get you a few current and popular GPUs to show you how the numbers fall.
We would like to thank the following companies for supplying and supporting us with our test system hardware and equipment: Intel, GIGABYTE, AMD, Kingston and Cooler Master and Sceptre.
As you can see below, the performance numbers come in very interesting. We see NVIDIA on top for the most part, but that PhysX advantage is certainly gone in the combined scores. This is something that we know Futuremark has been working on since NV bought Ageia as it put them in a very odd position.
We talked very briefly with them about the new test and they said that they knew they had the right balance with this one when both NVIDIA and AMD were not happy with them. We can see some of that below.
Once again, because of the pre-release nature of our testing we do not have detailed scores. We will be revisiting this test after today and comparing the detailed scores for you along with more platforms and CPUs very shortly.
Scoring
Now, for the thing that keeps some people up all night; the scoring methodology. Back when 3DMark Vantage came out and I published the scoring calculations, I received many e-mails telling me that the calculations were wrong, or unbalanced, or this or that.
It was as entertaining as it was annoying. First, I did not come up with these, nor did I compare and analyse the way the formulas are built. I am presenting them again for you to get a "feel" for how the system totals up that final score and once we can publish ours to the net, we will see how they stack up.
Final Thoughts
Although our time with 3DMark 11 was limited to a couple of days, we still have to say we rather like the new benchmark. It is much cleaner than any of the previous versions and also adds in more than one feature that we have been waiting for (IQ testing).
Futuremark is in an unenviable position. They have the job of building a test that is balanced and fair across the board. In a perfect world everyone would be happy with this. However, we do not live in that world. We live in one where we take sides and polarize everything. This new test will annoy AMD users as much as it will NVIDIA owners.
To me, if you can piss off both houses then you have probably got it right. We think that while no synthetic will ever be a replacement for real usage, Futuremark does an excellent job of combining the multiple parts of gaming into one test that can give you a good solid feel for how your system will perform under the stress of modern gaming.