Designing your smartphone hardware to cheat the benchmark system seems to be the trendy thing to do, but some benchmarks are ensuring that this form of cheating is no-longer a viable method to boost the perceived performance of you device. Today Futuremark announced that it has delisted several phones from its results.
The phones were delisted from rankings because of accusation that they feature hardware-based loopholes that allow them to run faster when benchmarking apps are activated to run test. All in all a total of six smartphones were delisted, including the :HTC One, HTC One Mini, Samsung Galaxy Note 10.1 Exynos 5 Octa, Samsung Galaxy Note 10.1 MSM8974, Samsung Galaxy Note III Exynos 5 Octa, and the Samsung Galaxy Note 10.1 MSM8974. While no mention of what each device did specifically to falsely boost scores, Futuremark did release a statement about the delistings.
"People rely on Futuremark benchmarks to produce accurate and unbiased results. That's why we have clear rules for hardware manufacturers and software developers that specify how a platform can interact with our benchmark software. In simple terms, a device must run our benchmarks without modification as if they were any other application.
When a device is suspected of breaking our rules it is delisted. 3DMark scores from delisted devices should not be used to compare devices. Delisted devices appear unranked, and without scores, at the bottom of the 3DMark Device Channel and the Best Mobile Devices list on our website."
~ Futuremark