Cheating on benchmarks is nothing new but it’s still not tolerated. It should never be because the consumers are the ones being lied to. In the past, we noted some companies like Huawei benchmark cheating with the Honor Play and HTC One M8 getting booted from 3DMark for cheating. We also remember Samsung cheating with benchmark results and OPPO phones being delisted from UL benchmark for cheating. A few years ago, there’s also OnePlus caught ‘benchmark cheating’ with the OnePlus 3T.
Now MediaTek is in hot water for doing the same thing. Mobile benchmark cheating is a real problem but OEMs aren’t stopping. They may be hiding the activities better this time.
As for MediaTek, the Helio P95 processor was found to be outperforming Dimensity 1000L. The latter is supposed to be more advanced but the old Helio chipset outperforms the new processor used on the OPPO Reno 3 Pro.
The results are odd and so guys at AnandTech took notice. They tested the phone on different versions of benchmarks and discovered the scores differ. According to them, there was a 75% difference in subtest and a 30% difference in the overall score
PCMark has been chosen because it is a performance and battery benchmark all-in-one. It is said to be representative of real workloads and responsiveness of a mobile device. There are many factors that affect scores from mechanisms to software aside from hardware.
MediaTek is believed to be offering mobile benchmark cheating as service just right under our noses. Simply put, a phone running a MediaTek processor shows power management hints on the code for certain benchmarks. A configuration like “Sports Mode” has been sighted.
MediaTek already published an explanation on its website. It’s an official response to Anandtech. Read on:
MediaTek follows accepted industry standards and is confident that benchmarking tests accurately represent the capabilities of our chipsets. We work closely with global device makers when it comes to testing and benchmarking devices powered by our chipsets, but ultimately brands have the flexibility to configure their own devices as they see fit. Many companies design devices to run on the highest possible performance levels when benchmarking tests are running in order to show the full capabilities of the chipset. This reveals what the upper end of performance capabilities are on any given chipset.
Of course, in real-world scenarios there are a multitude of factors that will determine how chipsets perform. MediaTek’s chipsets are designed to optimize power and performance to provide the best user experience possible while maximizing battery life. If someone is running a compute-intensive program like a demanding game, the chipset will intelligently adapt to computing patterns to deliver sustained performance. This means that a user will see different levels of performance from different apps as the chipset dynamically manages the CPU, GPU and memory resources according to the power and performance that is required for a great user experience. Additionally, some brands have different types of modes turned on in different regions so device performance can vary based on regional market requirements.
We believe that showcasing the full capabilities of a chipset in benchmarking tests is in line with the practices of other companies and gives consumers an accurate picture of device performance.
