Huawei was hit with allegations of cheating on benchmark tests yet again earlier this week and the company has now responded to the fiasco by promising to make the “performance mode” accessible to all users. Huawei’s decision comes after it held “comprehensive discussions” on benchmarking practices with the creators of 3DMark.
A report by Anandtech revealed that the company had optimized its devices to perform better on benchmarks like 3DMark. This prompted 3DMark to delist the Huawei P20, P20 Pro, Nove 3, and Honor Play from its charts. Huawei had apparently coded the devices to detect the software and subsequently ramp up their performance in a way that goes against the 3DMark rules.
A deeper dive revealed that the devices didn’t perform as well on benchmark programs that were not recognized, which meant that they weren’t actually identifying when they needed to perform better. The benchmark figures were based on thermal profile changes.
Identifying benchmarks to trigger app-level optimizations is frowned upon by benchmark developers because it goes against the goal of estimating performance for all apps. There’s nothing wrong with optimizing for a specific app, but targeted benchmark optimizations skews the data towards an unrealistic view, making the benchmark useless as it no longer represent the potential performance for “most apps”.
This happened on PC as early as 2001 with the infamous Quack cheat. A great many companies succumbed to the temptation at some point, including Samsung, but a few more lessons needed to be learned apparently.
Huawei says in a statement that this is due to “an artificial intelligent resource scheduling mechanism,” which is used to optimize resource allocation so that the hardware can show its capabilities to the fullest extent. UL, the makers of 3DMark, are opposed to devices using a “performance mode” by default when they detect a benchmark application. Such modes have to be turned on by the user in the settings.
Saying that it “respects consumers’ right to choose what to do with their devices,” Huawei has decided to open up the performance mode to all users in the EMUI 9.0 update. This will essentially let users force their handsets into a higher mode that could offer higher performance in all use cases. The Android 9 Pie-based EMUI 9.0 update is due to arrive for Huawei and Honor devices in the coming months.
Filed in . Read more about Benchmarks and Huawei. Source: consumer.huawei
