Huawei has recently been caught optimizing some of its top smartphones to over perform on benchmark tests. On Tuesday, AnandTech discovered that Huawei’s P20 had been programmed to maximize performance specifically when running 3DMark, a popular benchmarking app. Today, the company behind 3DMark followed up with a statement saying that it had confirmed the findings and would delist the P20, as well as three other Huawei phones with similar behavior, from its benchmark leaderboards.
The phones include the P20, P20 Pro, Nova 3, and Honor Play. Huawei admitted to this behavior in a statement given to Android Authority, saying that its phones are designed to adjust their performance based on the app that’s running.
But the way that Huawei implemented that behavior isn’t allowed. While phones can adjust their performance as part of their typical behavior under high workloads, they can’t be hard coded to maximize their behavior just because a specific benchmark app is running. That’s what Huawei seems to have done, according to UL, which is behind the 3DMark software.
When UL ran an internal version of 3DMark, which Huawei’s phones couldn’t recognize the name of, the phones performed worse in the test. That indicated that the phones weren’t actually smart enough to identify high performance demands on their own, which meant the benchmark score wasn’t an accurate reflection of how the phone would handle a typical app without special attention from Huawei.
3DMark has removed these phones’ rankings from its leaderboard and adorned their listings on its website with a note that the phone’s “manufacturer has not complied with UL benchmark rules.” Many of their results have been removed as well.
Huawei is far from the first company to get caught toying with benchmark results. Samsung got busted for the same behavior on its flagship phones in 2013, and just last year, OnePlus was found to have done the same. This timing is particularly unfortunate for Huawei, since just weeks ago it got caught trying to pass off a DSLR photo as a photo from one of its phones.
Huawei has also admit in its statement, that it “always prioritizes the user experience rather than pursuing high benchmark scores especially since there isn’t a direct connection between smartphone benchmarks and user experiences.” And yet, it still coded its phone to deliver higher performance when running a benchmark test.
But Huawei also claims that its phones’ include AI that’s smart enough to optimize performance based on whatever app is running, and clearly that’s not the case.
But if it was true then the company would be boosting performance for benchmark. The one thing that comes in mind is that are only the listed phones have been boosted or all of its phones are well we will found it out very soon.
