Tuesday, October 01, 2013, 10:22 am PT (01:22 pm ET)
Samsung again caught doping benchmarks for Galaxy Note 3An early review of Samsung's Galaxy Note 3 noted that it "scores really, really well in benchmark tests," but also explained that edge is achieved through some phony doping that kicks in a steroid boost only when running synthetic benchmark apps.
The review, by Ron Amadeo of ArsTechnica, details that Samsung is up to the same shenanigans it was called out on in an earlier report by AnandTech.
Snapdragon 800 vs Apple A7
Without any cheating, Geekbench 3 portrays the Samsung SM-N900P Note 3 as being essentially tied with LG's Optimus G2, with the same 2.3 GHz quad core chip and slightly more RAM (2.38 GB vs LG's 1.38 GB).
The AppleInsider review of Apple's iPhone 5s, running a dual core A7 at half the clock speed (1.3 GHz) and using less than half the RAM (1 GB), reported 64-bit scores that were essentially tied between the two others, and slightly higher in memory performance.
The two Android phones from Samsung and LG can't run 64-bit versions of the GeekBench 3 app because they are based on older technology. Instead, they are designed to scream through 32-bit benchmarks with more cores clocked twice as fast. That generates heat and wastes battery life, necessitating a larger device with a bigger battery.
Samsung cheating to look better than it does
Back in July, Anand Lal Shimpi & Brian Klug noted that versions of Samsung's Galaxy S 4 equipped with the company's Exynos 5 Octa were designed explicitly to report better that real-world scores that were not actually attainable.
Amadeo now points out that Samsung's repeat performance with the Galaxy Note 3 does the same for the company's latest flagship phablet even on the U.S. version powered by Qualcomm's Snapdragon 800 rather than Samsung's own chip.
Samsung is now covering its tracks, making it more difficult for reviewers to access the true CPU clock speed and discern that the company is faking performance.
Simply changing the name of the benchmarking app turned in scores that were between 20 and 50 percent lower, a phenomenon explained by extracting code on the device that revealed Samsung expressly designed the phone to turn in faster that real-world results when running known benchmark tests.
The report noted that Samsung earlier issued an official statement that claimed its "benchmark boosting" on the Galaxy S 4 was actually used in other processor intensive apps such as the web browser, camera and in video playback, not just in benchmarking apps. But that isn't true, ArsTechnica noted.
Instead, according to Samsung's configuration files, "the function is used exclusively for benchmarks, and it seems to cover all the popular ones. There's Geekbench, Quadrant, Antutu, Linpack, GFXBench, and even some of Samsung's own benchmarks," Amadeo wrote.
Samsung also appears to be faking its graphics performance, tipped off by a "suspicious 'frame rate adjustment' string" and the inclusion of GFXBench, an OpenGL ES benchmark targeting GPU performance.
Samsung doubles down on doping
Amadeo also reports that Samsung is now covering its tracks, making it more difficult for reviewers to access the true CPU clock speed and discern that the company is faking performance.
Doped benchmarks don't just show up in reviews; they also appear on benchmark sites as user submitted rankings, making Samsung's falsifying of its performance a case of false advertising. But Samsung has also launched its own benchmark app, branded "MobileBench."
"To measure and monitor system performance, you must install many applications for many different purposes; however, MobileBench provides an all-in-one solution," the company explains.
"MobileBench measures H/W performance, scores user experience, and compares results with other devices," the company says. "Take advantage of your results to promote your products and optimize your devices."
Now that Samsung is heavily invested in the design of its Exynos chip it can only sell on a fraction of its high end handsets, and faces a race to catch up with Apple's entirely different approach to performance (using a more modern, custom designed A7 engine) the leading Android phone maker is now focusing on fooling its customers with phony performance numbers.
On Topic: iPhone
- Despite faster A8, higher pixel count of iPhone 6 Plus drives lower FPS benchmarks
- Apple's iPhone 6 to arrive in China on Oct. 17 after receiving regulatory license [u]
- Going big: a review of Apple's new 4.7" iPhone 6 vs. the 5.5" iPhone 6 Plus
- Apple expected to begin selling iPhone 6 in China on October 10
- Adapting to change: a review of Apple's larger 4.7-inch iPhone 6 vs. the smaller 4-inch iPhone 5s