PCMark For Android Wants To Make Benchmarks Reliable Again
Tech sites and techies are suckers for benchmark tests, despite a few scandals that rocked that scene a few years back. While not to be taken as absolute truth, these suite of performance tests and numbers do at least give us some reference points to talk about. But there are times when benchmarks don't exactly match real-world performance. Which is why in the updated PCMark for Android app, developers Futuremark claim to use even more real-world tasks to measure your Android device's actual capabilities.
There are two schools of thought in the benchmarking community. One holds that tests should push hardware to the limit, demonstrating the device's full potential. Others, however, eschew that ideal situation and instead believe that benchmarks should measure how a device would perform under real-world conditions, performing real-world tasks. Futuremark is of the latter group, which is why the updated Android benchmark tool is getting more such real-world tests.
The app's new Work 2.0 battery of tests focuses on what users commonly do with their smartphones and tablets, like web browsing, document creation, and photo editing. It uses the same API that apps for those tasks use, ensuring they mirror reality as much as possible. Futuremark has also added two new tests to reflect growing trends in mobile computing: editing videos and gathering data from sensors, like a fitness tracker.
Some users do employ their devices for more esoteric tasks like machine learning and computer vision. That might become even more common in the near future as smartphone cameras and processors become even more capable. Preparing for that future, PCMark for Android now includes Computer Vision tests utilizing the same APIs that apps of this kind employ.
And while benchmark scores are nice to look at, they are really meaningless when taken in isolation. They only become significant when you compare them with others. In the latest app update, PCMark will let users not just compare against other smartphones but also compare against other Android versions. The reasoning here is that newer versions usually, but not always, offer performance improvements that might be nice to know.
Benchmarks, especially those performed by normal users outside of controlled laboratory settings, can never really be accurate measures of real-word usage and performance. But, for developers like Futuremark, the better you can approximate those tasks and conditions, the closer to reality you can get.
SOURCE: Futuremark