Qualcomm Snapdragon 865 Benchmarks - And The Problem With Them

The Snapdragon 865 is Qualcomm's latest-and-greatest chipset, set to be the 5G heart of most 2020 Android flagships, and unsurprisingly it blows right past its predecessor when it comes to benchmarks. I'm also here to tell you that, having sat with a Snapdragon 865 reference device and watched it clock up sky-high scores in apps like Antutu, Geekbench, and GFXBench, frankly you almost certainly shouldn't care about those numbers.

Over time I've become less enamored by benchmarks, and I know I'm not alone. Certainly, on the face of it, they serve a valuable purpose. It's our natural inclination to want to know whether one thing is "better" than another, and a numerical score on things like processor or graphics performance seems like a great place to start that filtering.

The problem, though, is not so much one of "the map is not the territory" but that nobody quite uses the map – or indeed the territory – in the same way. There are now dozens of different mainstream benchmarking tools, and that's before you get to niche apps and custom tools. Each puts a slightly different emphasis on what the perfect combination of talents and abilities should be.

In the "old days" it was a matter of running single-core and multi-core CPU stress tests. Then GPUs became more important, so things like frame rates in graphically-intense games started to be included. Now, though, the jobs we ask of CPU, GPU, and AI co-processors are comically varied. You do not use your smartphone in the same way that I do, and so the priorities you have – and the demands you place on its hardware – could be extremely different.

If you're a keen photographer, for example, you might be most interested in how quickly a camera app can use scene detection AI to adjust its settings. Someone else might put more emphasis on how well a chipset renders graphics using Vulkan. Then there are elements that modern benchmarking tools typically don't even try to test, like wake-word responsiveness, multimedia, and wireless management.

If benchmarking is so limited, then why bother doing it? That's an excellent question, and I won't argue with the fact that, for many people, having an idea of where a chipset falls in the speed rankings is something worth knowing.

The important thing to remember is that the further you delve into the depths of a chipset's abilities, the more caveats and explanations are required. For a start, some benchmarking tools aren't using the very latest models for things like AI, or they're using out-of-date models that the software world has moved on from. Others put too much emphasis on models that have little likelihood of being used on a smartphone-style device: the apps end-users are going to run are never going to demand them.

A single number alone, therefore, is basically irrelevant unless it's for something as top-level as how a CPU clocks in when you're running Geekbench. As soon as you get into the weeds with AI models and things like image classification, a number in isolation isn't sufficient. You need to explain the context of that result so that someone will know how it might impact how their app or task of choice will run.

Then there's the platform itself. Is the Snapdragon 865 "better" than, say, the Apple A13 Bionic chipset inside the latest iPhone 11 Pro? If it's "faster" in terms of raw CPU and GPU performance, maybe, how well does iOS implement that compared to Android? How readily can third-party software developers tap into it?

Run a single AI test, and you end up with a limited view of the overall usefulness of a chipset. Run a compound test, with multiple AI models, and maybe you're getting a broader view of the overall scene: however you also have to decide how you're weighting the individual scores as you assign an overall result.

"I hear you, Chris," you may be thinking, "but I still want the numbers." That's okay. I have numbers for you too.

Qualcomm's reference design has a 5.9-inch display running at 2880 x 1440 resolution, and pairs the Snapdragon 865 with 12GB of LP5 dual rank memory. There's 128 GB of storage.

In Geekbench 5.0.2, Qualcomm's latest-and-greatest scored 932 in the single-core test and 3,433 in the multi-core test. By way of comparison, the iPhone 11 Pro Max scored 1,321 in the single-core test and 3,287 in the multi-core test.

For AnTuTu 8.0.4, the reference phone scored 183,425 in the CPU test; 220,598 in the GPU test; 81,255 in the memory test; and 81,868 in the UX test. Its overall score was 576,146. On the iPhone 11 Pro Max, the scores came in at 163,747 in the CPU test; 206,860 in the GPU test; 73,297 in the memory test; and 73,702 in the UX test. Overall score was 517,606.

In AITuTu 1.1.1, the Snapdragon 865 came in with an overall score of 470,705. It scored 265,459 in the Image Classification subcategory; and 205,246 in the Object Detection Category.

Then there's GFxBench 5.0. There, the Snadpragon 865 managed 124 fps in the 1080p Manhattan Offscreen (ES 3.0) test, and 88 fps in the 1080p Manhattan 3.1 Offscreen (ES 3.1) test. It scored 20 fps in both the 1440p Axtec Ruins Vulkan (High Tier) Offscreen and OpenGL (High Tier) Offscreen tests. It scored 206 fps in the 1080p T-Rex Offscreen (ES 2.0) test.

Finally, in the AndroBench 5.0.1 storage benchmarking tool, the Snapdragon 865 clocked in with 1,451 MB/s sequential read and 224 MB/s sequential write. Random read was 187 MB/s, while random write was 170 MB/s.

On top of all the considerations I've already mentioned, here's something else to bear in mind. Each of my tests was run with the Snapdragon 865 reference design in "Performance Mode": effectively that's a way to make any particular workload look 20-percent more intense to the chipset's scheduler, thus making sure it'll always give it to the most potent core. It's not the same as overclocking – Performance Mode won't make each core actually run any faster, just ensure the Snapdragon 865 is always throwing its beefiest silicon at the task – but it does cast another wrench into the benchmarking works.

You see, no two models of phone will have quite the same Snapdragon 865 tuning. Manufacturers can choose how they balance the performance of the chipset with its demands on the battery. You'll be unsurprised to hear that, when pushed toward maximum potential performance, Qualcomm's chip is more power-hungry. In the grand decision of potency-versus-longevity, some phone-makers are going to err on the side of the latter.

I have zero doubt that this is the speediest silicon that Qualcomm has developed so far. The Snapdragon 865 will undoubtedly be the chipset to have in a flagship Android phone come 2020, when we see the first examples of it show up in devices from manufacturers like Samsung, Google, OnePlus, and others. Qualcomm's decision to make 5G a standard part – even if the modem itself isn't on-die – is also a significant one.

What raw numbers can't do, however, is tell you why the Snapdragon 865 is special. For that we'll have to wait for software examples, both from OEMs and from third-party apps. The good news is that the raw grunt is certainly there to deliver all those AI-powered features and experiences. It's just that benchmarking alone is a two-dimensional way to get a taste of what they might be.