Qualcomm Snapdragon S4 Benchmarking Part 1

This week we're taking a closer look at how the Qualcomm Snapdragon S4 (Krait) processor works as it exists first on the MDP8690, the newest Mobile Development Platform that the manufacturer has released for developers to fully experience the S4 chip in its Krait configuration. This version of the S4 chip, code-named Krait, is an ARMv7-A compatible CPU architecture and is made for TCMC's 28nm process, and inside the MSM9860 you'll find two of these cores running at up to 1.5 GHz along with Adreno 225 graphics, Qualcomm's new baseband, and improved ISP – it's all ramped up, basically. This device we're testing the Snapdragon S4 out on is not a consumer product, instead it's a device which accurately shows the power of the S4 chip exclusively.

This device is running Android 4.0.3 at the Snapdragon S4 chip at 1.5 Ghz currently, with a 4-inch 1024 x 600 display up front and 1GB of LPDDR2 inside. What this device replicates for the most part is the previous generation MDP which has basically the same specifications, including the 13 megapixel camera on the back, save for the new S4 chip inside. Have a look at our MSM8660 hands-on from 2011 for more comparisons if you'd like to compare.

What we're doing in this first of a series of benchmarking posts is having a look at the more common benchmark apps out there for Android. These are the tests we've used before on smartphones and tablets alike, and what you'll be seeing is how well the device and more importantly the processor inside it perform. The first test we're looking at is AnTuTu. This happy little bunny rabbit tests performance in memory, CPU Integer, CPU Floating point, 3D Graphics, SD card reading/writing speed, and Database IO via a series of on-screen tests running on your device in real time. The results we've gotten are thus:

Compared to what we've gotten with the only device in our hands with a comparably new chipset, the ASUS Transformer Prime with an NVIDIA Tegra 3 quad-core processor, results show that though the S4 does have significantly less of a high score in RAM, CPU integer, and CPU float-point, its nearly the same or better than the Tegra 3 in 2D graphics, 3D graphics, and Database IO. Take note that the Tegra 3 is running at 1.4GHz and the S4 is running at 1.5GHz, and that the S4's total score on AnTuTu in the end is about 3,000 less than what the Transformer Prime turned up, but that having a winning combination in graphics like we're seeing here is an interesting set of points when you consider how well the Prime and the Tegra 3 on the whole have been received in the graphics department thus far.

UPDATE: As you'll see throughout the post here, there are several addendums that need to be made to make sure you, the reader, understand the vital information surrounding not only the S4 developer platform device, but the Transformer Prime as well. The first important point to make here about the MDP is again, as mentioned, it's not a consumer product – but the Transformer Prime is. As such, the chipset will undoubtedly be changed in several ways which give it different results when it's placed inside a smartphone or tablet. The Transformer Prime is, of course, a consumer product, and has therefor been optimized for both battery life and thermal constraints, this putting the Tegra 3 at a disadvantage when benchmarking.UPDATE 2: It's also important to note that these devices are being compared with scores from the Transformer Prime as it existed in our original review before Ice Cream Sandwich was released on it. As such, these tests are also showing a system running Honeycomb vs a system running Ice Cream Sandwich, giving a significant advantage to the MDP.

For those of you that like to compare numbers, have a peek here, with Tegra 3 on the left and the Snapdragon S4 on the right:

RAM: 2055 / 1282

CPU Integer: 3532 / 2321

CPU float-point: 2787 / 1103

2D graphics: 296 / 297

3D graphics: 1208 / 1254

Database IO: 305 / 480

SD card write: (14.2 MB/s) 142 / (6.2 MB/s) 62

SD card read: (22.4 MB/s) 191 / (30.4 MB/s) 194

CPU frequency: 1400 MHz / 1512 MHz

Total Score: 10516 / 6993

UPDATE: What we need to notice here is that because the Transformer Prime has a display resolution of 1280x720 while the MDP has a display resolution of 1024x600, the Prime is processing 66% more pixels than the MDP. As such, the scores in both 2D and 3D can be incredibly misleading if you don't consider the fact that the Prime is doing 66% more work.

Next we'll have a look at Vellamo, a comprehensive test created by Qualcomm of how the system, CPU, and browser perform under pressure. For more information on how Vellamo came to be created by Qualcomm and what it does, head back to our Vellamo Benchmarking System Hands-on from back in July when the system was first released, and decide for yourself whether the fact that Qualcomm developed it is reason enough to use it – or reason enough to not trust it at all. The set of tests you see in Vellamo are compiled into one by Qualcomm and made to be a non-partisan test showing how well a device can handle them. Results here have been impressive, to say the least:

For our next test we're working with the Java port of the original Linpack, here made for Android to show off memory bandwidth and act as an FPU test. We've run the test six times total, three each for single-threaded and multi-threaded. Note that we've taken only the consistent results, finding an "inconsistent result" a couple times when running multi-threaded. This sort of situation has not been uncommon on Android devices since the app was first released though, so no worries!

The results you're seeing here absolutely decimate not only the previous generation MDP device running Qualcomm's 1.5GHz dual-core MSM8660 chipset, but the vast majority of the processors on devices released today. What this means is that Krait's improvements on both cache and memory control are working their magic quite well.

Next we've got Quadrant Benchmarks, both Quadrant Advanced and Quadrant Professional, both of these running a set of tests including CPU, Memory, IO, as well as 2D and 3D graphics. Quadrant is one of the few benchmarks that specifically claims to test multi-core processors to their fullest as well as specifically being tuned for Android as new as 4.0 Ice Cream Sandwich. The Advanced version is made specifically for, as Aurora Softworks, the developers of Quadant, say: "Tech savvy users experimenting with custom ROMs and overclocking."

This means the test is tuned to specifically attach itself to specific workings of the device in a way that will accurately tell them not only how well the processor is working, but how well its performing specifically based on the software and the hardware its running on. Quadrant Professional is similar, made instead for hardware vendors and game developers as well as publishers (like SlashGear) who plan on publishing results. It simplifies the results in the grid as well as comes with a license, essentially. Have a peek at these results and see what you think:

UPDATE: Again note that these two charts both show the MDP, one time running on Quadrant Advanced, the other on Quadrant Professional. Again the results showing the MDP up against the Transformer Prime are doing so with numbers taken from the original Transformer Prime review before it was running Ice Cream Sandwich. That said, one should also consider once more that the 2D and 3D subsets should take into consideration the fact that the Transformer Prime is processing 66% more pixels. Because of this, the MDP is again given a significant advantage. We'll be running additional tests in the future to more accurately compare these two devices such as GLBenchmark 2.1 with its offscreen test specifically to show the performance at the rather common 720p. In addition, consider the fact that the MDP is running at a higher clock speed than the Prime, here at 1.5GHz vs the Prime's 1.3 GHz – this will inevitably give a massive advantage to the higher clocked device for CPU subscore.

What we're seeing here is an unbelievably high IO, first of all. This test shows how well the device is able to perform with internal or SD card storage. The test shows what's the result of a set of read and write tests and, for example, overtakes the Transformer Prime with Tegra 3 by a full 3300 points. Next we've got 3D test showing an abnormally giant result as well, trumping the Transformer Prime by 2X. I'd be lying if I didn't say I was extremely surprised by these results thus far. Have a peek back at the full Transformer Prime review to check out any other specific points you'd like to access for benchmarks.

This is just the first of many sets of benchmark tests and tests of the device's Snapdragon S4 SoC we'll be performing in the near future. One thing we're going to continue to emphasize is the idea that it's not necessarily these numbers that matter most, it's the performance of the machine. Not just how well the machine processes data, but how engaging, full, and satisfying the processor will make the devices it's embedded in. Thus far what we're seeing on this beast of a developer platform device is that whatever this S4 chip ends up landing on, it'll be a fabulous experience – now we just need manufacturers of smartphones and tablets to bring their A-game to create some fabulous devices in the near future.

Stay tuned to SlashGear for more MWC 2012 news through our [MWC 2012 portal] and stick around for the extended future of our work with Qualcomm through, of course, our [Qualcomm portal]. Stay tuned for more benchmarks soon!