Microsoft Surface Vs Consumer Reports: What You Need To Know

Microsoft might be reconsidering its Consumer Reports subscription today, with the news that the Surface Laptop and Surface Book have both been stripped of their "recommended" status. Concerns about reliability of the Surface line-up as a whole have prompted the review organization to backtrack on its awards, in addition to suggesting that the Surface Pro shouldn't be among your top considerations for a 2-in-1 hybrid, either. Microsoft is suitably outraged, as you'd expect, and indeed there are some big questions around just what Consumer Reports is testing and how they're doing that.

As the company puts it, Microsoft finds itself in last place for reliability when it comes to Surface notebooks and tablets. 25-percent of Surface owners experience some sort of breakage by the second year of ownership, it's suggested. Apple, in contrast, is rated as number one, with a mere 10-percent; Samsung takes the second spot, with 16-percent. It is, at first glance at least, a damning judgment on what have otherwise glowing – including, previously, by Consumer Reports – reviews of notebooks such as the new Surface Laptop.

Ironically, though, the Surface Laptop wasn't even on sale when Consumer Reports was gathering the statistics that would go on to shape its opinion of the notebook's long-term reliability. That's because of the methodology by which the company bases its predictions on, looking at existing owners who likely aren't using the very latest version of the hardware in question.

Indeed, though this report is dubbed the "2017 Winter Product Reliability Survey", it actually only covers three months of notebook and tablet ownership in 2017 itself. Instead, it looks at experiences of more than 90,000 notebooks that were bought new between 2014 and the first quarter of this year. As a result a large percentage of Microsoft's current Surface line-up wasn't even on sale at the time.

The Surface Pro 2017, for instance, was released in June of this year. Microsoft's Surface Laptop began shipping in the same month. The Surface Book – which was updated with a new "Performance Base" in October 2016, but otherwise left unchanged – was available during Consumer Reports' survey period, as was the Surface Pro 4 that has been superseded by the Surface Pro 2017, though remains on sale today.

The Consumer Reports Nation Research Center's process highlights one of the issues that continues to affect electronics reviews: striking a balance between long-term performance and short-term relevance in a segment where rapid iterations are the standard. Neither the Surface Laptop nor the Surface Pro 2017 have been on sale long enough for anybody to have built up a significant period of time using them, certainly not the two year bracket that the research company uses for its reliability metric.

Indeed, although Consumer Reports doesn't publicly break down the number of each prior version of Surface hardware owned by respondents, it seems likely that only a minority percentage were using devices that Microsoft actual sells, as new, today. To fit within that minimum of two years use, in fact, applicable models for the survey will have included the Surface 2, Surface Pro 2, Surface Pro 3, and perhaps the Surface 3 if purchased right at the point of release. The Surface Pro 4 and original Surface Book, launched at the end of 2015, have not been on the market long enough to fit within that two year bracket, though it's unclear if Consumer Reports makes that a minimum requirement for inclusion or would include those models in its findings.

Spooling back, it's clear that some of those early Surface models did encounter issues. The Surface 2, as well as facing criticisms for Windows RT, encountered overheating problems. The Surface Pro 2, meanwhile, had some battery and sleep issues, while the Pro 3 added reports of touchscreen problems to that list. The Surface Pro 4 saw a number of high-profile complaints about Intel's Skylake performance.

It's not to say that the Surface Book, Surface Laptop, or Surface Pro 2017 are perfect machines. Certainly, there have been reports of sleep issues on the Surface Book, and no notebook ever has a 100-percent reliability rate. Nonetheless, we have a little sympathy for Microsoft when, in a statement on Consumer Reports' conclusions, it argues that each new iteration improves on design and stability.

NOW READ: Surface Laptop Review

"Every generation of Surface surpasses its predecessors in performance and in reliability," a Microsoft spokesperson said. "Surface return and support rates are in line if not lower than industry average for devices in the same class."

With no hard numbers from Microsoft, and Consumer Reports only making public its assertion that 25-percent – or 1-in-4 – of Surface models will encounter a problem by the end of two years of ownership, it's hard to go into any more detail on the matter. In the end, though it's sensible to take previous reliability into account, without more details it's arguably too extreme to knock the Surface line-up out of consideration if you're looking for a new notebook.