Consumer Reports said in a blog post Tuesday that Apple has a fix for a bug uncovered in CR testing. According to statements from both CR and Apple, there was a bug in Safari that caused battery life issues on Apple’s new MacBook Pro with Touch Bar. That bug played a role in inconsistent battery performance in CR testing, results that had previously caused CR not to recommend the new MacBook Pros.
Apple has released a fix for that bug to its beta program, and said it will release it to the public in the coming weeks.
MacBook Pro with Touch Bar Battery Life
CR raised a ruckus by not recommending the new MacBook Pro models in part because of poor and inconsistent battery life. “In our tests of three different MacBook Pro models,” the magazine wrote, “we saw battery life results as long as 19.5 hours and as short as 3.75 hours.”
Many Apple fans prefer to dismiss Consumer Reports when the magazine bags on Apple, but CR does carry weight with many consumers. Apple apparently knows that, because the company reached out to CR in the wake of those tests.
The magazine said that Apple requested “multiple rounds of diagnostic data,” and analyzing that data revealed a bug.
Hidden Settings
In addition to the bug, CR uses a developer setting that disables caching when testing battery life. The magazine said it does this on all laptops from all manufacturers when testing batter life to remove variability in battery testing results.
It may produce more consistent results, but…well…that’s not how modern laptops work. macOS, Windows, and ChromeOS all make extensive use of caching to improve overall performance, including battery life. Turning off that caching is just not how consumers experience modern computing.
It’s hard to imagine why CR thinks this is a better way to test. To wit, “Many of these settings are set by default to extend battery life. That’s generally a good thing. But because these settings are so variable and situation-dependent, we turn several of them off during testing.”
Again. That’s just kind of weird to me, but you can read more about it in the blog post on this topic. I should also note that said testing decision has been applied to all manufacturers, and that Apple isn’t being singled out.
Apple Statement
For its part, Apple said in a statement:
We appreciate the opportunity to work with Consumer Reports over the holidays to understand their battery test results. We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache…. We have also fixed the bug uncovered in this test.
Note the subtle mention of the bug slipped in at the end.
Apple’s statement also said:
This is not a setting used by customers and does not reflect real-world usage. [Consumer Reports‘] use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life…. This is the best pro notebook we’ve ever made, we respect Consumer Reports and we’re glad they decided to revisit their findings on the MacBook Pro.
That Fix
The reality, though, is that battery performance has been an issue on the new MacBook Pro models. Apple has pushed this fix out to its beta program, but it remains to be seen if it will improve performance for a large number of users.
An organization like Consumer Reports(CR) must apply a consistent testing methodology or its results would be considered suspicious at best. Apple should be thanking CR for finding a major bug. Brian
“Ironically, the first thing I do when I launch Safari is go to the dev menu and turn off the cache.”
Same here. I am wondering if there is a way to permanently turn it off.
Ironically, the first thing I do when I launch Safari is go to the dev menu and turn off the cache. I’ve discovered that the cache causes problems with some sites I frequent (forums, and such), so I turn it off almost as a reflex. Also, my subjective impression is that on my old MBP, with a hard drive (7200RPM), and a fast internet connection, that the cache is slower than just downloading fresh every time. I don’t know if this is true, but this is my subjective impression. And this holds true for me with Safari and Chrome.