
Apple’s statement on the Consumer Reports battery test - arm
https://marco.org/2017/01/10/cr-mbp-battery-update
======
BugsJustFindMe
Stop. Please stop. This is nuts. I know we all hate the new MBP, but this is
just not an honest take. Consumer Reports found a bug exercising a feature
that nobody in their right mind uses in real life and it invalidated their
test results. That's it. Consumer Reports was not trying to test cache
disabling. If they had been, they could have easily shouted from the treetops
"Oh god, look everyone, cache disabling eats battery life!" But they weren't
trying to test that. No sane person would. They were trying to reproducibly
test page loads and they failed to get meaningful data because of a not-their-
fault problem with their testing methodology.

 _Consumer Reports did get very poor and inconsistent results from their
battery test, which was a reasonable and valid test_

Wrong. Yes it was a reasonable test, but their results are invalid.

~~~
skc
>>Wrong. Yes it was a reasonable test, but their results are invalid.

I'm confused. In what way are the results invalid? The results were as a
result of a bug in Apple's software, no?

~~~
iaml
I'm not the person that you are asking, but I agree with them. The premise of
this test is to measure battery life in a common use case. To achieve this,
they use an option that nobody apart from testers would use and this
inherently makes a use case not so common. This is a methodology error in my
opinion. What they could have done instead was to use incognito mode of safari
which they reopen each cycle.

~~~
jdalgetty
I use this feature quite often when testing site development. I'm sure other
web developers do the same. I actually run two safari browser windows at once.
One with private browsing and one normal. This helps to see how a site
functions under different circumstances.

~~~
iaml
Do you actually refresh the pages as often as they do though?

~~~
jdalgetty
Nope :)

