

How iOS7 Stacks Up : Smartphone OS User Experience Shootout [pdf] - hokkos
http://www.pfeifferreport.com/v2/wp-content/uploads/2013/09/iOS7-User-Experience-Shootout.pdf

======
Systemic33
There is a really interesting comment on reddit regarding this survey,
specifically about how dodgy its methods was.

"This is not me saying that this is a bad study because I'm a Microsoft
fanboy. This is me saying this is a bad study because I'm a scientist.

The "study" says that "cognitive load is the sum of elements a user needs to
be familiar with in order to use the operating system." That's actually a good
thing to look at, and would be interesting. Unfortunately, they counted the
number of pre-installed applications and decided that was somehow linked to
the cognitive load? I don't even begin to see a connection between the two.

In the "efficiency and integration" section, they do not actually run a study
to see how quickly different tasks can be done (which would be efficiency)
-and I don't see anything about integration at all- instead they simply talk
about how many short cuts there are- except on Android, which is docked for
having too many short cuts.

The customization section showcases 2 problems. 1. They assume that more
customization is always better and 2. If what you're going for is
customization, than you better make Android a 10 and the next highest
operating system a 2. The fact that they have Android and iOS7 anywhere close
together shows they did not use any methodology for assigning score.

And finally, the UFX score. I don't even know where to begin. First, he claims
that you can close applications on iOS7 and Android to "free up resources,"
but in reality both camps have said explicitly that their app launcher does
not control resource use. He claims that iOS 7 can fit 24 application
shortcuts, while Windows Phone can barely fit 12. In reality, if all you want
is icons with counters (which is all iOS 7 offers you) then you can fit 28
icons on Windows Phone. He also complains about not having a background image
on Windows Phone, never once mentioning how you wouldn't be able to see it (he
also lists this under UFX- not sure how this causes any friction. Anyway, he
also list this under customization (which makes more sense) and double counts
this as a negative). And the oddest one- he claims that the Lumia 920 has
"sponsored links on the homescreen." I don't have a Lumia 920- so I can't
speak from personal experience, but none of my Google searches reveal anything
about this and it is something that I am quite sure would be railed against
frequently if true. So in that case, I'm pretty sure it is a lie.

Now, that being said- there are of course problems with Windows Phone.
However, I really don't think this article captured any of them." \-
Weed_O_Whirler

------
Joeri
Strange how trying to make something objective can make it inaccurate. For
example, I deliberately changed from android to windows phone because I much
prefer the windows phone user experience, but it scores the worst in their
objective comparison.

All the complaints about windows phone are mostly accurate, but largely
irrelevant for me personally. For example, I don't want a background image for
the live tiles screen, so I don't care that I can't configure one.

I do agree with their final conclusion that windows phone is a strange mixture
of careful user-centric design and complete disregard for actual user needs.
There are things which are incomprehensible that they haven't been fixed over
the past few years, like not having a separate ringer volume. If, like me, you
are lucky to have needs which mostly align with the intended design, then it
beats even iOS in usability. But if you are only slightly out of alignment, it
is a long fall down the usability ladder.

~~~
MartinMcGirk
I only just picked up a Lumia 1020 two days ago after using an iPhone for the
last 2 years, so the jury is still out for me on whether I prefer it or not
and how well it suits my particular set of needs. I'm not even sure I can
accurately define what my particular needs are right now (except that I wanted
the 1020's great camera). I wonder if you could elaborate on what needs it is
that you think Windows Phone suits so well?

------
poolpool
No you can't compare iOS 7 to android until you install my bespoke launcher
and remove all the default icons and widgets and customize the phone!

It's not fair picking both up out of a box and comparing the user experience.

~~~
Oletros
You're joking, don't you?

~~~
poolpool
Yes but unfortunately there are people making that argument quite seriously.

------
Oletros
When one of the negative points is the number of icons and widgets it says a
lot about that report.

~~~
briandear
Like what? Things that clutter the user experience are certainly valid points.
Otherwise we might suggest that the old godaddy website was well designed
because it had lots of "features."

~~~
Oletros
Number of apps installed is not what defines clutter.

------
rubidium
It's probably worth noting that Pfeiffer, the consulting group which makes
this report, seems to be a consultant for marketing departments.

One shouldn't read these reports as a scientific publication (b/c their
methods are certainly suspect at best), but rather a way to try and translate
product differences into things for the marketing department to emphasize.

------
k-mcgrady
Is more customisation really better? I would have left that out of the report
as it can be argued either way. High levels of customisation mean you can
tailor the device to your needs. On the other hand a device might be designed
to work perfectly for 90% of the users and offering customisation options will
just lead to confusion and people not getting the best experience.

~~~
rimantas
I really liked someone's observation: Android is about what you do to your
phone, iPhone is about what you do with your phone.

~~~
Tyrannosaurs
I think that's slightly unfair.

The reason a lot of people like Android is that they can customise it to
remove even the slightest friction from the things they want to do. They have
to make a bit of an investment in time to do that but to them that's a good
thing because it will pay off.

Android is absolutely more about what you want to do to your phone than iOS,
but I'm not sure it's fair to suggest that it's not about doing things with
your phone. That feels a bit like someone just trying to find a comparison to
make it into a nice phrase.

------
kalleboo
This seems like such an arbitrary report. Looking at the "analysis" for UXF it
just reads like a list of random annoyances their testers came across.

------
timmillwood
Nice to see they pick Samsungs version of android, the worst one going.

~~~
poolpool
Yes and its the one most people know?

Let's compare user experience. One phone is straight out of the box. The other
we will just take a minute to gain root and reinstall the entire OS. Just to
keep things fair.

~~~
SifJar
But it means they're not comparing iOS (and WP and BB10) to _Android_ ,
they're comparing them to _TouchWiz_ (or whatever Samsung are calling their
skin these days). In that case, they should also include HTC Sense and
Motorola Blur etc. The fact that TouchWiz is most popular is irrelvant; pretty
much any Android skin is probably more popular than BB10, and many are more
popular than WP as well.

~~~
poolpool
Relatively no one who walks into a store to buy a phone understands any of
those words you just said.

They just know they bought the most popular android phone and I guess this is
what android is?

Edit: By your definition you can't compare anything to android because android
doesn't exist.

Is it possible to compare windows to linux?

~~~
Lonik
> Relatively no one who walks into a store to buy a phone understands any of
> those words you just said.

> They just know they bought the most popular android phone and I guess this
> is what android is?

SifJar was talking to you, not the average person who walks into any store.

> Edit: By your definition you can't compare anything to android because
> android doesn't exist.

He didn't try to (re)define anything. To give you a clue, when someone talks
about Android to me, I think about the Android Open Source Project, the OS
managed by Google. It's essentially what the Nexus devices come with.

The problem with the study is that they claim to compare different operating
systems, while more precisely, they are comparing different user experiences
on different OSs. Big difference, I'm explaining it below, and they should
know better.

In the case of Android, most OEMs do everything they can to differentiate
their devices from other Android devices, and thus they offer a fundamentally
different _experience_. They even give names to these experiences, e.g.
TouchWiz or Sense. So the authors were actually comparing TouchWiz to other
operating systems. This is evident by the fact that some of the features they
felt were lacking are actually in Android trunk, and some of the features they
found problematic are not.

Thus, I feel like the people who did this UX study didn't actually know what
_exactly_ they were studying, based on their choice of words (TouchWiz is not
mentioned at all).

------
gafdlo
the fundamental flaw of these kinds of survey: if you have enough time to
participate then you're not majority, you will never value time-saving
features as highly as people who matter will, for example. lots of other
things. I never regard experiments using accidental sampling seriously.

------
poolpool
Can the experience being behind a wall garden buddy buddy with the NSA ever be
truly good, ignoring superficial things like icons and launchers?

~~~
pilsetnieks
Indeed, you make a good point for avoiding the Google Android ecosystem.

