
Much of the “science” used in design is bullshit (2014) - spking
http://mjparnell.com/bullshit_science_ux_design/
======
asark
My experience is that most people in business are 1) terrible at constructing
trials, tests, or experiments, of any sort whatsoever, 2) terrible at
interpreting the results of same, and 3) have a _very_ limited tolerance for
discussion of how bad they are at it and why their various tests are
meaningless, no matter how gently it's presented.

I've not seen a place where this isn't broadly true, no matter where the
business folks come from (top-tier B-school analysts, Bob's discount MBA
emporium, doesn't matter) or how good the reputation of the firm they're at.
This also extends to management and administration in the public
sector—schools are rife with obvious pseudo-science bullshit, bad attempts to
replicate results while skipping half the measures that were taken, et c., but
dumb-ass superintendents (you would not believe it, seriously, "dumb-ass" was
not chosen lightly) and principals eat it up.

[EDIT] which is to say I'm not at all surprised design "science" is full of
BS, because it's sold to people who almost all suck at evaluating those kinds
of things. Any "science" that largely exists to sell stuff to middle- and
upper-management or "stakeholders" probably tends to be awful, because it
doesn't _need_ to be good.

~~~
kadendogthing
What are one-dimensional business people actually good at? Besides just being
a piece in the system that soaks up resources?

~~~
bonoboTP
The good ones are good at convincing, negotiating, charming, manipulating,
uniting people, turning people against each other etc. All very important
skills out in the real world of human interactions and they translate to $$
quite directly.

------
andybak
I will never tire of reminding people of this:
[https://www.cbsnews.com/news/pepsis-nonsensical-logo-
redesig...](https://www.cbsnews.com/news/pepsis-nonsensical-logo-redesign-
document-1-million-for-this/)

~~~
lqet
The PDF linked in the article [0] is pure gold.

> C. The investment in our DNA leads to breakthrough innovation and allows us
> to move out of the traditional linear system and into the future

> BREATHTAKING is a strategy based on the evolution of 5000+ years of shared
> ideas in design philosophy creating an authentic Constitution of Design.

> B. Magnetic Fields: Magnetic fields exert forces on inner and outer surfaces
> of the Earth.

> B. Pepsi Energy Fields: Symmetrical energy fields are in balance.

> C. Magnetic Dynamics: Magnetic field are impacted by sun radiation and wind
> motion.

> C. The Pepsi Globe Dynamics: Emotive forces shape the gestalt of the brand
> identity.

Also, take a look at the deconstruction of the old Pepsi logos into arbitrary
ellipses ("Perimeter Oscillations") on pages 8ff.

It gets increasingly surreal ("Light Path with Gravitational Pull" vs
"Gravitational Pull of Pepsi", "Relativity of Space and Time" vs "Pepsi
Proposition / Pepsi Aisle", difference between a "Pepsi Galaxy" and a "Pepsi
Universe", ...) on the last pages.

Dimensionalize exponentially.

[0]
[https://jimedwardsnrx.files.wordpress.com/2009/02/pepsi_grav...](https://jimedwardsnrx.files.wordpress.com/2009/02/pepsi_gravitational_field.pdf)

~~~
cocochanel
This PDF is hilarious. How can anyone get away with so much bullshit is beyond
me.

~~~
VectorLock
I can't tell if this is real or not. It feels like the Turbo Encabulator of
design.

~~~
rospaya
It's one step away from VX junkies.

[https://www.reddit.com/r/VXJunkies/](https://www.reddit.com/r/VXJunkies/)

------
oedmarap
> Everyone bullshits at some point, but there seems to me to be a particular
> desire to show design decisions as being based in some empirical facts. In
> some ways this is admirable, and forces designers to justify their design
> decisions. Analytics have certainly been one cause of this.

As a designer I've always felt that the larger and more important chunk of
design work is purely intuitive. Analytics implies time-dependence which is
counterproductive to design when included upfront.

The only time quantifiable metrics are used is when a design is married to
user experience in the context of a user interface (in the end product). At
that point, patterns and practices dictate the baseline from which feedback
takes place. This is commonly referred to as a design system and is done on a
larger horizontal scale across interfaces (web, mobile, print, and so on).

Company branding, medium | message, target audience, color schemes/themes and
other aspects of functional design are comprised more of intuition than raw
analytics, in my opinion. Apple provides a great example of design choices
that focus on the human aspect first. [0]

Added to that, the fact that market share is happily split amongst vastly
different UI approaches is testament to the non-linear nature of design.

While designers are plenty, good design is not. Science based models as cited
in the article are there to bring up the rear in a standardized manner but
don't provide avenues to true "novelty" that defines great design. Just my two
cents.

[0] [https://developer.apple.com/design/human-interface-
guideline...](https://developer.apple.com/design/human-interface-
guidelines/ios/overview/themes/)

~~~
ben509
> As a designer I've always felt that the larger and more important chunk of
> design work is purely intuitive.

It's a highly trained intuition. From watching artists work, they have a
tremendous amount of experience with color, form, different materials and
composing those visual elements, and then with understanding the emotional
impact they have on an audience.

------
de_Selby
"the minimum ratio of positive-to-negative emotions humans need over time to
“flourish” to be 2.9013"

"It reflects the highest positivity ratio (observed ratio = 5.6) and the
broadest range of inquiry and advocacy. It is also the most generative and
flexible. Mathematically, its trajectory in phase space never duplicates
itself, representing maximal degrees of freedom and behavioral flexibility. In
the terms of physics and mathematics, this is a chaotic attractor.”"

They needed someone to write a paper to point out this was pseudoscientific
BS? Not only that, but the paper was cited over 1000 times?!

Absolutely terrifying.

~~~
bartread
> Absolutely terrifying.

Indeed. We seem to be increasingly awash with evidence that the standards of
rigour in academic journals across a number of fields are... quite poor.

Peter Boghossian, James Lindsay and Helen Pluckrose set out to demonstrate
this in the field of grievance studies by submitting and, in some cases,
successfully publishing a series of fake papers:

[https://www.youtube.com/watch?v=AZZNvT1vaJg](https://www.youtube.com/watch?v=AZZNvT1vaJg)

I mean, in some sense this is highly entertaining for the rest of us, but it
is also utterly horrifying.

~~~
marcus_holmes
That's not the first time fake papers have been published and even cited.

No researcher has the time to read all the papers published in their field.

There are no penalties for publishing bad science, and no rewards for
debunking it.

Recent moves to stop p-hacking by removing the need to show statistical
significance will (if they get through) only make this worse.

Something has to change.

------
intrasight
I've been involved in UX design for nuclear power plants, air traffic control
systems, gambling machines, and more recently lots of web sites. None was
"science" but all were very important for their intended audience. I the case
of the nuclear power plants and air traffic control systems, there was much
more thought and analysis put in by user interface specialists, but I doubt
that any of those specialists would consider themselves scientists or their
practice a "science".

~~~
arithma
I would definitely call that engineering though.

------
rangersanger
As a field, I think UI/UX has a ton of anxiety about it's place in the cannon
of design. There's a pretty clear path from that anxiety to these hollow
attempts to justify itself as a "serious discipline."

Look at the other paths that visual designers could take, from "pure art"
illustration to a much "harder science" like architecture, UI/UX sits pretty
square in the middle. So many of the people I've worked with want it
desperately to be both art and science. I'm sure we've all had conversations
with a UX/UI designer who will ping-pong between space "feeling better" with
different padding, and something like Millers Law, or perhaps a misapplied
statistical inference from Optimizely.

I'd love to see UI/UX split into a more art focused design field and a more
science focused HCI field. This might stop the navel-gazing and impulse toward
faux-science, and publications like this: [https://www.invisionapp.com/inside-
design/why-designers-shou...](https://www.invisionapp.com/inside-design/why-
designers-should-be-shaping-business-strategy/)

------
morley
I'm a little confused by the thesis of the article, and how it relates to the
title.

Unless I'm reading it wrong, this is the thrust of the facts:

* The Android design team describes their design process. * In their description, they cite a paper that suggests giving people at least 3 positive experiences for every negative experience they have. * That paper was debunked.

How does that translate to "much of the 'science' used in design is bullshit"?
That bogus paper they cited doesn't affect the actual content of Android's
design; it just influenced the team's design process. How is that bad? How
does that discredit all the other things they talk about? The FastCompany
article[1] linked from this blog post says:

> the CliffsNotes version is that Google creates design mantras from the point
> of view of the user, like “keep it brief,” “delight me in surprising ways,”
> and “it’s not my fault.” Each time an Android feature lives up to these
> expectations, they get a single marble in the good emotion jar. But every
> time they fail, that bad feature produces three marbles in the bad emotion
> jar. The marbles illustrate that bad ideas stack up quickly.

Even if this heuristic isn't scientifically proven, does it really result in
worse UX than if you don't use the heuristic? I just don't get the vitriol.

[1] [https://www.fastcompany.com/1672657/google-s-dead-simple-
too...](https://www.fastcompany.com/1672657/google-s-dead-simple-tool-for-
making-ux-decisions-2-jars-of-marbles)

~~~
DenisM
Wrong premises are still wrong even if they lead to correct result.

The habit of relying on dubious techniques is under question here, as is our
collective ability to find better guidance.

------
sizzle
"The veneer of mathematics and complexity science – a culturally-powerful yet
poorly-understood science – allowed many people to believe a very improbable
thing: that a simple model from fluid dynamics could explain the influence of
love, hate, anger, sadness, grief, joy, culture, time, geography, war, famine,
birth, etc. etc. on human behaviour. It also suggests a degree of gullibility
in the academy: researchers were willing to accept the claims of the papers
because they didn’t understand the maths. Worse, of course, is the implication
that people simply don’t critically read the papers that they cite."

Pretty much sums up the reproducibility problem of studies in the field of
Psychology.

~~~
steakknife
No, it doesn't at all. The reproducibility issue in psychology has mostly to
do with p-hacking, which is a very specific abuse of legitimate statistical
analysis in which statistical significance is brute-forced and cherry-picked
(rendering it no longer significant), and is often applied to otherwise sound
experiments and hypotheses. The paper in question is literally gibberish.

------
human20190310
> [I]t’s better for us to accept that we don’t always have good reasons for
> our design decisions and that much of our work is intuitive.

My intuition tells me that designers wish they could do that without having
their designs rejected. If the absence of bullshit empiricism is punished, who
can complain when people start providing it?

~~~
ianstallings
Agreed and I I think a lot of it comes down to financial factors. In other
words, _why_ are we paying someone so much to make design decisions and how
can we be assured they are doing it correctly? Because of _science_.

------
cptskippy
Speaking of bullshit designs...

Why does that blog have a load screen that takes about 15 seconds to
disappear? If you open up debug tools and delete the overlay, the blog entry
is perfectly readable underneath it. There's no network activity during those
15 seconds of "loading". It's just ... bullshit.

------
captainbland
Pretty interesting article. The only bit I'd have something to say about it
is:

> Indeed, it would mean that the designers would have to add a bad experience
> in for every three good ones in order to get the positivity ratio right!

Which appears to be incorrect as the positivity ratio was supposed to be a
minimum as according to that same article. Presumably having more positive
experiences doesn't make the situation somehow worse.

Nevertheless, maybe it's an interesting or even useful heuristic that you
could get away with one bad moment for every three good moments you provide,
even if it is about as scientific as an analysis of unicorns on a flat earth.

------
tabtab
The article is short of specific examples. But as far as UI's, unless you
really are genius of design, keep it mostly the same and only do incremental
improvements. I get used to most bad UI's by shere memorization of where and
how to go. Move the cheese, and I'm lost and cussing.

The MS Ribbon comes to mind. It still don't think it's notably better than the
original toolbar. But the problem is that shuffling everything around confused
many for roughly a year as they had to re-learn where everything is.

MS traded one randomness for a different randomness. Maybe they were thinking
a 5% to 10% improvement is worth it over the long run even if they piss off
existing customers during the learning curve. So, either they are idiots, or
they actually sat around and did "piss off accounting": intentional jerks who
willingly sock existing customers in exchange for future news ones. So we have
the idiots theory and the jerks theory.

Another thing, redundancy is not necessarily bad in UI's. Some get caught up
in the "keep the tree clean" mantra. However, having more than one way to do
or find an option often improves the UI. Perhaps an _ideally_ designed UI
could avoid the need for repetition, but most designers are not good (or
constrained by other factors) such that they should indeed fall back on some
redundancy. Rules for ideal conditions (such as great UI) often don't apply to
typical conditions (average UI).

For bigger applications, perhaps put all the options in a table(s) and let the
users search for options Google-style. That may be faster than digging around
in menu trees. You still have menu trees, but ultimately they are tied to the
table. Use synonyms and allow bookmarks to improve look-ups. I'd like to see
more experiments in Table Oriented Programming. To me, that looks the future.
OOP can't handle complex relationships well.

~~~
charleslmunger
I disagree - the ribbon was a massive improvement for feature discoverability
and usability.

1\. Lots of options visible up front. Instead of reading tons of text, or
guessing at hieroglyphics, you could find the tool to accomplish your desired
task visually - if you want to change text color, look for the color picker.

1\. Big improvement to mouse navigation. Navigating nested menus is a bit like
playing Operation - you position your cursor on the item you want, then slide
it horizontally to the next menu that expanded. But if you drift too far
vertically, which was easy to do of you had a disability or a crappy trackpad,
you have to start again. With the ribbon, there was no destructive effect to
moving the mouse around without clicking. You could always take the shortest
path to whatever you wanted to click on.

1\. Big improvement to keyboard navigation. Tap the alt key, and every
affordance in the ribbon displays its keyboard shortcut. This allowed the same
exploratory navigation as the mouse, but in a way that built muscle memory and
avoided RSI.

Of course it was a large change, but the ribbon was an amazing improvement.

~~~
tabtab
Many of these features could have been added to the existing tool bar and/or
menu. Anyhow, everyone likes things differently and no approach will make
everyone happy. If given my way, I would have made it easy to create a
"favorites tray" tool bar section because I find myself using a relatively
small number of features quite often, and then use the google-esque tabular
search approach mentioned for the non-frequent ones.

------
layoutIfNeeded
No surprise.

Much of the "science" used to support business decisions is bullshit, made up
long after a decision was made by someone powerful.

~~~
mathieubordere
Humans seem to be comforted when there's an explanation for something, even if
it's just plain wrong, it seems we don't care that much.

~~~
moccachino
Our whole lives consist of rationalizing after-the-fact what we observe
ourselves to be doing.

~~~
BaronSamedi
And rationalizing because we actually have no idea why we are doing it.

------
wayneftw
This reminds me of how Apple fans used to love to quote Fitts Law to convince
themselves that the Macintosh OS UI was any good. Meanwhile, Apple never
really followed Fitts Law very well with the Mac OS _or_ iOS and they quite
obviously can't even get _the most basic functionality right_ , like Window or
File management.

~~~
bitwize
Fitts's law is why the menu bar belongs at the top of the screen. But not even
Apple gets everything right, or applies the psychological principles that lead
to the easiest, most efficient UX. If they did, they would've used pie menus a
long time ago.

As for file management, that's one area where Apple had it right and then
buggered it up. Everyone agrees that spatial Finder was a revolution in
working with files visually, then Apple changed to navigational Finder for OS
X. Boo.

~~~
wl
I remember the spatial finder making my life difficult in the MacOS 7 days.
For example, if I had a parent and child directory open at the same time in
list view and then expanded the child to drill down to another directory, the
finder would obnoxiously close the child directory window. I found the
navigational approach in OS X to be a breath of fresh air.

------
wozer
I find it somewhat difficult to work at a company with a UX team. It is just
another stakeholder you have to take care of. One more obstacle to get
anything done as a developer.

------
thanatropism
Most knowledge is inarticulable. It’s not only learned by doing, it’s
intimately indexed by our lived relationship _with_ tools.

A culture of evidence and objectivity, well exhibited by this very blog post,
has had two implications: (I) actual knowledge that was not based on explicit,
technically appealing principles, has been demoted — witness the come-and-go
of architecture, for example and (II) people where explicit and technically
appealing principles didn’t exist fell for “objectivity strawmen” or, where
they were able to socially impose their deeper inarticulable knowledge, they
have used strawmen to get the culture of objectivity off their sense.

What’s perverse about this is that people who are cynical about the socially
inflated need for objectivity end up making other, more naive, people buy
bullshit like the 3:1 ratio; and people who debunk bullshit like this blogger
further perpetuate the misconception that insufficient objectivity and
bibliographical references are a relevant problem, even while going about the
motions of disavowing the role of science (a heightened mode of objectivity
discourse) on lowly design-engineering.

Is design even engineering discipline? To its core? Is the world better off
when Google does some ersatz research and we start trusting “Material Design”
more than the deep civilizational background that artists (industrial or not)
are supposed to have at their disposal?

------
dictum
This is a consequence of subjective choices (even when beneficial) being
perceived as easy to perform (hence, cheap and unworthy of value) and likely
to be incorrect, whereas choices borne out of systematic processes are seen as
reliable and accurate.

A hunch isn't always a trustworthy source, but too often it is taken as an
always untrustworthy source.

------
alanaut24
Someone needs to explain to me what is gender neutral typography:

[https://visme.co/blog/feminine-design-masculine-
design/](https://visme.co/blog/feminine-design-masculine-design/)

~~~
syockit
Wow. That somehow explains why we end up having very dull monotonous icons in
many things today. One of the GUI desktop applications in my company had a
redesign for a new release, and they opt to go for the Material Design[1]
icons. I remember that being a Google thing for their Android UX, so I thought
it couldn't be that bad.

When the icons are actually used in the application, they look so... bland.
They also look mostly similar to each other, basically being rectangular
outlines with rounded corners, that it takes slightly longer to recognise
which one is which, compared to the more traditional, colourful icons. (I
wonder if colour blind people have always felt this way)

I should've prefaced that our application uses Qt5 Widgets as the UI, with
hardly any styling. That may have contributed to the icons looking so out of
place. If we had used a more metro-looking theme (like the ones used in Adobe
PS) they would be a better fit. But our main users are engineers and
scientists, who I assume would prefer more traditional-looking UI (see for
example Paraview).

Meanwhile, I have no problems with the same icons being used on my smartphone.
I guess this is because on smartphones, icons are larger and more easily
distinguished even if they're monochrome. And they can afford to be larger
because they're mostly used in main menus or swipe menus. In contrast, in
desktop applications they always appear on screen (in the toolbar), and
monitors are usually further from your eyes compared to smartphones, and you
can't afford to sacrifice precious screen estate just to accommodate
distinguishable icons.

[1] [https://material.io/design/iconography/system-
icons.html#des...](https://material.io/design/iconography/system-
icons.html#design-principles)

------
byset
While I have no problem with the linked article I think the headline is a bit
unfair. How often is BS-ey pseudoscience like this really used and cited by UX
designers in the field? It would seem to be mainly an issue among the most
bloated, coddled, and self-important UX teams.

And was the BS paper in question really used to _design_ the Android UI rather
than _justify_ the design decisions? (The article's author basically makes
this same point.)

------
rajekas
Except that you don't need to publish journal articles anymore. Instead,
combine 1 cup pseudo-psychology, 1 cup vague design principles and 1 cup
quotes from Donald Norman and Massimo Vignelli. Sprinkle with Kahnemann on
thinking fast and Pinker on evolutionary strategies. Publish it on Medium and
watch it go viral - pop design science and Medium's business model being
perfectly aligned right now.

------
byset
It's noteworthy that Alan Sokal was involved in debunking the paper discussed
in the article.

He's the guy who caused an uproar in academic circles (in mainstream media
too) with his notorious 1996 hoax in which he submitted a paper loaded with
comically absurd assertions to a postmodern journal called Social Text and it
got published. Google "Sokal hoax" for a good laugh.

------
guelo
Once after I had a bad experience with adding a "FAB" button to an app I got
the chance to ask a Google Material Design designer if they had AB tested or
user tested the design and they said no, obviously, because users don't know
what they want. So. Much. Bullshit. in design.

~~~
snegu
That's just a bad designer. The foundation of UX is doing actual user
research.

~~~
guelo
No, I'm talking about the Material Design guidelines that Google has been
pushing for years. They publish their design system without testing.

------
edgarvaldes
Just a few hours ago in the frontpage: Bullshitters: Who Are They and What Do
We Know about Their Lives?

[https://news.ycombinator.com/item?id=19749130](https://news.ycombinator.com/item?id=19749130)

------
m3kw9
The problem with Android and UX is that they think human experience can be
treated like a stat.

~~~
beenBoutIT
The fact that there is no better mobile UX than that provided by Google's
Android 9 (Pixel phones, not Samsung) suggests that whatever they're doing is
working.

~~~
thirdsun
Seems like an opinion rather than a fact.

------
Razengan
“Design by committee” is always a bland, sterile and soulless experience.

------
_pmf_
"Yea, can you just give it a bit more 'zing'?"

------
Causality1
Does this explain why Google went from the tasteful white-on-black status bar
of Android 8 to the enormous blue balloon PlaySkool buttons on Android 9?

~~~
discreditable
It wouldn't be an android release without Google making random nonsensical to
the status bar.

