

Tell HN: New Google homepage, A/B testing gone crazy. - Loic

Today, I was served the new Google homepage, the first that struck me out was how inconsistent it was. I started to look at all the inconsistencies and found many of them:<p>- 3 different arrows for the more links.
- mixed use of upper/lower case for the "M" in the more links.
- the position of "advanced search" between the home page and the results page.
- the position of the search button (kind of within the input field on the results page but below with space on the home page).
- you see "Everything" with an icon in front of it (why the icon?) and then you have a link with more, what is more than everything?
- etc.<p>It can be a game to find so many design errors, things totally wrong when looking at the whole page.<p>I asked myself, why? Then I found, it is simply A/B testing gone crazy. They have tested gazillions of independent small changes in their multivariate tests, found the <i>optimal</i> combination, but, forgot the big picture.<p>So maybe the new home page and results page are the optimal ones, but they are inconsistently optimal. For the first time in many years, the Google pages now "feel wrong".<p>This is annoying me a lot, for 2 things:<p>- I hate it when I search.
- It will spread like wild fire because if Google has done it, it must be good, so bad designers will just copy the design without thinking twice (so I will need to support that in many other websites).<p>And you, what do you think about it?
How are you designing your A/B tests not to fall in this trap?
======
jacquesm
It is the first time for me that I think that changes to the google homepage
are really a step in the wrong direction. In the past they seem to have had it
together on this one, if there were changes they were subtle and did not seem
to get in the way of getting the job done.

Now they've gone overboard on the 'b(l)ing' factor, presumably to compete with
other search engines that look 'spiffy'.

I'd rather have 10 text links (or even 30) per page without adornment than all
the stuffy they've been adding lately.

Duckduckgo could easily capitalize on that, give us the very early google
interface with better results.

That would make me switch, their scroll-down-to-get-more trick already is very
elegant.

~~~
Loic
Thank you for reminding me about Duckduckgo, I always dismissed it because of
the javascript requirement, but trying again today I found it pretty clean. I
will need to commit myself to a one week long test to better assess how good
it is.

~~~
jacquesm
While you're testing, try gigablast.com as well, they're pretty impressive.

------
msy
Looks like Douglas Bowman was right and its only got worse since he bailed.
<http://stopdesign.com/archive/2009/03/20/goodbye-google.html>

~~~
jessriedel
Bowman is more describing why he doesn't _like_ working there; he really
doesn't make a claim that Google is making poor design choices from a business
standpoint. He criticizes the use of data (aka, you know, _evidence_ ) as a
crutch without giving a single example, theoretical or otherwise, of how
reliance on data rather than the gut instinct of a good designer leads to a
bad outcome.

~~~
lzimm
I think its the same difference between investing in company lead by a former-
consultant/bean-counter/"professional"-CEO and a real founder.

data is important, but it only gets you so far: table stakes. the thing that
differentiates us is the ability to use intuition.

don't forget that your "gut" has been developed over millions of years of
evolution to keep you alive, and its done a pretty good job. playing with
numbers? couple thousand?

~~~
jessriedel
Intuition/experience is important when there is some aspect of reality that is
hard to measure. Insofar as the data tracks your target, intuition is
unnecessary. That's why I would want an example in which data was failing.

For instance, a fast food chain might take some data and find that they can
decrease the quality of their ingredients without immediately loosing
customers because people have habits and don't really notice right away (and
because consumers will chalk up a few bad experiences to variance at a place
they know well). But, over time, the chain develops a reputation for lower-
quality food, and customers slowly leave.

The problem with this example is that the data can easily measure long-term
effects, especially the subtle damage to reputation. This is a case where
intuition may be needed to override data.

But where's the similar example with Google?

------
endlessvoid94
I mean....

these are such tiny gripes. Some arrows are different because they aren't of
the same priority..people will click "More" more often than they'll click
'more search tools', so that arrow is smaller. makes sense to me.

How exactly is the search button inconsistent with the homepage? You think it
would look better BELOW the textfield on the SERP?

I don't see any mixed case anywhere.

Not trying to be a troll or bust balls or anything, I just find your
criticisms a bit nitpicky. The overall design of the site is incredibly simple
and easy to use.

~~~
Loic
What really annoys me is that you have no space between the input field on the
SERP (first time I discover this acronym) and the search button. On the
homepage you have some space.

Also the advanced search link is now below, not anymore right from the input,
it kinds of make this new SERP cluttered. Note that when logged in, it is even
worse, I have another link for the safe search.

Also you will note that you have 4 or 5 different size of fonts.

I agree with you, roughly it is working (it what the data prove anyway, else
they would have not done it) but nevertheless for me it feels wrong.

<http://xhtml.net/documents/images/logged-in-google-serp.png>

------
kyro
Here's my problem with the design: With the old one, my eyes could act like
magnets, being quickly attracted to either the left or right side, sliding up
and down either side when viewing search results / ads. Now I have to do this
balancing act, putting in quite a bit of effort to keep my eyes on the center
column. Whereas I previously just let the magnetic forces direct my attention
with the old design, I now have to resist the pull from both sides to keep
center focused, making it extremely uncomfortable. Part of me thinks this new
design probably yielded more ad revenue because now people can more easily
succumb to the pull from the right. It's a lot less polarized than before.

~~~
adam_feldman
This is exactly how I feel. I have Adblock, but there is still a very strong
pull to the left column that's distracting as I'm searching.

------
_delirium
This is commonly talked about in the design field as well (since at least the
1960s, probably earlier). On the one hand, there's a very industrial-
psychology-derived HCI tradition, based on user-testing everything with
statistical significance and so on, that tends to do relatively small changes,
e.g. assuming that most of the design will stay as it is, how about widget A
versus widget B? Against them, there's more of a _design_ tradition, derived
from architecture, graphic design, art, and other areas, that looks at more
holistic sorts of issues, and tends to think that doing 1000 A/B tests does
not add up to "design", and isn't likely to produce good outcomes either. HCI
hasn't really resolved the issue over several decades, and it flares up now
and then (my impression is that the main HCI conferences/journals are ~80% the
industrial-psych sort, but the 20% dissenters have persistently stuck around
for decades).

~~~
msy
There is a fundamental misunderstanding of design at play here though. We're
talking about a culture that doesn't understand that when it comes to design,
1+1 should equal 3 or more. You cannot separate out individual elements and
A/B them without looking at the bigger picture. The net result is this - a
kind of unsettling, uncomfortable inconsistency. The element level granularity
of design hierarchy & planning is obvious. I love agile and I'm a fan of fast
iteration and lots of user testing but a strong, consistent design requires
top-down discipline and bold decisions, you cannot increment your way there.

~~~
_delirium
Yeah, I agree with that. It's not even particularly surprising from a
scientific perspective: any scientist knows that if you measure A, B, C, and
D, you don't necessarily know anything about union(A, B) or union(A, B, D),
unless you also measure pairwise and higher-order interactions. At some point,
instead of doing more A/B tests than there are atoms in the universe (since
this explodes exponentially), it might be worth thinking about the problem
using human intelligence.

------
samd
Perhaps another way of saying this is that Google's A/B testing has merely
taken them to a local maximum, but they need a strong, coherent design
discipline to reach the global maximum.

------
pbhjpbhj
You should provide some image links when commenting on visual appearance
especially WRT A/B testing as other users will get a different experience by
definition.

Google homepage is like <http://imgur.com/sc5h6.jpg> for me so I'm guessing
you mean the logged in personal home?

~~~
Loic
Your screenshot is for the home page before moving the mouse, of course you
see nearly nothing, so it is consistent. Anyway, here is a commented
screenshot of the search results page (not all the comments fit in, especially
the font size on the left for the tools, so I did not include them). This is
especially this one annoying me because I access google through the search box
of my browser.

<http://xhtml.net/documents/images/new-google-page.png>

~~~
isleyaardvark
I don't like inconsistencies either, but I don't mind them as long as they're
consistent within a conceptual section.

The arrows on the sidebar, for example, cause a different behavior when
clicked than the arrows up top. The pipes between the links on the right are
between links relating to your personal settings as opposed to the links on
the left, which are filters for the search results (with the exception of
Mail, which is inconsistently placed).

Overall I like the new look but I can see how it may not be the most cohesive
design.

------
alayne
Well, I'm losing my religion. I already have a Greasemonkey script to stop the
fading top row. Look at the results page for "cat toys" in Bing and Google.
Maybe Google should just redirect to Bing.

I get that they're trying to satisfy a diverse user base and drive UI changes
from testing, but it sure came out ugly.

~~~
thorax
Yeah, what struck me was that if style was your differentiator for using
Google versus Bing, you just lost your motivation to stick with Google.

Some examples:

Bing's style: <http://www.bing.com/search?q=mortgage>

New Google style: <http://www.google.com/search?q=mortgage>

Compare to the old google look:
[http://www.darrinward.com/img/articles/screenshot-google-
sea...](http://www.darrinward.com/img/articles/screenshot-google-search-
mortgage.jpg)

~~~
puredemo
Excellent post. I had no idea how similar the Bing and Google result pages are
now. What is up with that? It couldn't have been unintentional, could it?

------
nate
What exactly are they optimizing or testing for here? Just number of searches,
or clicks on that feature. Have they eliminated that people might be clicking
on all these odd ball changes because its novel? I know I've expanded some
menus and clicked on stuff and hovered over shit I wouldn't normally just
because these changes were all shiny and I wanted to see what they do.

But in the end, the only change I like is the logo. Everything just looks a
little odd. But my only real gripe is the location of the search results. Is
it just me, or have the serps been left justified or were they much more in
the center of the screen before? Every friggin machine is getting wide
monitors, so to make stuff go left these days and just give me all this
whitespace in the center and the right, makes me a little nuts.

------
piramida
Just a thought - it may feel uncomfortable because people never accept
changes, we all need time for brain patterns to rebuild. My first impression
was - get the old page back. Second impression was - it now exposes some
functionality I usually had to go to advanced for. I'd wait a month before
jumping to any conclusions.

Yes there are inconsistencies but nothing 3 lines of code won't fix.

------
mahmud
I have been stuck in an A/B hell myself, there is a fat sidebar to my left
that I can't get rid of. Googling for a fix turned up a page saying it was an
experiment and will be over soon.

For the last week I have been using Bing.

~~~
oneplusone
Delete your cookie for google.com. It drove me nuts too.

~~~
mahmud
I have some of the most aggressive security settings. And to be sure, I went
back and deleted all cookies again. No dice.

Well, this will give me an excuse to sample other search engines. I was a
heavy Altavista user until I found this new search engine out of Stanford ..

~~~
paraschopra
May be they were sampling based on user (you must be logged on?).

While doing such large A/B tests, they should definitely give an option to
opt-out. Or at least a prominent feedback button to tell them how bad is their
new design. It is as if Google will A/B test a new design and no-one will
notice it?

------
mr_justin
Nobody else in the world has anything even close too as many visitors as
google, so it's really hard to apply normal logic to this testing they are
doing. My feeling is that every single tiny and seemingly insignificant change
will have an impact on their bottom line. If it affects 0.1% of their users,
that's still a whole ton of people. I see nothing wrong with this multivariate
a/b testing.

------
Keyframe
Google is know for many things good, design not being one of them. Sure,
minimalism was their thing and it was good, but once they went in for design
it showed that they have no clue. Usability yes/maybe, but aesthetics
definitely no.

------
arihant
Talking about the new sidebar, I really think it doesn't blend very well with
the rest of the page. Compare this to the sidebar at www.clusty.com which
feels so much "at home".

------
swah
I kept adjusting the contrast of my monitor until I reailized the logo was
changed.

------
drKarl
I guess it's probably easy to create a Stylish user style to revert changes.

------
udfalkso
Who uses the Google home page anymore?

------
joubert
Experimenting on customers.

