
How We Lost User Engagement After a Redesign - rustoffee
https://icons8.com/articles/how-we-lost-47-of-our-users-after-a-redesign/
======
xtav
The thing that I'm still confused about is why they offered users the ability
to "Request an icon", when really, users would be _suggesting_ an icon. As a
user, I _expect_ "requests" to be answered. That is, I expect a special
service, just for me, that I'll probably have to pay for unless the site is
100% free.

Under that model of user expectations, I envision two groups of users: 1. the
users who want to casually _suggest_ an icon (and don't need it for anything
critical or want to pay for it themselves, but who would happily use it if it
became freely available) 2. the users who have a special request and would pay
to have it done, regardless of how many other users want it.

Group number 1 won't bother clicking on "Request an icon", because they figure
that the button is for members of group number 2. Group number 1 won't see an
option for them, which changes site perceptions, because group number 1 has an
impression that the site is for paid requests.

Group number 2 will click on "Request an icon", hoping to place an order, but
will be disappointed and/or confused by what they see. Members of group number
2 might note that "Oh hey, it'd be nice if I could get this for free". But
given that their need was sufficiently critical for them to want to pay
commission, the option to submit an idea that _may or may not be voted into
existence at some unknown time_ won't seem promising enough for them to invest
any more time in the matter.

~~~
developer2
You nailed the piece of information I didn't even understand from the article
at first. The feature is to _suggest_ an icon, not request one. _Requesting_
an icon sounds like custom work ("I'll pay for this"), whereas _suggesting_
one is clearly "here's an idea for you".

That one word choice would have made a great deal of difference to me as a
user understanding the feature. Yes there are glaring design issues with where
they went as well. But that one choice of word is huge.

~~~
m_mueller
How about "petition for an icon"? I find 'suggest' goes too far the other
direction, it's discouraging because it means nothing. But here, if I can get
a few votes from other users, I do actually get it. That's a petition for me.
Or maybe even 'start a referendum', but I don't know how commonly known that
word is.

------
Olscore
Just glancing at the design I can see that the whitespace is all messed up.
The previous design was more compact, one could easily view most or all of the
necessary information in a smaller space. But that is just glancing at it. The
newer design seems to be following the newer web motif of flat design,
generous space, etc.

Back in the 2007-2010 era of websites, somebody mentioned that "ugly" websites
work. Not that fancy or flat design is bad. But the sheer simplicity of ugly
websites forces people to focus on the content, context and functionality. The
ugly websites pointed out were MySpace and Craigslist mainly, but you could
probably add a few others like Drudge Report and Plenty of Fish. Even Hacker
News is pretty "ugly" but it obviously works great.

~~~
xenadu02
There is an ongoing trend of equating "simplicity/removing distractions" with
"hiding or removing information/making it less useful".

Make something as simple as it needs to be, _but no simpler_. Most designers I
see these days seem to forget that last bit.

In this specific example the whole purpose of the request system is to vote
and secondarily get other people to vote with you. The new design eliminated
almost everything that made the old one useful. It isn't simpler, it's
_simplistic_.

~~~
a-nikolaev
I may be wrong, but I think that this site's UI is far from simple. It looks
clean, but that's different thing.

The new interface feels very unconventional, in fact. It may become convenient
once you've learned what everything means, but until then you'd feel puzzled.

~~ Of course, I could not try to work with the old site, but the old
screenshots looked more informative than the new ones. Not surprised at all by
the results.

~~~
jandrese
What's complicated about it? Click on an article to read the article. Click on
the comments link to read the comments. Click on reply to reply.

It's hard to imagine something simpler. HN even maintains a good information
density.

~~~
a-nikolaev
I were talking not about HN, obviously.

------
tomsaffell
Looking at the 'before' and 'after' designs, I'd say that the biggest
difference is that in the old design the first question you ask (via a
textbox) is one to which the user already knows the answer ('what icon do you
want?'). Whereas in the new design the first question is 'please pick one of
these three options (with which you are not already familiar)'. So the
questions requires the user to _think_. See _Dont Make Me Think_ :) IANAUXD,
just my $0.03

~~~
dccoolgai
I would tend to agree. It runs afoul of the UX maxim "If you can't decide what
you want your user to do, they can _certainly_ not decide what to do." If you
really want them to pick "custom" or "fast-track" or whatever (for which there
is no coherent explanation visible in the mock I saw... fast-track? What does
that even mean?), bring it up _after_ they've committed in spirit to doing the
thing you want them to do (request an icon).

------
gervase
Even going into the new design with an understanding of what it was trying to
achieve, the tab title "Free for share" threw me off. It was so jarring that I
completely stopped reading the article and started trying to grok the UI.

Am I required to 'share' my icon in order for it to be made, like on of those
"Like on Facebook for coupon!" sites? Where can I share it? Can anyone vote
for my icon, or only people who see it from my 'share'?

I would suggest that simply "Free" would be a much clearer, with the
social/share component available but not required. Then, if they want more
people to vote on their icon, they can choose to share it or not.

"Free _for_ share" makes it sound transactional, like I'm paying for the icon
with a 'share', which is a concept I absolutely hate.

Okay, back to finish the article...

~~~
rconti
And, on top of that, the new options make you think that you're the
alternative. Instead of saying "hey, great, I can get a free icon!" you only
notice that you could also get a fast-track or custom (whatever those mean),
and you can't help but think "free for share" is the lowest priority thing.

Also, that entire page was impossible to parse. I had to go to their main
webpage to even try to understand the concept of what they're doing ,and try
to put the page snippets in context. Page snippets on a white background,
placed over another white background, are confusing as hell.

------
thaeli
I see that, as usually happens, the politics of website redesigns have
prevented the simple, user-centric fix of "go back to the old site which
worked fine." Like any other deployment, redesigns should have a rollback
plan. If you installed a major system update and suddenly production
throughput drops in half, you'd roll back to the old version. If you deploy a
new design and your traffic falls in half.. why is there so much resistance to
just rolling back to the old version?

I understand in cases where it's a big switchover and basically a redesign
riding the coattails of a huge middleware/backend rewrite. I don't understand
in a case like this when it wouldn't break major back office processes to just
put the old site back.

~~~
m_mueller
If anything, this should have been an A/B test.

------
asimuvPR
Imagine for a second if this service was provided through another medium.
Let's use radio for the sake of simplicity. The old design was full of
information. It was akin to a common radio station talk show. Few pauses. Lots
of information flowing in at every moment. The new design is akin to a radio
station talk show that takes a silent break every 5 seconds. The information
isn't quite flowing anymore. People lose interest. They change the station.
That's why they stopped commenting. They were _expecting_ and needed a certain
amount of information in order to use the service and join in the
conversation.

------
omgtehlion
I suppose this is not about desing (or redesing). Styles, whitespace, etc.

The new page is just confusing. What am I supposed to do on it? Nothing looks
like button or a hyperlink.

------
escobar
The title of this post is clickbait. It should be titled something closer to
"How we lost user interaction after our redesign".

They specifically mention that they did not lose any traffic/visitor numbers:
"Important: our overall traffic didn’t change. The same amount of people came
to this feature’s page."

It appears as though the "lost users" were really lost interactions with the
vote button: "our voting numbers decreased by ~50%."

It's fascinating to me how far people are willing to go when skewing the
information to pique interest. I'll bet a lot less people would read it
without "lost users" in the title.

~~~
OddsUXs
As a writer, I know quite much about clickbait name and use them extensively,
true that. However, in this particular case, the point is true: ~47% people
stopped using the feature (user = someone, who uses). Meaning the conversion
between people who found the feature and started using it has dropped by 47%
as well.

~~~
escobar
So, if traffic numbers are EXACTLY the same (as noted) but less people click
on a single button, you've "lost those users"? Doesn't make any sense to me.
You've lost interaction with a button from your unchanged user base. Like you
said yourself, click conversion from your users changed. Users changing
interaction ≠ losing users

If you had lost 50% of your users, you would have half of the overall site
traffic that you had last time.

~~~
jeffmould
Yes and no. I see what you are saying, but in my opinion every site can have
its own definition of user and visitor. For some sites the terms may be
interchangeable, while for other sites they have completely different
meanings.

Let's take HN for example (I don't know if HN considers this their metric or
what, just using it as an example.). Let's say they made a change that now
requires everyone to login to the site each time they visit. So if you closed
your browser window or were inactive for a period of time you would have to
log back in. While you could still read posts and comments, you would be
unable to submit new stories or comment until you logged in. While you are not
logged in you would be considered a visitor. But when you log in, you now
become a user.

While at first the traffic, or visitors, may remain the same, without
participating users on the site the number of visitors will eventually drop as
not as many people are submitting or commenting.

A site that does not require interaction, may on the other hand, simply use
the two terms (visitor & user) interchangeably.

------
minimaxir
The analysis seems more like a post-hoc rationalization of why the traffic
dropped, which, while fairly reasoned, may not provide the full story.

Specifically, did solving the issues identified _restore traffic back to
normal levels_?

~~~
OddsUXs
Going to apply fixes (or fight for them, in my case). And then publish the
results. Thing is, we're still getting requests and designers have something
to draw.

It's just me, who sees some drawbacks of this drop in the future. For others
in my company, it may be just 20 votes reduction.

~~~
zeveb
What's the business model behind drawing icons for people who get a dozen
votes?

~~~
OddsUXs
It's a fueling part of a bigger model: forming a community-driven product -
icon pack.

------
swalsh
So the proper way to do big changes like this is to support both versions and
to only roll out the change to select users. It's this radical new concept
called A/B testing.

------
xg15
I'm quite supposed another possible cause to the drop in user votes is not
mentioned in the blog that seems pretty obvious to me: _Where are the vote
buttons?_

Even if I already fully understand their system and are highly motivated to
vote, I wouldn't know how to actually do that in the new design.

In the new design, you have to hover over one of the numbers (without there
being any indication that hovering does anything) - after which a grey up-
arrow without any text appears. If you're familiar with reddit, etc and know
that particular language of symbols, and also know this is a voting system,
and you're not distracted by the gray color which could indicate "action not
available", you can make an educated guess that this is likely the vote
button.

In the old design, it was a button with the word "vote" on it.

------
ocdtrekkie
Cleaner/simpler isn't always better. It's better to be a little more
cluttered, if it's a lot more informative.

------
imron
What was the rationale behind the redesign?

Was there any sort of A/B testing done beforehand to see if the changes would
bring actual improvement or was it more just trying to mimic current design
trends?

~~~
matheweis
The "original design" was the UserVoice support system, a fact conveniently
missing from the article, which leaves us to speculating as to why they would
have moved away from that platform in the first place...

~~~
OddsUXs
The logic behind the redesign is not a mystery and was mentioned in the
article. Here I can elaborate it: 1\. To add several new features, UserVoice
wasn't flexible enough to provide 2\. To automate a few administrative tasks
(tracking requests, updating, uploading) - thus no connect the system with our
website backend more tightly 3\. To cover all these changes in a modern,
simplified version of the design.

------
hharnisch
Looks like you learned the hard way about hovering over to expose features. If
you've got touch screen users those features will be virtually useless as
well.

One thing that would have been cool to see your writeup would be how you used
direct customer feedback. Maybe it was none at all, but I suspect talking to
customers (or even internal employees) helped you reach some of the
conclusions you made. It's really difficult to look at a graph and go, yes
this is the thing that caused the change.

~~~
OddsUXs
What actually made me dig into analytics and numbers are usability studies in
the first place. They were planned as a routine I follow after every new
design needs to be tested, but made me reveal some deeper problems like
navigation, questions etc. Now, after covering the big picture, I maybe will
write an article on smaller ones.

------
gravity13
I feel like this is a classic example of abusing your knowledge about your
product. You gotta account for the fact that users will usually come in having
little knowledge about what your product is and will derive that from the
functions available on your page. In the previous designs, you look at it and
see:

\- user generated requestst \- crowdsourced voting \- how easy submitting a
request is, and what it entails \- the volume of requests you have

I see this in the larger context of icons8 and I get the sense that is akin to
voting for a feature. This is something I could do, if I wanted.

The new design shows me:

\- Weird titles that may or may not be some sort of request

And that's about it. Only five of them. Take that in the context of icons8 and
I really can't parse what it means so I would probably just completely ignore
it. On the request page, I don't see how approachable the request feature is,
so I ignore it.

Uservoice does it better.

------
allendoerfer
So essentially, they wanted to save $ 500 a month on uservoice.com and learned
the hard way, that software is not a simple as it looks. That is also why they
are not just rolling back but instead have used the opportunity to at least
get some PR out of it.

~~~
matheweis
Yea, it's not mentioned in the article at all - imho it should be - that the
"original design" is in fact the UserVoice system and not icons8's design at
all.

UserVoice has no doubt put a tremendous amount of effort into ensuring that
users interact with the support system as much as possible. Not very smart to
assume that you can re-do all of that work yourself.

Full disclosure: Happy user of the UserVoice system myself, grandfathered into
the free plan but otherwise no relationship.

------
guylepage3
Wow! Terrible redesign

Went from being Simple left to right, top to bottom style to some weird layout
that it very hard to navigate.

I would give that sucker a grade of “F”.

Great reminder to have users test products before shipping something.

As well as use common sense in your designs.

------
Animats
Compare the web site of the world's leading icon designer, Susan Kare.[1]

[1] [http://kare.com/](http://kare.com/)

------
EpicEng
And, of course, I'm greeted by a huge image that I have to scroll all the way
past in order get to the content. I'm not sure they learned their lesson.

------
r2dnb
People really like to make their life complicate, do not change the design of
an existing product. There are other ways to make life interesting.

On the one hand designs tend to be overcomplicate - any design that is
noticeable is overcomplicate - and on the other hand every 2 or 3 years people
feel a urge to fix things that work.

------
lotyrin
Why don't people ever test shit?

~~~
gravity13
Because they don't know how to. (so even if they do test it, they fuck that
part up too).

------
adrianggg
Amazon is great at A|B testing their way to a great user experience. You may
not agree just by looking at the UI but the metrics don't lie. Ok sometimes
they are misinterpreted or gamed. But an honest look at feedback will work
every time.

------
tmaly
stepping back and looking at the big picture, I get this same feeling about
many apps. Especially ones produced by Google, when you upgrade an app,
suddenly the UI is totally new and you don't know how to use the app in the
same way you did before.

Yes Google might not have chosen poor words in this case, but it does create
friction when you switch around the whole UI.

The example I always cite is when Microsoft switched the whole UI in Office
from the 2003 version to 2007 version. People who had mastered pivot tables
were suddenly up the river without a paddle

------
Wheaties466
I didn't know I was reading about lifehacker.com

------
joblessjunkie
Why not simply treat every search that returns zero results as a request, and
dump the UI entirely?

------
rocky1138
So... did they put it back?

~~~
thaeli
Nope, it's still the "new" design: [https://icons8.com/request-
icon/free/hot](https://icons8.com/request-icon/free/hot)

------
vit05
This could be the post of Snapchat team in one week.

------
eanzenberg
If it ain't broke..

