

CSS changes that increased Reddit account creation by 200K yearly users - alexis
http://www.donotlick.com/2015/01/05/8-increase-in-reddit-account-registrations/

======
Fogest
How does this actually indicate it had to do with the redesign? Sure the trend
may be going up, but the site is increasing in popularity so that makes sense
that it is going up. There are more people there to interact with and more
communities people can join that they may be interested in following and
discussing things in.

I am just wondering how this rise is attributed to the CSS change? It may have
had some effect, but I'm confused how this single graph says that it is
because of a CSS change?

~~~
richardwhiuk
In fact, from my reading the rate of signups was increasing before hand and
continued increasing afterwards, implying the design made no difference.
That's what I can see from the trendline anyway, and the rest of the graph is
too uneven to draw any conclusions.

~~~
eCa
They are wrong to draw the trendline the way they did. Since what they are
claiming is not steady growth (then the trendline would make sense) but a
_shift_ in behaviour they should instead have drawn one line for average
before the change, and one line for average after the change.

It is quite obvious that something has changed. Before the change there are
five days above 7.5%, and after there are more than 10 days above 7.5%. The
valleys are much higher too.

~~~
cbsmith
> It is quite obvious that something has changed.

Yup, but that's not an 8% increase in AR's. That's _maybe_ an 8% increase in
the _rate_ of increase in AR's. That could be attributable to a LOT of things
though (there isn't even enough context to control for network & seasonal
effects, let alone whatever increased attention there might have been to sign
ups after the rollout).

Ironically, probably the best way to isolate the impact to the launch is to
narrow the time window and resolution to the minutes before and after the
redesign launched.

------
toddmorey
One quibble: there are way more than 'CSS' changes here. They removed extra
wording, removed the CAPTCHA, and generally reorganized the window.

~~~
megablast
Well, if there were lazy they could have just hidden all those other elements.

------
AdamGibbins
They removed the CAPTCHA, I'm not surprised.

~~~
extrapolate
Quote from a previous article (so I wouldn't attribute all 8% to bots):

    
    
      We’re experimentally removing CAPTCHA and using other methods for spambot detection.
      We know it’s annoying for humans, and are starting to worry the bots are actually enjoying it.
      Plus, our visual CAPTCHA was failing on accessibility for non-visual redditors.

~~~
jbeja
What other method for spambots could be used?

~~~
bobbles
In addition to plenty of stuff I cant think of, you could see:

1) High percentage posts flagged as 'reported'

2) High percentage posts flagged as 'spam'

3) High volume of posts containing URLs

4) Posts containing identified spam messages (like how an email spam filter
may work)

5) Multiple accounts from similar/same IPs

Almost all spam in a subreddit I mod for is from accounts < week old. (I
believe there may already be a limit on how much you can post after
registration though)

~~~
ubernostrum
I mod a subreddit with >100k subscribers. We use AutoModerator and have it set
to remove anything posted by an account less than a day old.

It catches some false positives from people who sign up just to post
something, but it's also _drastically_ decreased the spam. Most spambots are
still at the "sign up and then blast shotgun-loads of posts before the
algorithms catch you" level of behavior.

~~~
tokenizerrr
Do you review the removals and undo those for legitimate users? If not, wow.
Talk about being hostile towards new users.

~~~
ubernostrum
Most people never notice. The ones who do get an explanation of what happened
when they ask about it in modmail.

It can be rough on a brand-new account that wanted to jump into a discussion
right away, but at the same time the quantity of spam hitting your subreddit
once you pass the 100k mark is just ludicrous, and this is by far the most
effective way to stop it.

------
TomGullen
Sorry but this chart: [http://www.donotlick.com/wp-
content/uploads/2015/01/newdesig...](http://www.donotlick.com/wp-
content/uploads/2015/01/newdesign.png)

Doesn't seem convincing evidence to me

~~~
rcfox
I agree. While it does show an average increase of ~7.5% after the line, it
also showed that before the line! It's hard to tell that anything happened.

------
underwater
And yet it is still impossible to use on a mobile device, due to a crappy
custom dialog implementation.

------
baddox
It's refreshing to see what is to my eyes clearly a cleaner and more modern
design actually performing better than what looks like a kludgy FrontPage
design. Too often these analyses purport to show the opposite.

------
chestnut-tree
The before-and-after image shows a much less cluttered sign-up form

[http://www.donotlick.com/wp-
content/uploads/2014/11/cheat.pn...](http://www.donotlick.com/wp-
content/uploads/2014/11/cheat.png)

However, the Nielsen Norman Group (NNG) recommend _against using placeholder
text in form fields_. They list 7 main reasons. This is taken from their
website

1\. Disappearing placeholder text strains users' short-term memory

2\. Without labels, users cannot check their work before submitting a form

3\. When error messages occur, people don’t know how to fix the problem

4\. Placeholder text that disappears when the cursor is placed in a form field
is irritating for users navigating with the keyboard

5\. Fields with stuff in them are less noticeable.

6\. Users may mistake a placeholder for data that was automatically filled in

7\. Occasionally users have to delete placeholder text manually

Obviously, not all of these will apply to the Reddit sign-up form (or to your
own website). It's worth reading the full NNG article to get a different but
valid perspective based on their research:

[http://www.nngroup.com/articles/form-design-
placeholders/](http://www.nngroup.com/articles/form-design-placeholders/)

------
pimlottc
Submission title is a bit clickbaity and does not match the more modest actual
article title, which is "8% Increase in reddit Account Registrations".

------
xpose2000
I wish I had a screenshot of the original version. It was absolutely awful. It
seemed like the code wasn't touched in at least 5 years.

Nice job with the redesigned version, but in all seriousness... I wonder what
took so long? Every few months a company should take a spin as a new user and
find ways to refine the new-user process.

Use this as a reminder to take another look at your registration sign-up
process in 2015.

~~~
fixie
There is a screenshot shown in the linked previous article:
[http://www.donotlick.com/2014/11/17/redesign-of-reddits-
logi...](http://www.donotlick.com/2014/11/17/redesign-of-reddits-loginaccount-
creation-window-and-reddit-comlogin/)

Direct link: [http://www.donotlick.com/wp-
content/uploads/2014/11/cheat.pn...](http://www.donotlick.com/wp-
content/uploads/2014/11/cheat.png)

------
vii
This is a typically poor example of ignoring the complexities inherent in
analysing new user acquisition.

Suppose you implement a change that causes the registration form to crash for
some subset of people? I am sorry to say for a while I was responsible for
causing such issues at a very large service ;) Depending where the crash
occurs and where you are counting signups you may actually get a massive boost
in your metrics, as the victims retry with every attempt met by failure.

Suppose you correct an issue making it easier for people to login? Then lower
registrations - the key here is to make sure you have a good metric to look at
(e.g. accounts created that are interacting well with the site after as long a
window as possible).

~~~
JungleGymSam
The article is about an 8% increase in _account registrations_ not hits.

~~~
vii
Exactly.

An account registration is basically like a hit metric.

The creation of an account does not imply that it will be used or the person
who created it has not created many more either accidentally or for example
because they could not log in.

------
supercoder
Just imagine what would happen if they applied better design to the rest of
the site !

~~~
megablast
The subs have a lot of control over what is presented. I think they often look
worse, but that is all a design choice.

~~~
JungleGymSam
Design is a choice? Radical thinking.

------
sideproject
A bit more data would be nice to corroborate the claim. I understand some of
the criticism in the comments. :)

If you are looking for some ridiculous correlation claims, check out the
spurious correlation.

Here's an example,

The number of people who drowned by falling into a swimming pool correlates
with the number of films Nicolas Cage appeared in.

[http://www.tylervigen.com/view_correlation?id=359](http://www.tylervigen.com/view_correlation?id=359)

:)

------
theVirginian
IMO the design of Reddit's website could use an overhaul. I can't even snap it
to one side of my screen without totally screwing up the front page and forget
about using the site on mobile. The sidebars contain way too much unnecessary
information as well. I do love me some reddit though.

------
micheljansen
Shame that they didn't do some A/B testing to more accurately quantify the
effect.

~~~
mahouse
They probably have the same answer they have for the "Why can't I search
comments" question: they can't afford it.

------
tuna
any way to check which ones are real people and which ones are bots creating
accounts for some reason ? maybe their activity posting vs subreddit
popularity... I guess you may be surprised.

~~~
xiongchiamiov
We ran some analyses and determined the rise is not due to an increase in
spambots. I don't know the specifics of how that was determined, sorry.

------
outsidetheparty
That chart is a textbook example of "we drew a trendline through noisy data
and drew completely unwarranted conclusions as a result".

------
jbeja
Yeah, that what a properly modern designed account form given today standards
can do this days.

------
herbig
I'd say they're correlated for sure.

~~~
brent_noorda
My sarcasm detector is having trouble. Was your comment sarcasm? Was the
original article and accompanying graph sarcasm? Is it sarcasm to say that
unix time is incrementing at the same rate in 2015 as it did in 2014, and
therefore it being 2015 is responsible for the increase in unix time?

~~~
herbig
No. As many of the other comments have here have said in more words,
correlation != causation.

