
The Evolution of Reddit’s Architecture (2017) - Jaruzel
https://www.infoq.com/presentations/reddit-architecture-evolution
======
TekMol
The Reddit redesign must be the worst redesign in the history of redesigns.

It's super slow to load.

It's slow to use (Scroll is sluggish. It loads giant images and puts huge
autoplaying videos in your face. Comment function now starts an ugly html
wysiwyg editor).

It misses features (outgoing links in the sidebars are gone).

It takes away screen estate by having a fixed element on top of the screen
that stays there when you scroll down.

It's buggy (often weird overlaps between elements happen when you resize the
screen).

It's confusing. When I click on a post I get a view that shows the post in an
overlay of the page I was looking at. When I reload the page I suddenly get a
completely different view with only the post.

It's almost unusable on my iPad (where I have to beg for the desktop version
every time I open it because the mobile version is even worse).

It seems like they are still A/B testing it. Because sometimes I get the
redesign and sometimes the original design.

Funny enough, I _can_ see how the redesign might increase user engagement. The
endless scroll with content right in your face _is_ kind of addicting. It
pulls you in a state of "I hate this. And I hate myself for using it. But I
can't stop. I will just scroll down one post further.". So the experience is
_worse_ but the KPIs might be better. I bet this is how booking.com became the
hell that it is.

I would _love_ to see how the numbers of those tests look like.

~~~
protomikron
Agreed. And I wouldn't consider myself to be nit-picky regarding design, as I
mostly read Reddit for the content in specific subs.

Despite some questionable design decisions my main grip with the new design is
that it often just does not _work_ at all. I rarely alter default options in
software but I had to use old.reddit.com quite often to even _use_ the site:

    
    
      - sometimes the whole page does not work because of cookie issues
      - sometimes posts don't load (I have to manually refresh)
      - sub's sidebars missing: there was often useful information (might be the job of the moderators though)
      - unable to do local search (CTRL-F) due to endless scroll
    

Regarding my last point, Reddit is not the only culprit and many websites
unfortunately switch to this style - hopefully at least Google keep their
pagination - but I find it terrible.

With traditional pagination you can CTRL-F for keywords, go to the next page,
and repeat the process, but with endless-scroll content is usually loaded
dynamically depending on your viewport, so it's nearly impossible to search in
a reliable way.

~~~
headShrinker
> sub's sidebars missing

I'm a mod for a few subs and for that reason was forced to start using the new
UI. Most of the sidebar can be developed to look and work similar to the old
markdown sidebar, but without all the custom css styling.

I see the new UI as the a less dramatic version of myspace as the old, and
facebook as new.* This includes but not limited to stripping out custom CSS.
(You don't have to be a front end developer to style your sub anymore.) The
reddit ecosystem as a whole will now look and function much the same
everywhere. Some sub designs were getting out of hand. This gives Reddit execs
and admins much more control and the lowly mod a lot less.

Alas the reddit mod is further relegated the jerk the community maligns for
setting arbitrary rules and unfairly limiting free speech. Also a lot of the
mod tools are streamlined, link/user flair, user access/banning, etc. I really
don't like the look and feel of the new UI and after a month of constant use
still prefer the old UI.

*(Avoiding any reference to dig 4.0)

~~~
Crespyl
> Some sub designs were getting out of hand

Hey, /r/ooer is a treasure, and the redesign is a lesser beast for not
allowing its like!

------
Fri31Aug
Reddit is the only major site I know which is regularly (well not so often,
but often enough for me to realize) unplanned, and planned, downtime. Makes me
wonder whether their architecture is really something to follow.

~~~
saosebastiao
Without even looking at architecture, they went from lisp to python to node.
While I certainly wouldn't have chosen any of those languages, it seems like
every step they took was a step backwards. What's next, tcl/tk?

~~~
vladadaba
Why is that and which language would you choose? I am asking because I want to
get into web development and started learning React + NodeJS recently.

~~~
fianhe
React and Node are well engineered and in demand, plus you can even use it to
build mobile app with React Native, which is in demand too. Don't listen to
advices from anonymous random guy from the internet, doubt even me.

------
justtopost
The redesign effectively ended my use of this site completely. It is so slow
and agressively user hostile, it blows my mind anyonw would want to 'learn'
from it in any way but avoidance.

~~~
vezycash
Old.reddit.com

Bookmark the version of your favorite sub-reddits for mobile use.

Manually edit https:www.reddit.com/post to old.reddit.com/post

OR

Use a chrome or Firefox extension on PC to automatically redirect reddit URLs
to old.reddit.com version.

Problem solved for now

~~~
endisukaj
You can also opt out in the user settings if you have an account.

------
overcast
The great irony here, is that Reddit is making the same foolish mistake Digg
did with a redesign, costing them their entire userbase, and propelling
Reddit.

~~~
DerfNet
The difference, though, is Reddit has no peers. The closest I'm aware of is
Voat, which is basically the Isle of Misfit Toys for Reddit's banned hate
speech subs.

~~~
Sileni
Eh, I didnt really need a "peer" to replace reddit. After the redesign I just
stopped going for the most part; I'm here or on google news if I'm just
killing time. I'll still search reddit if I want to see a discussion about a
specific subject, but even then I'm mostly looking at 1-2 year old threads. I
feel like the content is in a feedback loop with the redesign, and they're
both killing the host.

~~~
overcast
Not sure your age, but I think a lot of us are just growing up, and we're
seeking more stable, intelligent news sources. I stopped using Reddit a couple
years prior to the redesign. HN and Google News satisfy my news niche as well.

~~~
vaughnegut
Out of curiosity, why Google News specifically? I stopped going on Reddit and
mostly come here or glance at Twitter for social media. If I want news I
normally go to a specific outlet's website.

~~~
Sileni
Going to specific news sites always ends up with that feeling of being
"captured" and hearing my own opinions repeated back to me. I'd rather read a
handful of Fox News articles a month and consider if they have any valid
points than just dismiss them entirely.

------
rcavezza
I just watched the entire video and it was very informative. Fwiw most of the
comments on here did not watch the video and seem a little uninformed. The
node apis were built by a small group of front end developers and most of the
pulled out services they're building are in python .

There were also a lot about message queues, cacheing, and ops related things
in the talk.

I definitely recommend watching the entire video if you're interested in high
traffic architectures.

~~~
sethammons
While those basic strategies are technically good for scaling, I think the
other commenters have a very valid point: it seems a lot of folks say, "no,
this strategy is not something that appears to be working because the site is
slow, buggy, and, oh, did I mention slow."

I think many folks consider scaling and fast a success and scaling and
slow/buggy a failure. Who wants to emulate failure?

------
pjc50
(Based on the transcript, thanks for providing that, I don't have time for
videos)

The summary would seem to be that the first generation of developers built a
monolith. A reasonably well scaling monolith, but only up to a point. Newer
developers fear the blob and prefer Node to Python, so new development has
been done in a cluster of menhirs of node around the python monolith.

The monolith suffered from lock contention because it was inadequately sharded
across subreddits. They have now addressed this.

(Not mentioned: the redesign which shifts a large amount of CPU usage onto the
client)

------
pg_bot
If I were to design reddit I would have enforced the following rule to handle
their scaling problems. You cannot touch the database when you are reading the
homepage. Instead every minute we generate the canonical list of ~500 links
that is "reddit". Cache the html for that and throw it on a CDN. The "next"
and "previous" timestamped links should reference when it was first loaded so
you can continue paging and avoid the duplicate link problem. You can cheat
for logged in users by loading in their data after page load with JS, but the
site would never fail entirely if it were overloaded. Votes should never
affect another record, they should always create a new record in your
datastore to avoid locking problems.

~~~
threeseed
I think you are really confused here.

The problem with Reddit isn't the subreddit home pages it's the article
comment pages. There are so many of them, people routinely comment/upvote
months old articles and the distribution is unpredictable so it can be
difficult to know what to cache.

And there isn't a datastore on this planet that could handle an event log like
what you are proposing. Reddit would receive hundreds of thousands/millions of
votes a second.

~~~
pg_bot
Nah definitely not confused about the architecture, it's fairly simple to wrap
your head around. What I'm advocating for is making the general site semi-
dynamic instead of fully dynamic. (Break expensive caches at regular
predefined time intervals instead of every time data changes)

The key insight is that reddit still functions as you expect if you are seeing
stale data from other people so long as your posts/comments/votes are visible
when you navigate.

Shard the databases by subreddit and you will be able to horizontally scale
the load. Use JS to post to a separate subdomain for each service. I.E
everything under www.reddit.com/r/funny points to funny.reddit.com which can
live as its own independent service.

Internally I would treat reddit as a group of individual high traffic sites
instead of one massively trafficked site. Then you have another service that
queries each site to build the main page.

~~~
cagenut

      Shard the databases by subreddit and you will be able to horizontally scale the load. Use JS to post to a separate 
      subdomain for each service. I.E everything under www.reddit.com/r/funny points to funny.reddit.com which can live 
      as its own independent service.
    
      Internally I would treat reddit as a group of individual high traffic sites instead of one massively trafficked site. 
      Then you have another service that queries each site to build the main page.
    

this is terrifying to read, in that this is exactly what the media companies
did for the last decade to try, and _totally fail_ , to compete with facebook.

having a unified platform is a competitive advantage.

also your "sharding and subdomains" trick is straight out of 2010 and a mix of
irrelevant (everything's sharded now) and anachronistic (h2 is everywhere
now).

also-also, you clearly have missed the best use case of reddit, which is to
have unsub'd from all/most of the "default" reddits and replaced them with
your own customized set.

you are optimizing for everyone reading the same thing (newspaper homepage)
not everyone reading their custom thing (social network). that has been a
proven losing strategy for a solid decade now, you could argue 2.

~~~
pg_bot
Everyone _is_ reading the same thing on reddit, just different combinations of
the same thing. Reddit is just a combination of multiple newspapers
masquerading as a single custom newspaper. No reason why the internals should
leak to what is shown to the user. You could keep the exact same user
experience that you have today and guarantee that the site never falls over.

You need to make the distinction between UX and Operations. Everything the
user sees still lives under the same structure as today but internally it is
structured so that you don't overload yourself and if you do fail, those
failures are isolated and don't take down the rest of the site.

~~~
cagenut

      Everyone is reading the same thing on reddit, just different combinations of the same thing.
    

this is categorically untrue, you don't seem to understand reddit.

    
    
      Reddit is just a combination of multiple newspapers masquerading as a single custom newspaper. 
    

that is exactly the anachronistic thinking that leads you down this path.

its movable type vs wordpress all over again. wordpress won.

the engineering constraint of pre-calculating caches is worse, in the long
run, than the occasional outage for casual/non-logged-in users. its optimizing
for the wrong audience, it is a long slow path to failure.

------
justinhj
I hate using Reddit on mobile. I’m generally reading on a bus or train in
Safari and it will virtually force me to use the app. 90% of those times it
will open the store instead and maybe want to update the app, or I’m not
logged in so I will have to take a few minutes to reset the password, even
though I’m logged into the web app. If the app does load it’s slow and buggy.
I wish I could make the mobile web page the default.

~~~
slazaro
Have you tried 3rd party apps? On Android I use Relay and it works great, I
assume on iOS there are similarly good apps.

~~~
justinhj
Thanks for the tip, I'll investigate

------
matkins
reddit.com's time spent on each request and uptime are shameful, and they are
not getting any better after the redesign.

~~~
Longhanks
The redesign is such a horrible user experience, the day old.reddit.com gets
shut down (assuming there is no Desktop program that comes close in quality to
the mobile app Apollo) is the day I'm going to stop visiting.

~~~
rando444
This same sentiment was echoed just as strongly after the last major redesign
(when they went from straight text to adding thumbnails and more whitespace).

The majority hated it, rebelled against it, said that the change in the way
content consumption was focused on images more than text would push out
quality content in favor of image macros that can be voted on quickly, that
the quality of the people using the site would drop, etc, etc.

10 years later, I still feel all of those concerns were valid and correct,
however these changes did not hurt reddit, and in fact the company grew by
leaps and bounds.

And even now, I still share the same sentiment as you and others, but I think
whether or not these changes are actually detrimental to reddit as an entity
are yet to be determined.

~~~
wycy
I think the problem is that people always, without exception, hate UI/UX
change, and companies know this and have learned to just ignore the objections
of their users. Every single change to the Facebook UI/UX, every single change
to Windows UI/UX, is met with an enormous backlash from users--"I just want it
to look and work like it used to!" Then the next redesign comes around, and lo
and behold, now people are attached to the current design that they previously
hated, and now hate the new design. Tale as old as time.

I've never been like this. I've actually really liked virtually all of the
widely panned UI redesigns over the years: iOS 7, every version of Windows,
every evolution of Facebook.

However, there have been 2 redesigns that even I didn't like: the Digg
redesign, and this Reddit redesign. Both are objectively bad. We know what
happened because of the former. It remains to be seen what will happen due to
the latter.

~~~
mattkevan
A redesign of a system you use regularly and have some sort of personal
attachment to is like someone sneaking into your house at night and
rearranging the furniture.

I work for a major UK community site and we get a furious backlash against
even the slightest of changes. And sometimes against things that users think
have changed but actually haven't.

(Very) generally we as humans hate having to think. Figuring out a change to
something we use regularly means all our automatic shortcuts no longer work
and we have to start paying attention.

It's like how annoying it is when they keep moving stuff round in the
supermarket but *100 worse.

~~~
emodendroket
Yes, but nobody would want to go back to Windows 3.1 now, and new users get
put off by old-looking pages. There can be real advantages to those redesigns.

