
Reddit 1.0 source code - dchest
https://github.com/reddit-archive/reddit1.0
======
notthemessiah
It's unfortunate that Reddit silently went closed-source before being forced
to admit it. Certainly a betrayal of trust with their long-standing userbase.

~~~
jschmitz28
Maybe having it be open source would represent a security risk?
[https://github.com/reddit-
archive/reddit1.0/blob/master/data...](https://github.com/reddit-
archive/reddit1.0/blob/master/data.lisp#L32)

~~~
Sylos
I mean, it shouldn't. Humanity is perfectly capable of building secure web
services without having to keep the way it works a secret. You don't publish
your encryption keys with your source code, which is what your security should
be depending on.

And what's more, Reddit themselves did not even use that excuse in their
official statement for it, even though to me their excuse felt even less
logical.

Basically, they don't want to leak the crazy features that they're developing
and have such piss-poor source code management that they cannot provide
tarballs of clean states of their source code.

I mean, how do they deploy new versions, if they cannot cleanly separate
feature development from stable code?

[https://www.reddit.com/r/changelog/comments/6xfyfg/an_update...](https://www.reddit.com/r/changelog/comments/6xfyfg/an_update_on_the_state_of_the_redditreddit_and/)

~~~
amelius
> Humanity is perfectly capable of building secure web services without having
> to keep the way it works a secret.

That's the same as saying that humanity is capable of building a large system
without bugs, in an economically viable way.

~~~
theon144
No, it's just the most basic argument against security trough obscurity - how
is this still a contentious point?

~~~
throwaway3233
The origin of that statement (Kerckhoff's principle) refers to cryptography,
not to application security.

If you take a quantitative, cost-based approach to modeling security through
adversarial capability, obscurity becomes a perfectly valid security measure
if it's not used _in isolation._ We don't use it for cryptography because the
tradeoffs aren't worth it. It's better to design cryptography with provable
security based on mathematically rigorous computational hardness assumptions
than it is to make secret algorithms.

In the context of application security, if the decision to obscure some or all
of your system incurs a non-trivial cost to an adversary, it makes sense. We
can't rigorously and mathematically prove the security of applications in the
same way we can prove e.g. an algorithm is sub-exponential instead of
polynomial time.

You often see "security through obscurity" mentioned in the same way that
people cite "appeal to authority" or "ad hominem" fallacies in internet
debates. The reality is more complex than that. Fundamentally, anything that
increases the effort required by an adversary to successfully compromise your
system is worth considering. You just shouldn't depend on it in its entirety.
Closed-source software is a good example of robust security through obscurity,
as basically any security engineer will tell you (I'd rather look at source
code line by line to find vulnerabilities than try to find them through trial
and error in a penetration test).

------
CodeArtisan
Why Reddit has been rewritten in python:

[https://redditblog.com/2005/12/05/on-
lisp/](https://redditblog.com/2005/12/05/on-lisp/)

[http://www.aaronsw.com/weblog/rewritingreddit](http://www.aaronsw.com/weblog/rewritingreddit)

~~~
pknopf
What is it like maintaining a large project in such a dynamic language? I
guess if you have proper unit tests and good coverage, you could quickly find
any runtime bugs that you may unintentionally introduce with
refactors/changes.

When in JavaScript, I'd always lean on TypeScript. Is there something similar
for Python?

~~~
hguant
Python 3.6 has typing/type annotations. It's opt in but seems to work fairly
sanely.

[https://medium.com/@ageitgey/learn-how-to-use-static-type-
ch...](https://medium.com/@ageitgey/learn-how-to-use-static-type-checking-in-
python-3-6-in-10-minutes-12c86d72677b)

------
nodesocket
Never played around with Lisp, so excuse the ignorance. Is this typical to
construct HTML in Lisp? This feels incredibly verbose and error prone, not to
mention confusing and hard to grok. Good luck having a designer mockup/write
HTML.

    
    
        (defun login-panel ()
           (pbox "login/register"
             (:form :id "logform" :class "nomargin"
               (:table :style "border-collapse: collapse"
                       (:tr (:td :colspan "2"  "username:"))
                       (:tr (:td :colspan "2" (:input :id "loguser" :class "txt" :name "user" :type "text" :size 15)))
                       (:tr (:td :colspan "2" "password:"))
                       (:tr (:td :colspan "2" (:input :id "logpass" :class "txt" :name "pass" :type "password" :size 15)))
                       (:tr (:td :colspan "2" (:input :id "logmem" :type "checkbox" :name "mem" "remember me")))
                       (:tr (:td :colspan "2" (:span :id "logerror" :class "error" "")))
                       (:tr (:td (:input :id "logbtn" :class "btn" :type "submit" :value "Login" :onclick "login(); return false"))
                            (:td (:input :class "btn" :type "submit" :value "Register" :onclick "register(); return false")))
                       (:tr (:td :nowrap t :colspan "2" :align "center" :class "little" (:a :href "/password" "what's my password?")))))))

~~~
geocar
> Good luck having a designer mockup/write HTML.

What you are looking at was designed by a designer.

Maybe you think another designer you know could make something that looks
better if only they could use their own tools, instead of tools that _this_
designer liked to use.

I remember an age when I would receive photoshop "designs" that I would have
to re-code in HTML. Then they got dreamweaver and thought they didn't need me
to code their stuff up anymore, just to debug their stuff and tell them why it
didn't work in X/Y/Z browser combination.

When the lisp programmer is writing their login panel, they start out writing
"code" like this,

    
    
        (defun login-panel ()
           (pbox "login/register"
    

then decide they need some HTML. `pbox` recognises HTML as long as it looks
like lisp so:

    
    
             (:form :id "logform" :class "nomargin"
               (:table :style "border-collapse: collapse"
                       (:tr (:td :colspan "2"  "username:"))
                       (:tr (:td :colspan "2" (:input :id "loguser" :class "txt" :name "user" :type "text" :size 15)))
                       (:tr (:td :colspan "2" "password:"))
                       (:tr (:td :colspan "2" (:input :id "logpass" :class "txt" :name "pass" :type "password" :size 15)))
                       (:tr (:td :colspan "2" (:input :id "logmem" :type "checkbox" :name "mem" "remember me")))
                       (:tr (:td :colspan "2" (:span :id "logerror" :class "error" "")))
                       (:tr (:td (:input :id "logbtn" :class "btn" :type "submit" :value "Login" :onclick "login(); return false"))
                            (:td (:input :class "btn" :type "submit" :value "Register" :onclick "register(); return false")))
                       (:tr (:td :nowrap t :colspan "2" :align "center" :class "little" (:a :href "/password" "what's my password?")))))))
    
    

can be transformed into the much more complex:

    
    
             <form id="logform" class="nomargin">
               <table style="border-collapse: collapse">
                       <tr><td colspan="2">username:</td></tr>
                       <tr><td colspan="2"><input id="loguser" class="txt" name="user" type="text" size="15"></td></tr>
                       <tr><td colspan="2">password:</td></tr>
                       <tr><td colspan="2"><input id="logpass" class="txt" name="pass" type="password" size="15"></td></tr>
                       <tr><td colspan="2"><input id="logmem" type="checkbox" name="mem">remember me</td></tr>
                       <tr><td colspan="2"><span id="logerror" class="error"></span></td></tr>
                       <tr><td><input id="logbtn" class="btn" type="submit" value="Login" onclick="login(); return false"></td>
                           <td><input class="btn" type="submit" value="Register" onclick="register(); return false"></td></tr>
                       <tr><td nowrap="t" colspan="2" align="center" class="little"><a href="/password">what's my password</a></td></tr>
               </table>
             </form>
    

But what is the alternative? To embed the html so that it still looks like
HTML means making a string of some kind, and running the risk that there's
some kind of variable expansion or XSS or other _thing_ that will bite them in
the butt. Maybe not in this simple example, but the need to get into the
display-language is very common in a web application, so there are lots of
fragments like this to answer to:

Maybe this one and bigger we could move into a file and either read it every
request or add some _extra code_ to save the contents to a global variable.
But we haven't gained very much for all that complexity -- this hypothetical
"when" we get a designer who knows HTML and not Lisp, and can contribute in a
way that meaningfully moves the project forward without making more work for
other programmers is far away at this point, and the reddit developers are
just going to rewrite the whole thing in Python anyway.

~~~
mratzloff
> But what is the alternative?

I mean, templates are a thing people use.

~~~
geocar
Can you explain what you mean, if it's not one of the things I mentioned (that
are often called "templates") and how it doesn't have the drawbacks for a
small team that I mentioned?

~~~
testb
I'm guessing they meant something like jinja,ejs,jade,etc. where you have your
html in a separate file and your lisp code would just render said html file
with the necessary variables.

I think you implied that in your last point but I'd be curious to find a
designer who knew lisp better than HTML.

~~~
geocar
> I'm guessing they meant something like jinja,ejs,jade,etc. where you have
> your html in a separate file and your lisp code would just render said html
> file with the necessary variables

That doesn't benefit our designer who knows lisp well enough, and it doesn't
benefit our programmer who knows lisp very well (or they would have done it).

> I'd be curious to find a designer who knew lisp better than HTML.

Well, if your team is a handful of lisp developers, then _all of them_.

Perhaps buried is the assumption that hiring/resources are free, or that there
is some kind of gatekeeping for the term and title "designer". If it helps to
make the point clearer: I program and I design, and I know lisp better than I
know HTML.

------
lifthrasiir
I chuckled over this bit of lines [1]. Fortunately (for Reddit) this name is
being consistently used throughout the code. Can't guess why.

    
    
           (email
            :type string
            :accessor user-emai)
    

[1] [https://github.com/reddit-
archive/reddit1.0/blob/bb4fbdb5871...](https://github.com/reddit-
archive/reddit1.0/blob/bb4fbdb5871a32709df30540685cf44c637bf203/view-
defs.lisp#L32-L34)

~~~
vzcx
Heh. If you see [1], they define a function called "user-email" (spelled
correctly). The lisp convention here would be to define the accessor as
something like "%user-email." While the code isn't too bad, these little
things betray a lack of knowledge of lisp lore. Perhaps another reason for the
rewrite was that the team actually didn't have much experience writing lisp
code.

[1] [https://github.com/reddit-
archive/reddit1.0/blob/bb4fbdb5871...](https://github.com/reddit-
archive/reddit1.0/blob/bb4fbdb5871a32709df30540685cf44c637bf203/data.lisp#L336)

~~~
kazinator
[https://github.com/Clozure/ccl/blob/master/lib/case-
error.li...](https://github.com/Clozure/ccl/blob/master/lib/case-
error.lisp#L51)

 _%assertion-failure_ doesn't look like an accessor.

[http://clhs.lisp.se/Body/f_car_c.htm](http://clhs.lisp.se/Body/f_car_c.htm)

 _car_ described as "accessor", not named _%car_. No public function in CL is
named this way. In fact, the character % doesn't make an appearance at all:

[http://www.lispworks.com/documentation/lw50/CLHS/Front/X_Per...](http://www.lispworks.com/documentation/lw50/CLHS/Front/X_Perm_9.htm)

I've never named a function or macro %something in 18 years of Lisp.

~~~
vzcx
Sure, sure. But while it's definitely not in the standard, I do see it all
over, and it's something I picked up from reading a lot of other people's
lisp. SBCL internals, for example. You might not like the convention and chose
not to follow along with it, but I would be surprised if, after 18 years of
lisp, you had never seen it before, and would choose to misspell an accessor
to prevent a clash instead of naming it differently.

Edit: ah, I think I understand. I don't mean to imply that all accessors
should be named in this way. That would be gross.

~~~
jjwiseman
To be explicit, the old Lisp convention I know of, which is not a super common
one but one I've definitely used quite a bit, is that you might name a
slightly lower-level/more primitive version of a function with a %-prefix.

So if we have a user-email function that is just an object slot accessor, We
could have %user-email that actually does a database query. I can't remember
for sure, but I wouldn't be surprised if I'd seen a double %%-prefix used,
too.

You don't always want to put those functions in separate packages
(traditionally in Lisp, packages are relatively heavyweight--e.g. an entire
web server framework might have only one or two packages).

Another similar example might be the convention of defining a macro with the
name my-macro, and a function that actually implements much of the macro, or
is an equivalent of the macro that takes a thnk, that is named my-macro*.

(24 years of lisp here--I wouldn't be surprised if the conventions of
mainstream lisp during the earliest six years were somewhat different than the
following decades.)

Edited to add: Now that I think about it, this convention may have been
heavily used in the Macintosh Common Lisp community.

~~~
kazinator
> _Macintosh Common Lisp community._

That's why I linked to some CCL code (descendant of MCL).

I use foo-impl and such for the implementation of foo.

If I have a my-macro, which is implemented by an expander function, then I
call that function expand-my-macro.

------
Mister_Snuggles
Is this really all the source code? It seems to be missing the database
schema, which is pretty critical to using it.

Beyond that, this is actually fascinating to look at. It's small enough that
you can actually understand it, but it's more complete than a simple 'toy'
application. I'm also fascinated to see how SQL constructs are expressed in
LISP. I don't know LISP at all, but it's pretty obvious how the queries in the
language map directly to SQL.

------
melling
Why Reddit switched to Python:

[https://news.ycombinator.com/item?id=3815491](https://news.ycombinator.com/item?id=3815491)

Of course HN is written in Lisp. pg is a big fan:

[http://www.paulgraham.com/lisp.html](http://www.paulgraham.com/lisp.html)

~~~
bringtheaction
Thanks, was about to ask what made them switch.

------
1wd
What kind of Lisp system was used to run this? Would it work with any Common
Lisp? What other things are required, like libraries or a database etc.?

I'm guessing reddit.asd lists some:

:tbnl (a toolkit for building dynamic websites)

:cl-ppcre (a regex library)

:trivial-http (a HTTP client)

:cl-who (a library for using S-expressions as HTML markup templates)

:clsql (a SQL database interface and ORM)

:clsql-postgresql (a PostgreSQL connector)

:cl-smtp (a SMTP client for sending email)

:ironclad (a cryptography library)

~~~
adwf
The interesting thing is you can still use pretty much that entire stack
today. The only one that stands out as outdated is tbnl which changed name to
hunchentoot.

The stack I used for a website just the other day:

hunchentoot

cl-who

postmodern (prefer it over clsql-postgresql)

cl-smtp

ironclad

drakma (fuller featured than trivial-http)

------
nathell
It's fun to look at the ASDF system definition: [https://github.com/reddit-
archive/reddit1.0/blob/master/redd...](https://github.com/reddit-
archive/reddit1.0/blob/master/reddit.asd)

Some Ediware in the dependencies (CL-PPCRE, CL-WHO and Hunchentoot from back
when it was called TBNL). I guess they used CL-SQL rather than Postmodern
because the latter didn't exist back then.

Should be reasonably easy to get it running on a modern CL.

------
kjdsiheioahef
Unrelated to the original post: Does anyone know if Reddit keeps or shares
records of content that ends up on the (unauthenticated, uncustomized)
homepage (top 30 results at any given time) for US-based users? If so, how far
back does such data go?

I don't believe the mainstream archive sites would be a definitive source.
Perhaps there is another?

------
forkLding
Is Reddit 2.0 the one in Python? Was slightly confused and expecting Python
when opening up the github repo.

~~~
SquareWheel
Reddit was lisp, then python, then python, then React (and some other stuff).

~~~
grzm
Can you expand the last one? The first three are server-side, while the last
is front end. Are they using node on the back end?

~~~
Figs
Recent post about it:
[https://www.reddit.com/r/announcements/comments/8830oa/and_n...](https://www.reddit.com/r/announcements/comments/8830oa/and_now_a_word_from_reddits_engineers/)

~~~
grzm
Cheers. Looks like this most recent work was primarily front-end:

> _" When we set out to rewrite our code to solve these problems, we wanted to
> make sure we weren't just fixing small, isolated issues but creating a new,
> more modern frontend stack..."_

------
rurban
With mail username and password

