Hacker News new | past | comments | ask | show | jobs | submit login
Why Erlang? (smyck.net)
156 points by hukl on Apr 22, 2012 | hide | past | favorite | 70 comments

A lot of intros to Erlang miss an important point: Erlang is about fault tolerance, not concurrency. The concurrency is a means to an end -- if your systems has to survive process crashes, hardware failures, and so on, it needs to have multiple components that can supervise and restart each other. Unlike (say) node.js, in Erlang, error handling is priority #1. Its ability to juggle lots of concurrent connections is just a consequence of its priority on localizing errors.

Also, Erlang tries to address many failure cases that other languages/platforms do not; this isn't free. It's often a good trade-off (making systems resilient after the fact can be quite an undertaking), but worth noting.

I think this is a great article but misses some of what I think are the most essential points of what makes Erlang great.

Superficially, Erlang is Actor Model for functionalists. I could go on about the various advantages this model provides, but as other posters have so eruditely pointed out, MPI provides the same abstract framework for dealing with the problems that actors are intended to solve.

I'm not a professional erlang developer, but I've based and written a new NoSQL database thats intended to store billions of records in Erlang, altogether clocking in at less than 1000 lines of code, and after this experience, I can safely say that I will never try to build distributed apps, save some niche cases, in another language (even Haskell).

Here's what stands out about Erlang to me, in contract to comparable options.

Firstly, HiPE and BEAM may be the most advanced virtual machine systems in the world. People working with JVM will claim this an exaggeration, but in terms of reliability and speed, my experience with HiPE is unmatched.

Secondly, and perhaps more importantly, OTP.

Open Telephony Platform is the single best collected library for dealing with various problems that I have ever seen. The Generics (gen_server, gen_fsm, etc.) are a quick, fast and easy solution to problems and edge cases that would bog down even the best MPI developer. For example, dealing with internal state as a finite state machine is not new, but it's not nearly as popular as it should be, perhaps to this end it's apt to say that Erlang does for parallelism and distribution what Ruby on Rails did for web development -- It forces you to make intelligent decisions.

The systems responsible for controlling emergency services, global telecom infrastructure, many large MMOs, and countless other applications are Erlang. If you trust 911, you can trust Erlang.

I could go on, Hot code loading while old processes still depend on usable code? Built in failover? Function takeover? Automatic workload offloading? Good luck if it's not Erlang.

Erlang is a CSP system, first and foremost, and not actors.

[CSP style concurrency on JVM]: https://www.youtube.com/watch?v=37NaHRE0Sqw (Kilim, it should be noted, was benchmarked as scaling better than Erlang/OTP.)

http://lambda-the-ultimate.org/node/4350 (paper in top comment is worth the read.)

Disclaimer: I think Erlang/OTP rocks. But JVM has its moments, as well, and in fairness has earned the "most advanced VM" title.

Erlang isn't CSP by a long shot. If you take a look at


Note that Erlang is different in all three aspects: Processes have identity, messaging is not rendezvous and there are no channels on which messaging occur.

HiPE is the native code compiler, BEAM is the VM.

otp is fantastic, however

> Firstly, HiPE and BEAM may be the most advanced virtual machine systems in the world. People working with JVM will claim this an exaggeration, but in terms of reliability and speed, my experience with HiPE is unmatched.

Reliability, maybe, but these benchmarks disagree with 'speed':


It's not bad - it beats a lot of dynamic languages - but it's not great either. Go and Javascript V8 both beat it.

The Erlang VM tends to shine when it is given situations with an enormous amount of load and a high level of concurrency. None of the "ShootOut" problems really reflect where the time is spent optimizing the Erlang VM, so it is not that much of a mirror.

The BEAM interpreter is an interpreter though. And while it is rather fast, it is still just an interpreter and for purely CPU-bound problems, you have many languages, most compiled, that will beat it soundly.

Uh, is anything I said here actually inaccurate, or is somebody just downvoting because I said something "bad" about Erlang (which jlouis actually confirms...) ?

I hate this kind of mentality. If you can't look at things honestly, you'll never improve your favorite language(s).

I don't think you have said anything wrong, for the record. Go outperforms Erlang, mainly because it is a compiled language. V8 is a JIT-compiler (and it doesn't even interpret per se). In other words, those two systems are usually much faster than the BEAM VM when it comes to raw CPU bound computation.

HiPE does help a lot, but even with HiPE you can't easily beat a language with a static type system like Go: You have much less data to shove around when there are no type tags needed on most data.

Here are some slightly more detailed benchmark results of Go vs. HiPE: http://shootout.alioth.debian.org/u64/benchmark.php?test=all...

Note: Go development is done mostly on 64bit, and the 32bit port is a bit of a second class citizen specially regarding performance.

Is there anything specific that makes BEAM special compared to JVM? Would a "Clobure" or "BRuby" see any upshot vs. their JVM counterparts (barring library support)?

BEAM has this very interesting concept of scheduling, working from a very high level point of view as follow

1) there are a number of processes wanting to be executed

2) schedulers (generally one per core but is a configuration option) schedule on process and run it for a fair amount of operations. This makes systems with heavy parallelism run very smoothly since single processes won't hog cores but will only get a fair amount of scheduling.

On top of that the fact that the BEAM is directly aware of processes (actors) makes the system very transparent (you can look in depth into a single process and examine it's heap size, scheduling consumption google for erlang process_info to get an idea)

Very interesting with this regard are the slides about the erlang VM by Robert Virding http://goo.gl/PHfeh and from erlang "half word" VM talk by Patrik Nyblom http://goo.gl/2LD9S

Can anyone comment on writing erlang style apps in Haskell?

Haskell appears to have many of the required ingredients: Cheap multi threading, immutability and the Cloud Haskell[1] framework, with the added benefits of speed and the awesome type system.

I currently use eventlet and ZeroMQ in python to emulate a similar (and probably crappier) style of writing message passing concurrency apps. With the addition of protocol buffers this type of architecture is easily applicable to a multi language set up. I'd be very interested to see what others are doing.

Has anyone who's used erlang tried playing with Haskell in the same role?

[1] https://github.com/jepst/CloudHaskell

They're targetting slightly different points on the power/safety spectrum.

* Erlang: fault tolerance, distributed systems, absolute performance less important. * Haskell: correctness important, multicore+shared memory performance a main focus, distributed systems less of a focus (cloud haskell)

I wouldn't write my webserver in Erlang, for example, but my distributed object store, that's another story.

Yep, see my Combinatorrent and eTorrent projects, respectively. The long story short is here:


Thanks for the link, I remember reading it a while back. Have you played with the cloud haskell stuff at all? I seems you were able to achieve most of what you wanted with STM channels for combinatorrent - do you think that the presence of cloud haskell might have made any difference?

Cloud Haskell came after my work, so I do not know how it fares. I remember that I saw it and thought that there were neat ideas in there.

My guess is that it would simplify the amount of work I had to do in code since CH would subsume many of my lines.

What I gather from the slides:

1. server loads the data from S3 on start (sessions are sticky to a particular server)

2. user data gets mutated on that server for the lifetime of the session

3. the data gets written back to S3 on session end

How are failures handled in 2? If a box goes down, do you not only lose the open connections but also any data that has been changed in that session? If these are game sessions it seems like they'll be long-lived; do you periodically dump back to S3?

I've been curious about this for a while: If you can do concurrent processing so easily by message passing, why not just write Java or C++ that just passes messages around instead of sharing state? That way you don't have to learn a whole new programming language to do it.

You can do that, but as far as I know Erlang (not far, mind you) it takes care of a ton of details for you. For instance to send a message to another actor, whether they are on you machine or a remote machine you just need a PID (process identifier, an Erlang id concept similar to, but not the same as an OS level PID). In C++ you would either need to deal with local/remote differences in message transport (ignoring the non-trivial nature of serializing arbitrary types in C++) or create an abstraction that would do so on your behalf. Second, in Erlang all actors have their own heaps, so their local data is spatially close in memory by definition making actors NUMA/cache friendly (assuming the runtime is "doing it right") heap allocated data passed between actors in C++ wouldn't have that attribute by default (again, you could make it so with some effort, but it certainly isn't free). Third actors in Erlang don't use full OS level threads, so the overhead of spawning say 10k of them is not the overhead of spawning 10k C++ threads (also ignoring that C++ really only acknowledged the existence of threads in C++11). These are a few issues/benefits, but as I said you could do all of this in any language mostly, but in Erlang a lot of the gross/tricky details are already handled for you.

In other words, it would require Greenspunning Erlang.

Rob Pike and the rest of the (ex)Bell Labs/Unix gang have been on both sides of this for over 20 years: both with Squeak/Alef/Limbo, then switching to using C with libthread (http://man.cat-v.org/plan_9/2/thread not to be confused with pthreads, for details see: http://swtch.com/~rsc/thread/ ), but in the end having language support for concurrency is something that is really worth it, and one of the reasons the ended up building Go.

You can build a library for doing Erlang-style programming, but there are lots of features that you can't reproduce.

(1) In C++ you may get hard crashes, so to get reliability you need to use high-overhead OS processes. Erlang allows you to hundreds of thousands (yes, really) of processes because they are really lightweight (76 words + heap/stack). Also, copying within the same address space has much lower overhead (no system call/context switch) and serialisation is much simpler.

(2) Because of immutable data you can do have nice fault tolerance as follows. Imagine you have a request handler function that takes a state and a request. However, the request is malformed or triggers a bug in the handler. In a language that doesn't enforce immutability you have to restart the handler process because the state may be in an inconsistent form. In Erlang you would just discard the request (or send an error message) and handle the next request with the unmodified state.

(3) Pattern matching makes receiving a message very convenient.

I probably forgot a few things, but this gives you an idea.

The feature those languages are missing in order to do actors correctly, is enforced isolation of state. While you can emulate actors and message passing in nearly any language, you need this guarantee so that any actor can be executed anywhere at any time, without worrying that it is going to cause a race condition or hit a mutex somewhere. The big sin in Java and C++ which makes them quite unsuitable is "static"

It might be possible to achieve that isolation if you stick to strict coding practices, but the point where it's going to fail is when you import another library, written by someone else.

Because C++, Java and many other languages (including F#, Scala and others which have message passing libraries) do not have the enforced isolation of state, nor the ability to add notation to indicate that some piece of code is free of side effects (including all of it's dependencies), you're always going to have the risk of breakage when importing a library. The only way you can be sure that a library is actor-safe is to read it's source code - and if the source code is not available, then you're out of luck.

Java and C++ could have the potential to do actors correctly by adding the notation for side-effect-free code, using custom annotations/attributes on all functions which are actor-safe, and using some compiler extension or static analysis tools to prove correctness. I'm not aware of anything that does this for the mentioned languages though.

On the other hand, if you implement message passing in any purely functional programming language, your actors are automatically safe for free.

I guess it is easier deal with them and using them is more idiomatic in Erlang. For example you can write your own persistent data structures in Java and use them in your concurrent program. Actually you don't have to write your own, Clojure's persistent data structures written in Java and you can use them in Java, but these data structures makes much more sense with the rest of Clojure's tools, syntax, semantics...

See Akka: http://akka.io/

Keep in mind that in a language like erlang this is enforced for all it's users. So the whole eco-system is build around that premise.

I heard Jonas Bonér (Akka guy) talk about his motivations for building Akka. He said he started the project because he fell in love with Erlang but couldn't convince enough companies to deploy it. So Akka is a port is the essential ideas to Scala and Java.

That's what I've done on maybe my last 5 projects. In C++, Java and Python.

Not sure why this is so tough for people to understand. You usually don't need a whole new language to use a paradigm.

You don't need anything but assembler. Yet you just named three languages and not a one was assembler. Once we've admitted that the niceties offered by more specialized languages are worthwhile, that "you don't need a new language" argument becomes a whole lot less compelling. For all you know, maybe programming this stuff in C++ is like programming sequential code in assembler — dismissing it just because the tool you have can be made to do the same thing isn't bad (I'd never disparage people who accomplish things), but it doesn't make a very compelling case for anything besides your own comfort zone.

What you've just laid out is what I guess people call a "straw man". No one is arguing assembler over C here, just like no one is arguing the virtues of vacuum tubes over the transistor.

I'll say it again: the benefits of the concurrency model described in the original article can probably be achieved in peoples' existing/preferred dev env. It's easier and faster to look into that then to throw what you have under the bus for erlang.

If you want to get into logical parlance, it isn't a straw man, it's reductio ad absurdum. That is, you take somebody's reasoning and apply it to a different situation that illustrates the problems in the reasoning more clearly.

In this case, your thinking appears to be, "I can accomplish the same thing in C++, so why use Erlang?" My point is that an assembly programmer might just as well say, "I can accomplish the same thing in assembly, so why use C++?" The answer is that it's a lot easier and more natural to write OO code in C++ than it is to use, say, assembler macros. The fact that something is possible is less interesting than how simple and well-supported it is.

Again, I'm not saying your choice is necessarily wrong. For your use case, it might have been right. The benefits of using a language with pervasive and highly developed support for the paradigm might not outweigh the benefits of using C++ for you. But that reflects more on your personal circumstances than on the benefits of Erlang in general.

>In this case, your thinking appears to be, "I can accomplish the same thing in C++, so why use Erlang?" My point is that an assembly programmer might just as well say, "I can accomplish the same thing in assembly, so why use C++?" The answer is that it's a lot easier and more natural to write OO code in C++ than it is to use, say, assembler macros. The fact that something is possible is less interesting than how simple and well-supported it is.

What you seem to miss is that C++ also has some inherent value that cannot be replicated in Erlang. Namely: C++ is not assembly. You are already working on a higher level language. So the question is not "assembly or some higher level" but "how high a level should I go"?

Your response is essentially: "you should go to the highest level you can for concurrency, which (in your opinion) is Erlang".

The problem with that: you essentially reduce the whole problem domain to handling concurrency. Not what I call a good engineering analysis.

How about reusing HUGE EXISTING C++ libraries for his problem domain, instead of replicating them in Erlang?

How about reusing a ready team of C++ experts in his company, instead of retraining them in Erlang?

How about reusing existing tooling and infrastructure his company has for C++, instead of throwing it and using Erlang?

How about interfacing with external systems for which he has C++ drivers, but no Erlang ones?

You say: "The fact that something is possible is less interesting than how simple and well-supported it is". Maybe. But well supported is also not just a language attribute. How well supported it is within the industry, within his company, with his toolset, with the code he has etc?

You basically just restated the last paragraph of my comment as though it were something I haven't thought of.

I'm not trying to tell everyone to use Erlang. Getting things done is more important than the tool you use to do it, so whatever does that for you is good. But I'd be really amazed if somebody had used both Erlang and a C++ actor library and come out thinking that C++ was about as good as Erlang at its own game.

>You basically just restated the last paragraph of my comment as though it were something I haven't thought of.

Guilty as charged.

I tried to answer both you and the other guy in the same thread that advocated Erlang-over-C++ without making the considerations you made in your last paragraph, and I guess I should have added something to distinguish your two cases.

> the benefits of the concurrency model described in the original article can probably be achieved in peoples' existing/preferred dev env.

No, they can't. This has been well explained elsewhere. You'd have to rework the language/dev environment down to at least the C level if not assembly.

>It's easier and faster to look into that then to throw what you have under the bus for erlang.

No, it is slower and harder.

People think Erlang is difficult simply because the syntax is weird. Spend a couple weeks learning it and you'll be up to speed.

I think people are scared off by the syntax and so are trying to rationalize that they don't really need erlang.

It is possible to replicate erlang elsewhere, but you'd have to replicate erlang. You can't just add a library to ruby and get it.

The problem with the syntax isn't that it's weird. It's just ugly. Even Perl and C++ look nicer. It's the 1998-geocities-site-full-of-animated-gifs of programming languages.

I thought this way about Erlang code at first too, but after a short time I came around and now I think it can be extremely elegant and even beautiful.

> It's just ugly. Even Perl and C++ look nicer.

Totally disagree. Erlang is odd, but you have to understand what you're getting with that. Variable unification, etc., if very powerful (kind of like Haskell's pattern matching but a bit more powerful).

Why not just whip up a simple method dispatch mechanism in C and skip Java/C++ entirely?

edit: </internet sarcasm> ducks

Sure, if you like. I like (for example) the RAII and templating features of C++, but if you want to just use C go for it. I guess what I really mean is, what does Erlang offer beyond "concurrency is easy if you don't share state"?

I believe the point that oconnore was trying to make is that asking what Erlang offer besides message passing concurrency is like asking what C++ offers beyond RAII and templates. You could add that sort of functionality to C, but it's not going to be as nice as just using C++. A lot of the things which make Erlang ugly are safe guards that prevent the kinds of bugs that allow you to accidentally share state. For instance, if I send a pointer between distributed systems, my program is hosed. Any distributed C++ library is going to either be extremely limited in the kinds of objects it can send or, more likely, just declare via fiat that it's the programmers responsibility not to send anything with a pointer. That's okay until the moment that you import a third party library and you have to go through every object to make sure that it doesn't use a pointer somewhere as a private variable. Meanwhile, with Erlang, there aren't any pointers, so I never even have to think about this stuff.

Yes, which one can I use?

You could re-implement erlang in C++, but then you'd end up with erlang again. Why not just use erlang in the first place?

I don't believe it is possible to do real concurrency on the JVM, without rewriting key parts of the JVM.

> You could re-implement erlang in C++, but then you'd end up with erlang again. Why not just use erlang in the first place?

Because rewriting or shimming the 10MM lines of code in the libraries one depends on outweighs this one particular benefit of erlang.

> I don't believe it is possible to do real concurrency on the JVM, without rewriting key parts of the JVM

Define "real concurrency".

This one particular benefit is not available in any other language, unless you start with a relatively low level language like C and recreate it.

Multi-threading is doable in any number of languages, but the problems inherent with it are why people move to concurrency.

Concurrency needs to be supported in the language itself.

You'd spend 20 years recreating this in C++ vs 2 weeks learning erlang.

On a related note, I have heard of quite a few game companies that are using Go for their server side code.

That's quite interesting. I'd like to know what kind of games. Care to name names?

The server code for AirMech by Carbon Games is written in Go: see http://carbongames.com/ Basically, a modern Herzog Zwei.

not sure of specific games, but ngmoco has a Go stack they've open-sourced. They say they use it on their games, but I'm not sure which ones. https://github.com/ngmoco/falcore

I was recently talking with an Erlang developer working on the Battlestar Galactica MMO

I'm curious to know what type of CPU & Memory utilization there was on the ruby backends vs the erlang backends between the two games

He states that the Erlang server only needed a single box vs. the 80-200 for the Ruby; I'd wager the Erlang is dramatically lower in terms of resources consumed (or at least much more efficient)

In lthread, a coroutine lib, (http://github.com/halayli/lthread) you can make blocking calls inside coroutines. It gives you the lightness of event-driven, the advantages of threads, and the simplicity of sequential statements. And it can scale over cores.

Remember, Erlang scales between machines.

Erlang's actually all about reliability. It's the secret decoder ring to its design, not the actor model or message passing, which serve as the ways it gets to reliability. (Even to some extent its syntax. Excepting comma-period-semicolon which is just dumb.) And one of the principles of Erlang is that you don't have reliability when you are running on only one set of computer hardware.

lthread allows you to do so by message passing.

The problem with lthread is that it does not really provide blocking calls inside coroutines (you simply cannot do this with current OSes like Linux or Windows). Just take a look at the docs:

If you need to execute an expensive computation or make a blocking call inside an lthread, you can surround the block of code with lthread_compute_begin() and lthread_compute_end(), which moves the lthread into an lthread_compute_scheduler that runs in its own pthread to avoid blocking other lthreads. lthread_compute_schedulers are created when needed and they die after 60 seconds of inactivity. lthread_compute_begin() tries to pick an already created and free lthread_compute_scheduler before it creates a new one.

It just does the blocking call on its own pthread. That simply does not scale the way Erlang does.

One day there is hopefully an OS where each file, socket, device, etc. is an actor with a message queue. Then you can really be concurrent :-). Even Erlang can then be more concurrent...

I wrote the docs. :) . It really does provide blocking calls inside a coroutine. The coroutine can totally block on a non-socket resource (CPU calculation, disk IO in case you aren't using aio) without effecting other coroutines.

I do agree that actor models implemented at the OS level is much cleaner, but when it comes to CPU hogging sections, messages queues are not the answer.

Frankly, I'm surprised (but I shouldn't be) that so many people are desperate for an alternative to Erlang. Going so far as to try to pretend that libraries bolted on to other languages give you the same thing at lower cost. This is impossible, and you'd understand that if you understood what erlang does.

Stop that, right now. You're supposed to be hackers. New technology isn't supposed to scare you this much.

Erlang doesn't use the C style syntax. BIG WHOOP. Yeah, it looks "weird" at first but don't be scared off by it.

You can't get the fault tolerance (and consequential scalability) of erlang in any other language/platform. Period. Full Stop. It simply doesn't exist.

Yeah, theoretically, one could build it with some low level language like C. You whipping up crappy C++ code using an "actor" library is not the same thing, by a long shot.

So, unless you want to spend 20 years reinventing the wheel, why not just use the wheel. Yeah, I know its round and you're used to square.

I promise you that you'll really enjoy the much smoother ride.

Don't be a blub programmer. Learning erlang takes you about 2 weeks. The syntax is perfectly fathomable once you put in those two weeks.

But be serious about it. Do a chapter each night from the armstrong book.

This really is one of those languages that separates the hackers from the hacks. Not because its difficult, but because the hackers are not scared of it, while the hacks are.

Seriously. Stop rationalizing.

I would highly people learning to look at: http://learnyousomeerlang.com/

Erlang really is the simplest of any serious language I know, after a little of erlang you will be pining for it when debugging some accidental misuse of globals in javascript.

Saying that no other environments will be able to provide the same isolation and elegant message passing is wrong, erlangs vm is good but it is not magical, but for now it does it best.

While I wouldn't disagree with your points, I think you've chosen a fairly narrow view of reasons people might want an alternative to erlang.

An obvious case you've omitted, is that people already have existing codebases written in other languages, and rewriting them all in erlang is not an option. Attaching an actor library to "X" language can give us many of the features offered by erlang with our existing codebase, and allow us to continue using our existing skills and paradigms.

Erlang does an excellent job at solving certain issues, but it's lacking in many other areas that "X" might do a better job of. If my project needs both the fault tolerance and concurrency of erlang, and feature "A" which language "X" is good at, but erlang not so good at, then choosing erlang all the time might not be the right solution.

I'm personally a fan of polyglot programming. We should pick the most suitable languages for each task, then we wouldn't need to keep re-inventing programming languages and libraries to emulate other languages. It's quite an idealistic view, but there's certainly ways we can improve the interoperability between languages, including erlang's - which is quite limited so far.

An example of such approach is the (now discontinued) Axum project from Microsoft Research. It offered many of the ideas of Erlang, but built onto the .NET framework and largely compatible with existing code. Axum could call, and could even contain almost any C# code inside it, although it was a restricted version of C# which had isolated states, and no static variables - something that high standard codebases omit anyway, regardless of the whether language allows it, because it's been known good practice for a long time that shared state causes problems.

The Actor Model existed long before Erlang, and will feature in many other languages other than it - whether or not they are successful is a different matter, but programming language designers would be wise to have a good knowledge of Erlang, and many other languages, so they're not re-inventing the wheel badly. Concurrency and fault tolerance should be considered from the start.

I personally think we can do better than Erlang - or at least offer the same without sacrificing every other paradigm to achieve the aims of erlang. (And Haskell will probably be the language to better it).

Since when do you have to rewrite everything in Erlang? Erlang is happy to call out to code in C (or whatever) via ports, which are a way for Erlang to run untrusted code in isolation and restart it when it crashes. Unlike (say) Ruby or Python, buggy C libraries can't bring the whole system down in Erlang.

Erlang is indeed impressive. But there is a big learning curve (at least for me!) and a whole new tool-set to support. It would be great if a Ruby/ZeroMQ based framework modelled on Erlang became widely adopted.

Most of the learning curve really is OTP. And as with Cocoa, the only way to skip those is to reinvent them, or not even reach an understanding of the problem they're solving.

I'd expect an equivalent toolset on a different platform to have an equivalent learning curve if it existed at all.

Not exactly what you're asking for, but check out Elixir, a Ruby inspired language that targets the Erlang VM:


Big learning curve, pretty piecemeal support for lots of things compared to popular languages, and a mental model that is difficult for less advanced programmers spell something that will likely never be all that popular.

Here's something I wrote about it 5 years ago:


Check out Akka Actor framework in scala. It does actors very well. Asynchronous and highly vertically scalable.

Celluloid/DCell are going that way.

That wouldn't really work. You can't just bolt on concurrency support.

IF you spend about 2 weeks working on erlang, it becomes really easy to use.

The learning curve is not nearly as steep as it appears-- its just that the syntax initially looks so alien that its quite scary.

I know the feeling, I went thru it, but it is well worth it.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact