Hacker News new | more | comments | ask | show | jobs | submit login
Stripe open-sources Einhorn (stripe.com)
173 points by gdb on May 29, 2012 | hide | past | web | favorite | 32 comments

This looks extremely useful. We have several EventMachine proxies that I'd love to make redundant and load-balanced with something like this.

I took a couple minutes to try to get one of them "einhorned"; I lost most of my time to trying to satisfy EM gem dependencies (we use em-http-request, which has a fussy gemspec). I got Stripe's EM built, file descriptors passed to my proxy, and EventMachine::start_server set up with the fd instead of the address; I'm getting accept callbacks, but not received data.

Will tinker later. High hopes. Thank you very much for open sourcing this.

No problem! Feel free to ping me (gdb@stripe.com) if you run into issues.

Are you hanging around in the IRC somewhere?

I lurk around #stripe on freenode usually.

Depending on your platform, how many children you've forked, and your rate of incoming connections, this design could expose you to a thundering herd.

You might find that you get better performance by having the parent call accept() and hand off sockets to the children. With sufficient abuse of LD_PRELOAD you can probably do this without making any changes to the child processes -- you just need to override accept() with a function which checks if any sockets have been handed off.

That's a fair point. Even if your accept() implementation is fine, event-driven workers would instead be select()ing on the shared socket, which can also lead to a thundering herd. I think you'd need quite a few worker threads before this became an issue in practice though.

Indeed, it's an edge case... but it's a painful one if people do run into it.

(The following is off-topic:)

I noticed that the Stripe blog doesn't allow comments. I think most people find it odd or frustrating when they can't add their 2 cents to something (for better or for worse!). Any idea which things might have driven this decision?

- Not wanting to deal with how most people act on the net? (Or maybe they're in a highly competitive space where the juggernauts wouldn't think twice about trying to destroy their image.)

- To encourage people who want to discuss the blog entry to submit it to their favorite discussion communities?

- To avoid possible PR issues sitting right on their company's blog? (Related to point #1)

There is a kind of beautiful simplicity to the page that comments would probably destroy.

I made the same decision on my personal blog. When there are comments both on a blog and here on HN, the conversation feels unnaturally fragmented to me.

Further, when I did have both, comments that self-selected the blog (vs HN) are - in my opinion - much more likely to be spam/promotional, or generally less 'valuable' (e.g. more likely to be simple questions that are already answered in the post).

There's an angle I hadn't considered. The spammy and promotional type of comments would be far more likely to appear directly on the blog since that's where the value (for the commenter) is in leaving them. That's something that moderated comments would control, but you can't moderate your way out of fragmentation.

I would expect more negative comments to be left directly on the blog as well since the commenter won't have to deal with the potential backlash from their community. Which is probably also a good reason to never allow any kind of anonymous of guest comments. (Although that wouldn't stop the truly motivated negative commenters.)

Moderation can also get tricky. Pure spam is simple but you will run into comments where the commenter somehow works in their (product || experience). So long as they provided some commentary it is not pure spam. So the situation becomes a bit touchy.

This is a problem we ran into at PhpPhotoUploadr. In our case we have a very talented team so we hacked together a solution.

PS: We're always hiring 忍者!

Makes sense. Then maybe it would be a good idea to link to the HN discussion at the end of each article?

I also thought about it. Think about your blog - your posts can be: articles, essays, discussions. I chose essay, and removed comments. In a sense:

"This is what I have to say, I've wrote it and re-written it, and reformulated it and I think its good for publishing. I then published it and if you have comments, send me an Email and I'll consider amending my post".

Am I to understand that Einhorn doesn't require the use of Unix sockets to delegate work to workers? If so, it could work quite nicely for JRuby apps as Java doesn't support Unix sockets and it's therefore not possible to just plop nginx in front of the Web server (Tomcat or Jetty) instances and have it queue up requests to a shared Unix socket like the typical Unicorn setup does.

That's correct. In fact, at the moment Einhorn only supports TCP sockets. (That being said, I might be mistaken, but I think that Unicorn can also bind TCP sockets.)

Thank you for the follow-up. Even though Unicorn does support TCP sockets, it is irrelevant for our JRuby apps as I believe (IIRC) it won't even install under JRuby due to being written in C only, with no native-Java or pure-Ruby alternatives available. So it's not a Web server that can be used for serving JRuby apps.

Now, since you've extracted the capabilities of Unicorn out into something that isn't coupled to a Web server (and it's written in pure Ruby, so is portable), we should be able to use it as a shim between nginx for managing Trinidad (Tomcat) instances. Very much looking forward to giving it a try - thanks for your efforts!

EDIT: Whoops, guess I spoke a bit too soon. I get this error:

  NotImplementedError: fork is not available on this platform
I think a typical workaround on JRuby is to use the posix-spawn or spoon gems. If I have some time this next weekend maybe I could test this a bit more and maybe formulate a patch. Example posix-spawn usage [here][1].

[1]: https://github.com/ddollar/foreman/blob/master/lib/foreman/p...

Ah, interesting. Yeah, would love a patch!

Awesome news. Still waiting for you guys to come to Canada so I can give you all my money!

is stripe only available to us customers trying to buy through your site? or only us startups, and all of their customers?

Only US merchants. Customers can be anywhere (except North Korea, Iran, etc. presumably).

The logo is confusing me a bit. What's the connection between icosahedral gems and unicorns? Is this some pop culture reference I'm missing or is it basically a stopgap considering that Unicorn itself already has an eponymous logo?

Yeah, it's mostly a stopgap.

Does this work for long-lived connections? Say I've got 1000 TCP connections open to my backend workers and I want to roll out new workers. Will the old connections die or survive the config change?

Yep, it will indeed work. New workers will be spun up, the old workers will get a signal asking for a graceful restart, and those old workers can hang around for as long as they need. You'll have twice as many workers alive as you might like, but there's not much Einhorn can do about that.

I'm interested to see some more posts about Einhorn. I would like to see some example use cases for it. Looks like a very interesting project.

i'm going to fork Einhorn and call it Finkle.

Finkle and Einhorn... Einhorn and Finkle... Finkle is Einhorn... Einhorn is Finkle...

Einhorn is a man!

Dude you're too late, Finkle is Einhorn and Einhorn is Finkle.

I just heard of Unicorn / Rainbow as a way to get more out of Heroku workers/dynos. I wonder if this is also compatible, or a better alternative?

I'm new to this whole concept, so am just about to get my feet wet.

Hopefully, Heroku will implement something like this internally, as it seems it would help them to eliminate the greatest flaw with their system - that each deployment require 5-10 seconds of downtime as new dynos are started.

To my knowledge both FastCGI and Unicorn are not free. This is definitely an excellent alternative to those - and probably even easier to use for some use-cases. I am glad and thankful that Stripe decided to make it open-source.

Actually, Unicorn is indeed free (you can grab the code from git://bogomips.org/unicorn.git -- looks like it's GPL). FastCGI is an open specification, with many free implementations. There's definitely a lot of cool prior art in this space.

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact