

Show HN: Pushpin – A new way to build realtime HTTP and WebSocket services - 650REDHAIR
http://pushpin.org/

======
paulftw
Great idea, but all the marketing speak on the landing page just didn't make
sense to me. Things immediately fell into places when I saw the diagram in
the"About" page.

Pushpin is a proxy server that adds websockets to existing request-response
apis. Why don't you just say that straight away?

~~~
jkarneges
Thanks for the feedback.

One thing I've observed over time (the website is new, but the project is not)
is that Pushpin's value isn't immediately obvious from a one-sentence
description. So instead of starting out by saying what it does, I often like
to start out by discussing the problems that it is trying to solve.

But yeah, you're right. I, too, prefer when projects plainly say what they do.
I'll see about reworking the text.

~~~
lion_del
Yea. Front page looks nice, but does not explain to me in the terms i
understand (technical terms) what it does. Assuming your target audience are
developers/CTOs/etc I would re-think what text you have on your homepage.
Possibly even get some kind of diagram on the front page.

------
j_s
What is the intention of licensing the core server under AGPL?

Does the REST API represent enough of a boundary behind the Pushpin server to
separate the web application from the AGPL's requirement to give the source
code to public users?

Edit: Are there any widely-used proxies this tightly coupled with applications
that are released under a similar license? (Translation: is this normal?)

~~~
jkarneges
Yes, the intent is that the Pushpin<->backend interface is enough of a
boundary that you don't need to provide the source code of your backend
applications.

The AGPL is used to protect Pushpin itself. Open sourcing it all was a big
step for us. :)

------
jdoconnor
I saw a presentation on this last month, and found myself wanting to invent a
problem just so that I could solve it with this type of scaling. Great
presentation and great product.

------
foolinaround
Can you provide a real world example of an interaction, and how it would be
improved with pushpin in front of it?

~~~
jkarneges
Sure. For example, we made a tool called WebhookInbox that collects HTTP
requests. It has a realtime API that lets you know when requests have arrived:

[http://webhookinbox.com/docs/api.html#live-
updates](http://webhookinbox.com/docs/api.html#live-updates)

The above API has the quality you'd expect from a long-polling or streaming
API produced by companies like Dropbox or Twitter, except that the backend is
just a basic Django app. There's no crazy stateful event-driven code in
WebhookInbox.

So basically it makes realtime API development like this much easier.

------
z3t4
Trying not to offensive. Two years ago I would find this useful. But I'm so
glad I learned Node.js where this can be accomplished with high level sockjs
module and nginx in less then five minutes.

[https://github.com/sockjs](https://github.com/sockjs)

~~~
jkarneges
No worries! Pushpin isn't really about providing the easiest answer. It's
about radically changing how we architect and operate realtime web services.
In fact, Pushpin and Node are complementary, and Pushpin has a Node library.

If you're already using a language/framework capable of WebSockets or long-
lived HTTP connections like Node (or Tornado, or Go, or countless others) then
Pushpin's value may not be immediately apparent. It just seems like yet-
another-realtime-server thing, right?

The _long_ answer is that Pushpin restores the division between API designer,
engineer, and operations that we all enjoy in the RESTful API world but that
is currently non-existent in the realtime world. But, until you have many
servers, a multi-tiered architecture, and a team, this may be hard to
appreciate. ;)

------
jkarneges
Hey folks. The idea with Pushpin is to make realtime APIs easier to create.
Think of it like Socket.io, but aimed at API creators instead of application
developers.

Most realtime APIs today are implemented using low-level network code, because
until now that was the only sophisticated way to do it.

~~~
michaelmior
I found Pushpin interesting when I heard about it a couple years ago. Haven't
had the chance to use it yet, but I'm curious if you could give a summary of
what's changed.

~~~
jkarneges
Aside from a host of fixes/refactorings, the biggest change is that it now
supports WebSockets, with either a WebSocket origin server or HTTP origin
server.

Also, the backend protocol (GRIP) was refined to work via HTTP headers instead
of JSON instructions in the response body. This change helps simplify backend
code and makes it more readable. The response body mechanism is still
supported though, since it's more extensible and may be needed in rare cases
(such as status code 304 with Apache).

------
simonw
I absolutely love the API design here - running a realtime layer as a proxy in
front of a traditional request/response webapp (which controls the realtime
proxy through custom HTTP headers) is really elegant.

------
boundlessdreamz
This is fantastic! Thank you :)

For my use case, each client needs a separate channel which is short lived
(they are generating a report which is unique and takes some time). Would
pushpin have any problems with such a setup?

~~~
jkarneges
Shouldn't be a problem with Pushpin. There's no major resource overhead in
having lots of channels, and since channels exist on demand there's no extra
administrative work either. Just assert unique channels at hold time, and
publish to those unique channels.

------
kondro
I notice that you can have Pushpin talk directly via ZMQ.

I would love to see an example ZeroMQ-based Ruby "web server" that can be used
a backend for Pushpin to avoid the HTTP stack on the application layer.

~~~
jkarneges
The communication protocol is called "ZHTTP" which works in either REQ/REP or
PUSH/SUB modes. It basically wraps HTTP in ZMQ messages. We submitted a draft
here: [http://rfc.zeromq.org/spec:33](http://rfc.zeromq.org/spec:33)

I'm not much of a Ruby developer, but I just committed a Python example using
the REQ/REP interface. It should be readable enough so you can get an idea of
what's involved:
[https://github.com/fanout/pushpin/blob/master/tools/zhttpreq...](https://github.com/fanout/pushpin/blob/master/tools/zhttpreqhandler.py)

To activate the realtime stuff, you send the same Grip-* headers as you would
with normal HTTP.

A word of caution: while this path is more optimized, it is super bare bones.
As soon as you start doing anything advanced, you may find yourself wishing
you were using a real web framework (I know, because this happened to me once
already! haha).

~~~
kondro
Thanks for this, probably enough to get started.

The plan was to create a Rack-compatible injection point that uses ZMQ instead
of HTTP which could be used with any of the standard Ruby frameworks, rather
than getting too bare-bones.

I'm not sure whether there are any real _wins_ here though.

Although my real use case is to take queries from Pushpin and put them
directly into a persistent queue for processing by workers before them pushing
the responses directly back at Pushpin so that we can do a zero-downtime
deploy, even if the workers are down for up to 55 seconds (with long-polling).

~~~
rurounijones
[https://github.com/rack-amqp](https://github.com/rack-amqp) 's Jackalope
might be of interest to you. It uses AMPQ rather than ZMQ though.

~~~
kondro
Looks interesting. May be a good starting point, although it's ultimately
Pushpin that makes this model interesting to me.

------
kondro
I'm curious as to how much Pushpin is part of the Fanout.io stack?

Is Fanout.io just a (admittedly very affordable) hosted version of Pushpin?

~~~
jkarneges
Yup, Fanout.io is basically a hosted Pushpin service, among other things (like
Webhooks, Bayeux, etc).

------
klibertp
Meta question: is the use of "realtime" for this kind of thing already
accepted or is there some discussion about it? I ask because the other
meaning[1] is more natural for me and once in a while I get confused about
which meaning is intended in articles' headlines.

[1] [http://en.wikipedia.org/wiki/Real-
time_computing](http://en.wikipedia.org/wiki/Real-time_computing)

~~~
650REDHAIR
For anything web development it's generally acceptable to use the term
realtime/real-time, but it's really "near real-time".

------
NIL8
Whoever is responsible for your site's design has done an excellent job.

------
picsoung
That's pretty exciting. Good to see new players in this space :)

------
amccloud
Is this similar to Faye?

~~~
jkarneges
Sort of. Faye and Pushpin are both publish-subscribe servers. The difference
is that Faye speaks the Bayeux protocol with clients, whereas Pushpin is
client-protocol-agnostic.

Use Faye for simple JSON messaging to applications. Use Pushpin for developing
APIs.

