
Backbone.js, now realtime with Pusher - KarlFreeman
http://blog.pusher.com/2011/6/21/backbone-js-now-realtime-with-pusher
======
bradleyland
I'm having a hard time wrapping my head around why someone would platform-
source something as tightly coupled as client-server push services. I can see
not _developing_ it in house -- something like Socket.io comes to mind -- but
signing up for a service to proxy communication between your app and the
browser for "push" services seems a little bit on the lazy side.

Pushing complexity out to specialized platforms is a popular trend these days.
PaaS vendors are popping up in every area, but there's a threshold where this
type of outsourcing makes sense.

* How complex/specialized is the problem?

* How close is it to our core product/competencies?

* What are the comparative costs in development and maintenance vs using the service?

These are the core questions I ask when I'm evaluating PaaS vs in-house
solutions.

Q: How complex/specialized is the problem?

A: Unlike HTTP, WebSockets aren't widely supported, so real-time browser
communication is a complex problem. However, it is a generalized issue. It is
not a problem that we will have to solve alone. There are probably libraries
available to assist in the implementation of our own "push" stack.

Q: How close is it to our core product/competencies?

A: A core feature of our product is real-time exchange of pricing data between
clients (browsers) and the server. The faster clients receive feedback on
their pricing, the better our product works. This means real-time browser
communication is central to our product. It is not core, but it is very, very
close.

Q: What are the comparative costs in development and maintenance vs using the
service?

A: An initial review shows that there are popular libraries that abstract a
significant portion of the complexity away, including fail-over for browsers
that don't have support for WebSockets. Availability of commercial support
isn't clear, but source code is available. We will have to build our own API,
but we have the benefit of tailoring it to our application, rather than using
a generic API. Back-end message queuing services will need to be added to our
monitoring suite.

In a more general sense, we also evaluate how the use of PaaS would affect our
product in the event of downtime. Every PaaS you use in your application
spreads your exposure out over their systems as well. Can you afford for
"push" to go away? What will a fail-over system cost you? We opted to make
real-time communications part of our app, so the redundancy we built in to our
app hosting also covers our real-time services.

For my money, I just can't see outsourcing something like client-server "push"
communication.

~~~
miksago
Hi, I work for Pusher and wrote that article, so I feel the need to respond:

When people ask "Why would I use Pusher instead of running my own realtime
services in-house?", they often overlook the other complexities involved in
the problem. Eg, scaling, maintaining, ensuring uptime, etc.

Looking at any service (IaaS, PaaS, SaaS) from the outside is an iceberg
problem. You can see the public API (REST based API + WebSocket API), and you
know what it does for you, however, what you don't see is what lies beneath
the water, the other factors and things that have been thought about for you
so you don't need to think about them.

The thing that really needs to be thought about is this: by building your own
in-house realtime solution, you'll probably face similar problems to those
that we have to face building and maintaining Pusher, such issues might
include: 1a) How will you communicate with your realtime service? Will it be a
REST API (like Pusher) or something else? 1b) How will you maintain the
infrastructure required for the persistent connections and consistent
distribution of messages throughout your cluster? 1c) How will you handle
sudden (unexpected) growth in usage, and do you want to have to worry about
that? 1d) Who will ensure that you're up-to-date on the latest websocket
specification (we're at draft-07 being released in firefox 6, and draft-09 is
currently being edited).

I should note at this stage, that hosting websockets is in no way similar to
hosting standard HTTP, there's a lot more to the problem.

By the same logic, you could apply that to any PaaS or SaaS or IaaS product.
Do you use Amazon Web Services or another cloud based host, or do you use a
non-cloud based host? Thats a place where you're outsourcing the management
and maintenance of your infrastructure; that is, unless of course you maintain
your own data-centers and install all your own servers?

Having built a websocket server for Node.js, I know the issues involved with
maintaining and keeping up to date with the latest protocol specification. Is
it cheaper for you to use Socket.io instead of role your own solution, because
honestly, something might go terribly wrong within Socket.io. Do you want to
spend time maintaining that?

This is all about choices, for some projects, it may make sense to handle
everything where you need absolute control over every aspect of your
application and infrastructure. However, from experience, I can say that I
probably wouldn't want to be having to maintain absolutely everything in my
application. If AWS/EC2 goes down, then yes, I should worry, however, I think
EC2 is less likely to go down than my own hand-installed network
infrastructure.

I'm not going to try to sell you on Pusher, however, just take into full
account absolutely everything before you decide to roll your own realtime
solution.

– Micheil.

~~~
bradleyland
I'm really happy you responded :) My critique was more of the decision making
process than the product, so I'm really happy to have the feedback. I try to
always assume that my thinking is flawed, so after I read your post, I tried
to go back and read my own as if someone else wrote it. I know that sounds
silly, but it often leads to insights I wouldn't otherwise have.

You make a good case for the complexity, which is one of the factors I
consider. One of my unspoken concerns is that an open source library like
Socket.io is going to get stale. I keep an eye on the commit log, and it looks
like development is pretty active, but a look at the impact graph shows that
there was a lull, and it looks like development might be changing hands. This
is par for the course with open source projects, but I question whether "par
for the course" is best for our project.

My other big concern is running "yet another server". We're currently running
a very standard Rails stack on VPS infrastructure. We're not on a full-blown
"cloud" provider, because I don't trust them. Heroku has had too much downtime
for our company. AWS offers multiple AZ redundancy, but there's an inherent
complexity there that we're not ready to tackle.

Unlike most startups, our revenue generating traffic isn't spread out over
thousands of visits. Our revenue is made during a purchasing event that runs
for a half hour. Some days, there might only be a single event. If our
infrastructure is down for that event, we stand to lose $25k-$50k. I know
we're not the only one with this problem (some would lose millions over the
same time period), but we're a bootstrapped company. A major outage like the
ones Heroku has experienced could put a serious hole in our cashflows.

Because of this, we run a very straight forward Rails hosting stack
(Apache/Passenger/MySQL) on a straight forward Xen VPS host at data centers in
Dallas and NYC. I fully expect that at some point in the future, one of these
data centers is going to disappear off the net and we'll have to be ready. If
our entire hosting provider disappeared, I could bring up every one of our
customer app instances in a half-day. I know this because I've done the drills
using local VPS builds from scratch, and we use third-party, off-net backup
locations.

Any XaaS product we integrate in to our core product would have to demonstrate
a similar focus to uptime and redundancy. The recent AWS outage that took
several high-profile websites offline is a good example. Lot's of seemingly
smart companies hadn't deployed across multiple AZs because of the complexity.
I can't partner with a company that makes those kinds of decisions.

I know you don't want to try to sell me Pusher, but I'm genuinely curious
about Pusher's commitment to uptime. I'd love to see a blog post, or knowledge
base article that talks about Pusher's redundancy and disaster recovery plans.
I'd love to "just trust" that XaaS companies have this sort of thing covered,
but history has taught me otherwise.

Thanks again. I really appreciate the response.

~~~
pubnub
You are smart to ask these questions. Reliability is a serious topic that
<http://www.pubnub.com> fits in. I work at PubNub and believe it is the best
offering for reliability and uptime being a globally distributed service.
During the EC2 outage on the East Coast USA, we lost our servers there.
However! Our automated redundant routing system transferred traffic to the
surrounding nearby healthy data centers. Customers didn't notice the switch,
and went about their day.

