

Ask HN: Can you build a rock-solid massive backend (e.g. like Google's)? - sendos

Thinking of the backend of websites like Amazon, Google, Facebook, etc, where they have to serve hundreds of millions of users, have great response time, security, and uptime, I was wondering how many developers exist that can design and execute such a system? How common or rare is this capability?<p>I thought a quick poll here on HN might be a good indicator.<p>So, are you capable of designing and executing a backend that is on par with the ones used by the major players mentioned above?
======
kls
While, the systems I designed and built may not be considered Google or Amazon
scale, I have build some of the largest travel systems the web uses. I have
build systems that not only the biggest use with hundreds of thousands of
request per second (hotels.com, travelocity, Expedia, Orbits) but thousands of
other smaller clients where supported simultaneously. Anyways to get to the
point there where two major decision that we made early on that saved us years
and millions of dollars in frustration. Those two where statelessness and
decoupling.

Statelessness is important because it drastically simplifies your software
architecture and ejects the concern of scalability into the infrastructure.
This is a good thing because with a good automated install strategy new
capacity can be online in minutes (now with the cloud seconds). When you are
stateless you don't have to worry about inter-application communication among
nodes, rather you can have dead simple load strategies like load based, round
robin, load balancing where the most under utilized node gets the next
request. Statelessness is paramount to scalability, at least to cost effective
scalability.

Now on to decoupling, I don't get too caught up in the enterprise buzz word
bingo, but there is one that developers should pay attention to and that is
SOA. SOA is just a name for a good architectural strategy that has been
employed for years at the big guys. Summed up SOA is just a form of
decoupling. If a system does an independent task then that system should be
decoupled from the system as a whole. You should not think hey lets build a
reservation application you should think lets build an availability service, a
booking service, a concierge service. And these services as a whole will be
assembled together by a UI into a work-flow that allows people to make online
reservations. REST is just an extension of this in a well defined and
intuitive manner.

This allows the services to be used in new and novel ways, not only that but
it isolates the system and makes capacity planning very easy. If you see a
good deal of load on your availability service or your search service you can
scale that area with precision, allowing you to put the horse power where it
counts. As well with a decoupled system, that has well defined interfaces if a
portion of the system in unreliable the entire piece can be snapped out and
replaced with no effect to the larger system as a whole. It becomes very easy
to trim out the rot in a decoupled system.

Going further, and on a side tangent, a decoupled system is easy to bring in
contract development capacity in to work on it. You define the interface and
what the component should do, and you can hand those component documentation
off to a third party to build with far less risk than "hey build this whole
system". This becomes critical if you experience the two year growth rate that
we did.

I would say those two choices above all other allowed our small start-up the
ability to scale on demand and grow from a handful of niche client to
supporting every major travel system in a two year time frame.

Had we not made those choices we would certainly had a few year time-frame
that would have been painful while we redesigned the architecture to handle
the scale at which the business grew.

~~~
aitoehigie
How do you implement user access control, authentication and authorization
while being stateless?

~~~
kls
I am a big fan of SAML identity providers, they are flexible and robust they
also work with browsers and mobile clients and allow you to provide access and
authorization to all of the systems and services. I really liked Sun's
offerings in this space, OpenSSO. Oracle no longer supports it but a company
ForgeRock, formed out of former Sun employees an OpenSSO maintainers, took the
project and are continuing it.

I like OpenSSO because it provides a service layer that makes authentication
and authorization look like just another service offering on your bus. That
being said you do have to look at the architecture of the SSO provider some
are easily scaled and some are not.

------
jeffmould
The thing to remember here is that these systems, to the point they are today,
were built by teams of people over lengthy periods of time. There are many
pieces that have to be looked at from the wiring infrastructure, location,
network equipment, monitoring solution, server solution, backup solution,
security, OS, etc... While an individual may be capable of putting together an
comparable infrastructure with only one or a few servers, once you cross a
threshold with requirements and capabilities it would be almost unbelievable
to find a single person who could complete the project to the same scale.

------
spitfire
Isn't that the idea of hadoop? Build open source replacements for googles
infrastructure?

------
jet3june
Yes DuckDuckGo.com guy built a search engine by himself.

