
Why most clients will not get a scalable server for just $100 - p0larboy
https://medium.com/@p0larboy/why-most-clients-will-not-get-a-scalable-server-for-just-100-299d3b3003bf#.eq6ji7fdy
======
tracker1
TLDR: Outsourcing development is more expensive than doing it yourself if you
have better skills...

In the end, there's still some differences, real-time chat has different needs
than other types of applications. Scaling chat applications beyond a few
million users has it's own complexity... FB spends a _LOT_ of money to scale
it's infrastructure. Twitter had a lot of hard growing up in a similar space.

As a hobby endeavor for a free app, it's bound to be harder still. It's a
mixed blessing. Maybe time to get a corporate sponsor for the app.

------
wahern
I'm both surprised and not surprised that a mere 600 hits per second is
considered crushing, and needs a $4000/month budget to handle.

In a recently failed startup, I was able sustain a minimum of 5000 live audio
streams with dynamic, per-listener ad spots inserted into the stream server-
side (the idea being that ads can't easily be blocked by client-side ad
blockers). That's 5,000 unique, dynamically-generated audio byte streams; not
the same stream with the same ad spot broadcast to listeners, or with ad spots
being fetched out-of-band by special client-side code. And it was doing codec
and format transcoding so that the same URL generated whatever was natively
supported by the client (MP3, AAC, Vorbis codecs; ADTS, Flash, OGG, RTSP
formats).

On top of that, you could dynamically inject ad spots per stream, and during
each ad spot each context queried a single-threaded logic controller. So every
30 second interval there were about 10,000 small IPC messages passed over
sockets between the front-end thread and the controller thread in under 2
seconds. And the controller thread maintained state for each session so for
each listener you could see various real-time stats, including the list of ad
spots delivered.

And all of that that was on a simple E3 Haswell. The front-end ad splicer and
output format transcoder wasn't yet threaded, so the vast majority of the work
occurred on a single core of a single E3 CPU. Linux handled NIC IRQs on the
same core, so 5,000 wasn't even close to the limit. All-in-all, basically a
single core using a single thread to deliver 5,000 streams (with 5,000 * N
messages being generated at times) simultaneously using non-blocking I/O; much
more work than any chat app would ever be asked to handle anywhere under such
constraints.

(Though, to be clear, because it was live streams the codec transcoding didn't
need to be done 5,000 times, and the codec of the ad spots could be cached.
Rather, in this case there were only about 5 unique codecs for the incoming
live stream. The codec transcoding occurred on a couple of back-end threads,
with each thread multiplexing several input streams.)

The lesson I had to learn the hard way (_failed_ startup) is that good design
and good architecture isn't worth anything. The vast majority of companies
succeed with rather poorly written software, and by the time they hit real
scalability problems they've already "succeeded" in the most important sense
of the term.

So if you want a successful startup, use the environment that you're most
productive in and that allows you to churn out features the fastest. Worry
about scalability later, because if your competitor is first to market all the
scalability in the world matters for precisely naught.

