

Ask HN: Where to turn for architecture questions? - partisan

Hi all,<p>I am currently transitioning away from the typical monolithic .NET application architecture and would love to bounce some questions off of people who have experience dealing with scalable systems.<p>Where does one turn for questions such as this?<p>As an example, I am looking into doing something like the Heroku architecture approach. The front end web servers receive an http request and route it to a worker to process and return the results. I would manage the communication through message queues. Is offloading the web server processing to a worker an insane idea?
======
mrits
I would start with building the app to test the throughput. Once you have a
repeatable routine to A/B benchmark each change then you are off to a great
start.

One you mentioned is not a crazy idea. A lot of high traffic websites are
loading data from a cache so processing might not be a big deal. If there is a
significant amount of work to do than getting a queue involved is a great
idea. Especially when you are sending a message to be processed and don't
actually care about the response.

However, keep in mind that traditionally one of the hardest things to scale on
a web site is the concurrency. If you are queueing messages to different
systems and leaving sockets open on your single web server you might run into
some concurrency problems.

This would lead you to do some research in non-blocking IO.

~~~
brudgers
How is this better than serving static content from a separate domain
distributed across a CDN?

By which I mean that if the problem is latency for mobile users in Tunisia,
putting a cache in front a server in Oklahoma doesn't improve the speed of
light. If `n + 1` joins contribute to the problem it doesn't necessarily solve
that either...though it might depending on the read/write ratio. And of course
the read/write ratio might make performance with a cache in front worse if
there are lots of cache misses and consequently flushes.

Without measurement and analysis, there's no way to tell what does and doesn't
constitute a _bad_ idea. The only crazy _architectural_ idea is changing the
system around without having measurements and analysis.

------
davismwfl
Honestly, my suggestion is to find a reasonably simple use case and to create
a quick sketched architecture.

For example, draw out an architecture where you take input data (emails or
whatever), queue it up, and have workers operate on it outputting analytics
concurrently. It is a fairly common and reasonably simple (at least initially)
use case to help you work through the types of issues, like communication
drops/lag, caching, queue contention, processing concurrency etc. From there
you can either write up some code to implement it in a simple form or start
asking more specific questions on specific forums. e.g. if you select Rabbit
MQ as the queue and have questions around it ask on their forum, if you use
Ruby or Node or .NET etc, ask on those forums for specific issues or ideas.

My 2 cents is nothing makes you learn how/why better than trying to do it and
having success and failures teach you where to go. And of course, researching
and asking questions to avoid repeating preventable issues/mistakes is always
smart. There are a number of reference architectures you can find online as
well. Study them to find the weaknesses and strengths (see what others said
too), the more you do that the more you will learn what to look for.

EDIT: below

I suggest my example or one similar over your initial idea simply to remove
some complexity so you learn the concepts first. As someone else pointed out,
once you add in the http component, possibly web sockets etc you are adding
another layer of complexity on top of a basic distributed system itself.
Hopefully I make sense here, cause it does in my mind.

------
brudgers
What problem are you trying to solve?

What is the business case for the approach you are advocating?

What is the business case for making any change at all?

