
Architecting Back end for a Social Product - kilimchoi
http://highscalability.com/blog/2015/7/22/architecting-backend-for-a-social-product.html?utm_source=feedburner&utm_medium=twitter&utm_campaign=Feed%3A+HighScalability+%28High+Scalability%29
======
cihangirsavas
having worked on & maintained & architected 5 different social backends and
failed at 4 of them, here are my key take aways;

data storage: * keep data structure as simple as possible, * do not go with
the hypes, stick to old?, proven technologies * have your db constraints as
strict as possible in early stages, later you can remove them as for
performance improvements * test the key features of your database choices(does
transparent sharding really works?), you will see them failing... * you dont
need a graph db, you need a graph-like access layer to your data * your >1
month old data wont be accessed/modified at all (mostly) chose your shard key
accordingly * keep duplication of your data as small as possible in early
stages

indexing: * >1 month old thing might not apply here * ACL can be
problematic/hard to manage, try to keep it simple.

queuing: * to me this is one of the most important elements, if you want a
simple way to keep every component in sync with others, use event publishing,
really. * have your re-try mechanism, dead letter queue, delayed processing in
place.

> We can use ActiveMQ, which is the most reliable queuing software. a very
> bold statement :)

caching: * think your cache invalidation strategy from the beginning * keep
your immutable and dynamic data cache separately in your code (at least
visually) * try not to mix your business logic and cache code

> We should be ready for anything in the order of billions of queries per
> seconds. if only you are next/already facebook...

~~~
manig
Thank you for reading this and sharing your thoughts.

------
mianos
Pretty good and much the same sensible architectural choices as many other
already successful companies aside from the neo4j choice. Just use a normal db
like postgres like everyone else. There are good reasons why the biggest sites
in the world use or have moved to using an RDBMS. They may not be cool but
they work. I'll not expand on all the reasons but a trivial search will
enumerate them extensively. You already have some cool stuff like mongo, a
store I love, and hate. ActiveMQ? I can't think of anyone else using it. Is
there a good reason why everyone seems to be using rabbitmq?

~~~
manig
Thank you for reading this and sharing your feedback. Well this is more of a
template to get started. It is not really "one size fits all" solution. I
agree, instead of getting carried away by all these no-sql solutions, one may
get started with RDBMS and then as they grow, may take a decision to move
towards a no-sql solution or what ever solves the problem. But from my
experience I can tell you that, if things go "as-planned" in terms of data and
traffic, soon one will have to move a part of their data to a no-sql store.

>> ActiveMQ? I can't think of anyone else using it

I have used ActiveMQ multiple times, and found it to be a stable MOM. I love
its scheduler support and some of its cool features like message grouping,
virtual destination, composite destinations and wildcards. Of course, like you
said there are many other alternatives too.

