

Ask HN: Questions about MongoDB and Redis for my use case - brandoncordell

(Warning this sentence may contain buzzwords that insight rage in some people) I'm building a social curation application. For the purpose of this question, let's just assume it works similar to pinterst.<p>I plan on using MongoDB as the main datastore of the application, and storing files on S3. I was thinking I'd like to add Redis into the mix so my first thought was to use Redis for caching.<p>What other features/aspects of (pinterest) would you move away from storing in MongoDB and move to Redis? Would it advantageous to store user follows/followings in Redis over MongoDB?<p>Would using Redis for more than a cache store be overkill in my application?
======
mryan
s/insight/incite/ :)

> Would using Redis for more than a cache store be overkill in my application?

It depends what scale you are at. I work for a startup that does something
similar (not Pinterest) and we make heavy use of Redis for all sorts of
things.

But IMHO you are looking at this the wrong way around - instead of "I need a
caching solution... I'll try Redis" you seem to be saying "I want to use
Redis... what can I use it for". Which is all well and good from a hacker
perspective, but looks a bit like premature optimisation to me.

Why not use MongoDB, and move things to Redis if/when they become a
bottleneck?

~~~
brandoncordell
Ahh I knew I should have checked for grammar fails! It was late when I typed
this.

As for "I want to use Redis... what can I use it for" that's exactly what I'm
asking. I think this project can gain some traction in it's targeted
demographic, but more than anything it's gaining experience with technologies
that I've been dying to try, so whether it fails or not I got some useful
experience from the project (Trying new technology, especially something like
Redis, just run on a local dev machine won't do much for getting the real
world experience).

I agree that it may be a bit of premature optimisation, so if you want we can
speak in hypotheticals. In an application like (pinterest) what are some use
cases for moving parts of the data from MongoDB to Redis? Would it be wise to
use something like Redis for follow/following and facebook style updates
(Bill, John, Mary, and 4 others loved this photo; Bobby, Rick, and Jane are
now following you; etc)?

~~~
latch
If I knew anything about pinterest, I might be able to help you more..

One area where MongoDB (and almost everything else) suffers is with a large
skip/offset value for paging. Getting the first page of results from MongoDB
or from Redis will yield similar results. However, the greater the offset the
worse the performance. In Redis, whether you get the 1st of the 1000000 page,
the performance is constant. So, if you are doing paging with large offsets,
at scale, a Redis sorted set is almost a must. It's a pretty specific use
case, but I think that's what you asked for :). YMMV on the exact #...you
should test it..but if, like 95% of all apps,only the first 5 pages are
important, it's really a non-issue.

Redis sets are great for follows..and you can do some nice set operations on
them (which you can't in MongoDB)...so something like which of my friends, and
which of your friends, are following the same people. However, if you want to
go more than 1 level deep (a graph), Redis isn't great (nor is Mongo for that
matter, but it's probably better).

Low hanging fruit for Redis would be analytics...both can handle basic
analytics, but Redis will handle it faster. For more complex analytics, the
two differ substantially...MongoDB will invovled writing Map Reduce...Redis
will require that you do the transformation for reports in code. Still, for
simple counters...Redis. In fact, maybe this will inspire you:
[http://blog.getspool.com/2011/11/29/fast-easy-realtime-
metri...](http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-
using-redis-bitmaps/)

