
The Evolution of Data at Reddit - yarapavan
https://redditblog.com/2018/02/28/the-evolution-of-data-at-reddit/
======
maslam
"Third party vendor". Is this Qubole? If so, would be nice to give him an
Internet high-five by naming them :)

------
RandComment
[https://www.flydata.com/blog/hive-and-redshift-a-brief-
compa...](https://www.flydata.com/blog/hive-and-redshift-a-brief-comparison/)

Used Redshift for our large data needs. No reason to look to anything else,
given our time-shifted data could easily be fit into Redshift. For larger
data, we would have to have some other solution.

------
danielvinson
This is extremely interesting and I'm eager to see the follow-up.

I'm currently planning something very similar at my company and any reference
to how other smaller-scale companies handle this when scaling is invaluable.
Are there any other blogs from other companies which have talked about similar
things?

~~~
dchuk
Not a direct answer to your question, but I maintain an HN-like aggregator of
engineering blogs here: [https://engineered.at](https://engineered.at)

These types of posts pop up there pretty frequently

~~~
overcast
These need to be sorted by publishing time, not by your database entry time.
You can tell because you've recently added Percona, and it's filling up the
top with backdated stuff. I'd also have preferred just a minimalist interface.
We don't need anymore voting/commenting stuff at this point.

~~~
dchuk
Interesting feedback. Not sure how I could make the interface more minimal,
it's pretty stark as it is.

The backdated stuff is weird I'll admit, but they also posted a bunch of
things in the last couple days so could be screwing with the ranking algo.
I'll check it out.

------
huac
> The tool we used did not support using Hive as a backend, and given the
> speed of even relatively basic Hive queries, it wouldn’t make much sense to
> use that for reporting metrics.

Did Reddit try setting up Presto? Analytics, not just visualization, with Hive
is really hard.

------
citilife
I've had a great experience with Hive!

One thing that has made me sad is that there is that Rails / Ruby has little
to no support. Many of the repos are over four to five years old.

------
mephitix
I think nowadays they use Microsoft PowerBI for dashboards/reporting

