
Show HN: I built an auto aggregating bot, collecting trending funny pictures - rezashirazian
http://www.pixpit.com
======
wingerlang
Why are you watermarking the images?

~~~
rezashirazian
Because many people hotlink directly from the site. And If I'm going to host
the images myself and eat the cost of bandwidth I might as well derive some
value from it.

I provide a link to where the original images was taken from, so if anyone is
interested in the image without the watermark they can get it from there.

------
0xfff
Is this just a reddit scraper or am I missing something?

~~~
rezashirazian
I aggregate from Buzzfeed and 9gag as well, although I don't have them
actively running.

I try to give the images a clutter-free, mobile friendly, reliable, fast and
ad-free home. (I compress and host all the images myself)

I try to provide an easy mechanism for navigating to previous snapshots of
what was aggregated for anyone who is passively browsing and has viewed all
the recent images.

------
ksashikumar
How did you do this? What tech stack do you use?

~~~
rezashirazian
Ruby on Rails for the backend, Angular on the front. It's hosted on Heroku, it
uses cloudfront, S3 for storage, Postgres for DB along with httpClient and
mechanize for crawling and Paperclip for image manipulation.

~~~
ksashikumar
Do you use any kinda algorithm for crawling/fetching content from reddit?

~~~
rezashirazian
Not particularly. The one thing that helps but most people don't know about
reddit is the fact that adding a .json to the end of each url displays the
content of that page in json format.

for example: reddit.com/r/funny.json

This make crawling/fetching content from reddit much more trivial than old
school web crawling.

------
zackKidd
Is Reddit the only source? If so, do you plan to add any more?

~~~
rezashirazian
I have written a crawler for buzzfeed and 9gag but I don't have them running
because there is a lot of overlap between 9gag and reddit and they require
more work.

To be honest this was a weekend project I did awhile ago. If there is viable
interest I may pursue expanding it. But for now I'm going to keep it as is.

