On: Ask HN: What is your weekend project?

[WmyEE0UsWAwC2i]

I didn't know i wanted this.

How do you scrap all that data?

[garysieling → WmyEE0UsWAwC2i]

I wrote up some more info here on data acquisition here (just notes right now) - https://www.findlectures.com/articles/2017/01/22/Software-Ar...

[WmyEE0UsWAwC2i → garysieling]

Thanks for the write up!

[garysieling → WmyEE0UsWAwC2i]

I started out scraping sites manually, and started automating more pieces (a lot of sites use wordpress, so they are pretty structured). I'm working on a talk on the subject, so I'll have an article soon that explains better :)