Hacker News new | past | comments | ask | show | jobs | submit login

I had a similar problem that I solved with goquery and otto. You can use goquery to traverse the DOM and otto to execute the script fragment. Then just grab the data from otto's VM.

Your scraping being slow and using Chrome might be a blessing in disguise though. If you aren't careful you can get detected as a bot and banned from the site.

https://github.com/PuerkitoBio/goquery https://github.com/robertkrimen/otto




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: