
Show HN: AnswersEngine – A flexible web scraping platform for Ruby devs - paramaw
https://answersengine.com
======
paramaw
Hello, I'm a co-founder here. I've recently built a web scraping platform for
Ruby developers. It is intended to solve the pain points of what I've
experienced over the years of doing web scraping for my clients.

Things that are just a hassle to do, are already taken care of:

Proxy and user agent randomization, so that you don't get banned easily.

Parallel scraping, that allows you to scrape at scale.

Page Caching. Imagine your scraper already runs for several days, and it
fails.... would you rather re-download the pages all over again? or would you
rather just parse from the cache?

HTML parsing using Nokogiri, and/or any other gem.

A pretty granular RESTful API, so that you can integrate it with your app.

Currently, it's in Beta. Feel free to join. I'm giving away $100 free credits,
so that you guys can test it out. Any feedback, is greatly appreciated.

