Hacker News new | past | comments | ask | show | jobs | submit login

How is this at all sustainable as a business model? My company is in charge of scraping a few TB/week. Would we be allowed to use your service?

This is a browser extension that takes the place of copy/pasting from web sites (ie manual scraping), what you’re looking for is probably more automated.

I think this is all done client side, manually, through a browser extension. They aren't providing you with an API to retrieve this data live afterwards.

Still a cool project but I think this limits a lot of its utility. Providing an API to access the data as a service would be a lot more profitable.

Go watch the demo video. From what I can see it's not automated enough to do tb/week. That said I bet you could hook it into scrapy and automate it. Probably will be slow though

It's all client-side, it's not what you think, it's more a light scraping-on-the-go type product than a permanent enterprise solution. Good for what it does tho.

from a cursory look it seems that this software runs on your client, i.e. it's not a service.

Scrolling down:

" Can I scrape password protected stuff with Spider?

Yes! It’s a browser extension, so as long as you log in first, you can scrape whatever you like. "

It's just a browser extension

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact