Hacker News new | past | comments | ask | show | jobs | submit login

EDIT: snippet from the website to help answer this question without requiring you to click. I didn't intend this to be a rude answer, I don't think it deserves downvotes.

" + No server involved (zero subscription fee for you!)

...

Unlike other web scraping softwares, it requires only one time payment to scrape for unlimited time and data. No more subscriptions or huge fee for your small data analysis projects! "




How is this at all sustainable as a business model? My company is in charge of scraping a few TB/week. Would we be allowed to use your service?


This is a browser extension that takes the place of copy/pasting from web sites (ie manual scraping), what you’re looking for is probably more automated.


I think this is all done client side, manually, through a browser extension. They aren't providing you with an API to retrieve this data live afterwards.

Still a cool project but I think this limits a lot of its utility. Providing an API to access the data as a service would be a lot more profitable.


Go watch the demo video. From what I can see it's not automated enough to do tb/week. That said I bet you could hook it into scrapy and automate it. Probably will be slow though


It's all client-side, it's not what you think, it's more a light scraping-on-the-go type product than a permanent enterprise solution. Good for what it does tho.


from a cursory look it seems that this software runs on your client, i.e. it's not a service.

Scrolling down:

" Can I scrape password protected stuff with Spider?

Yes! It’s a browser extension, so as long as you log in first, you can scrape whatever you like. "


It's just a browser extension




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: