Hacker Newsnew | comments | ask | jobs | submitlogin
epi0Bauqu 1487 days ago | link | parent

About 6 months before I launched something, which was really a MVP, if that. I've invested in infrastructure, but not a ton. I crawl some from the cloud and some from my own machines.

leej 1486 days ago | link

How do you do the crawls without getting banned? Are you using like 80legs or AWS for from cloud crawling?


jacquesm 1487 days ago | link

Would you qualify it as a meta-search engine or as a search engine ?


epi0Bauqu 1487 days ago | link

I classify it as a search engine. I'm in to doing whatever produces the best results. I started with my own crawler before BOSS existed and then switched that piece to it (highly modified of course) to concentrate on where the marginal benefit lies. Techcrunch initially classified it as a "hybrid search engine," so maybe that is a better term?


Lists | RSS | Bookmarklet | Guidelines | FAQ | DMCA | News News | Feature Requests | Bugs | Y Combinator | Apply | Library