Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Our customers do things like the following:

1. Monitor online ad distribution

2. Build directories of people on the web

3. Check for IP violoations (digital watermarking, etc.)

4. Build niche, localized search engines (picture searching, regional searching, etc.)

5. Collect information on product pricing, retail data, etc.

Crawling gets a bad rap. It's useful for a bunch of stuff.



Yup, it certainly gets a bad rap... that's why I was curious what people use this for. Thanks for the info.




Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: