

Ask HN: legally can i build search niche web search engine? - umenline

i want to build search engine that searching dally 
niche sites , like blogs first page's 
then present statistics based on the crawling. 
legally can i do this ?
======
yamural
There is no legal restriction for crawling any web site. But you should
respect robot.txt document for every single web site. Such as:
<https://www.facebook.com/robots.txt> <http://web.archive.org/robots.txt>

------
manidoraisamy
Yes. You can. Google successfully defended it in the past :
<http://en.wikipedia.org/wiki/Field_v._Google>

