Curious to see if ignoring robots.txt, which has almost miraculously held up for... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

winstonp on June 24, 2024 | parent | context | favorite | on: Perplexity CEO Aravind Srinivas Responds

Curious to see if ignoring robots.txt, which has almost miraculously held up for decades, could eventually lead to a lawsuit.

Consider applying for YC's Winter 2027 batch! Applications are open till November 2.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact