Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It appears to have been changed, just seconds ago. Now it reads:

  User-Agent: * 
  Disallow: /x?
  Disallow: /vote?
  Disallow: /reply?
  Disallow: /submitted?
  Disallow: /threads?


That seems like a nice middle ground--it should save server load, but search engines can still get to useful content.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: