Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Has anyone noticed that the response for the blog page has a header: "x-robots-tag: noindex, nofollow"? What's the purpose of this header on a content page?

UPD: Sorry, never mind, I inspected a wrong response.





I don't see it. With Chrome devtools, for the posted URL I see X-Clacks-Overhead, X-Content-Type-Options, and X-Frame-Options. No X-Robots-Rag.

And no <meta name="robots"> in the HTML either.

What URL are you seeing that on? And what tool are you using to detect that?

Edit: cURL similarly shows no such header for me:

  curl -s -D - -o /dev/null https://journal.james-zhan.com/google-de-indexed-my-entire-bear-blog-and-i-dont-know-why/

Sorry. I am an idiot. Checked the wrong url. Please ignore.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: