
Nike.com's robots.txt file - chirau
http://www.nike.com/robots.txt
======
antongribok
In Firefox I got access denied... In case anyone else can't get to it, here's
a pastebin:
[http://pastebin.com/raw/amTGwEEa](http://pastebin.com/raw/amTGwEEa)

------
atombath
Not even disallowing crappy crawlers like majestic 12? They're going to scuff
your Jordans!

------
JoshTriplett
Odd to prohibit crawling PDFs; modern search engines handle those just fine.

~~~
John23832
They probably just want to avoid having to serve them.

------
Ffaine
That's a nice work.

------
kombucha2
thats pretty great

------
ryanbertrand
just crawl it...expect SWFs and PDFs.

