
Craigslist reportedly removes listings from search engines - protomyth
http://www.theverge.com/2012/8/7/3225476/craigslist-blocks-3taps-padmapper
======
tjpd
Isn't the way for Craigslist to remove its posts from search results via
<http://www.craigslist.org/robots.txt>?

~~~
testing12341234
If you take a look at a page like
<http://portland.craigslist.org/mlt/eng/3189881249.html>

You can see that they added a <meta name="robots"
content="NOARCHIVE,NOFOLLOW"> tag. So Google will index it and show it, but it
won't cache the full text. This change means that companies like 3taps can no
longer get the full text from the Google cache, while still allowing search
results to show in searches.

~~~
sigmadelta
<http://www.seroundtable.com/archives/019533.html>

That article is from 2009, noting that CL had added the meta tag to posting
pages. So I think this is old news.

I think storborg has it right in <http://news.ycombinator.com/item?id=4353120>
when saying that "3taps seems to be claiming that Craigslist has cut off
Google, but I think it's just that Craigslist has cut off 3taps."

------
griffindy
am I the only one who uses craiggers? although I suppose they'll have to go
the way of the dodo soon enough now

------
sigmadelta
I just tried the following:

visited <http://sfbay.craigslist.org/bia/>

saw ad for "Burning Man Beach Cruiser Bicycle"
<http://sfbay.craigslist.org/eby/bik/3189652975.html>

performed the following search on google
[http://www.google.com/search?q=site%3Acraigslist.org+neon+gr...](http://www.google.com/search?q=site%3Acraigslist.org+neon+green+cruiser+bike&btnG=Search)

first result returned is that ad, with the following snippet: Burning Man
Beach Cruiser Bicycle sfbay.craigslist.org/eby/bik/3189652975.html 22 minutes
ago ... Neon Green cruiser bicycle, perfect for the playa, or cruising around
town. Single speed, great shape. Available nights and weekends.

\--

"22 minutes ago" sounds like Google is still actively crawling content from
craigslist to me.

