
Bringing Craigslist Back - ericd
http://blog.padmapper.com/2012/07/09/bringing-craigslist-back/
======
trotsky
I think it's perfectly fine if you want to play games dancing around what's
legal and forge ahead against Craigslist's wishes. There's a long history of
that kind of behavior in information services and they may or may not be able
to stop you.

But making a big deal out of being some kind of ux hero is pretty tacky. At
the end of the day, no matter how small you are you're still a commercial
enterprise gaining benefit at another commercial enterprise's expense. You're
bidding on eyeballs with an offer of time savings.

A vast majority of the web deliberately degrades their user experience in
order to achieve business goals. Ads are almost always at least somewhat
distracting, retailers stretch out checkout flows for up selling, sites
"recommend" their high margin items ebay requires paypal and facebook
sabotages their privacy management so you're less likely to bother.

If CL was to build a bunch of sweet vertical specific apps for their sections
they'd need a ton more developers, designers, testers, project managers and so
on. To compensate for the greatly increased burn rate they'd need to monetize
a lot better which almost always means features, content or policies that
negatively impact the user. Or maybe they'd have just taken too much VC to pay
for it and gone under in '02 when the bottom fell out.

So by all means jockey away it's healthy for the industry. But crowing about
it is a bit like taking a shit on the guy whose shoulders you're standing on
while bragging about how tall you are.

~~~
ericd
I'm not making a big deal about this, and I have a lot of respect for
Craigslist, don't put words in my mouth. I think Craigslist is nearly perfect
for the vast majority of things, I'm not on a crusade to "disrupt" them. It's
just that for a long, drawn out, location-based search, it's objectively
terrible, and I'm trying to patch that bug.

~~~
trotsky
It would seem like the definition of respect mostly conflicts with your
behavior.

    
    
       re·spect
         1. To feel or show deferential regard for; esteem.
         2. to pay proper attention to; not violate "to respect Swiss neutrality"
         3. to show consideration for; treat courteously or kindly

~~~
potatolicious
His behavior is not incompatible with respect, and you're being pedantic. I
really wish "beating an argument to death with Merriam-Webster" wasn't as
common on HN as it is.

One can respect Craigslist for their history, and what they have done for the
online community in the past, without conceding the right to say that they're
wrong _now_ , and that you'll be proceeding without their blessing.

One can also respect and understand Craigslist's objection to Padmapper
scraping their site, without conceding that the good vastly outweighs whatever
valid concerns may exist.

Respect is a complicated social concept, the colloquial usage of the word even
more so. Using dictionary definitions to try and corner complex social
phenomena is at best a poor idea, at worst an attempt at being disingenuous.

~~~
zem
back in the usenet days, it was generally[0] held that the first person to
reach for the dictionary had implicitly conceded the argument.

[0] at least on the newsgroups i used to frequent

------
phillmv
>I’ve been wrestling with whether to bring back Craigslist listings in the
search results. I’ve found a way to include them that I’m told is legally
kosher since it doesn’t touch their servers at all, but it still seems
somewhat dickish to go against their wishes in this, and I’ve always had a lot
of respect for what they’ve done for the world. Also, who wants to waste their
time in court?

>But then I did some back of the envelope estimates of how much of people’s
time and effort it would waste if I didn’t, and it became clear how much less
nice it is to waste the time of millions of apartment hunters out of
stubbornness or some clearly inaccurate assumption about the will of the
community.

Christ, what an asshole.

Translation: "We found a loophole that lets us get around the spirit of what
was communicated to us so we could continue to build out our product. I'm
going to conveniently step over the moral grey lines of using someone else's
data without their consent by claiming my service is better."

~~~
ericd
Ouch, sorry, I thought it was the right thing to do, on the balance. Like you
said, it's a really morally gray decision, either way, but I think it's
actually worse to do nothing.

~~~
phillmv
Dude, I have no doubt your service IS way more useful than craiglist. I look
forward to using it when I next move.

Just… don't dance around your motivations here. You want to build a better
service, and you need CL's data for that. CL doesn't want you to use their
data. You said that's too bad.

That's all there is to say.

~~~
stcredzero
Is it really Craiglist's data? When I post to Craigslist, I think of the
listing as my listing, not Craigslist's.

~~~
raganwald
Here’s a comparison. You walk into a used instrument store looking for an
eight string bass. Another fellow walks in at the same time looking to sell
the bass. You wink at each other and meet outside, buy that bass, and cut out
the middleman.

It’s his bass and your money, but the reason you knew he had a bass to sell
was the store. Likewise, it’s “your” listing data, but the reason padmapper is
able to hire someone in the third world to read the internet and manually
screen scrape the data—or whatever it is they’re going to do—is because
craigslist aggregated it for them.

This is very similar to the kerfuffle over allegations that Bing was scraping
results from Google rather than organically indexing the web. It’s quite
possibly legal, but it’s not particularly admirable.

And I join Phil (disclosure: He once called me a “giant,” so I owe him) in
suggesting that arguments that the ends justify the means are suspect.

I’d prefer to see the padmapper folks put up a pirate flag and openly declare
war on craigslist. Just come right out and call them the evil empire already.

~~~
stcredzero
_> the reason padmapper is able to hire someone in the third world to read the
internet and manually screen scrape the data—or whatever it is they’re going
to do—is because craigslist aggregated it for them._

Here's where the rubber meets the road. As far as I can tell, 3taps is taking
data out of the Google Cache and re-aggregating it. While stealing someone's
aggregation of data is clearly wrong, re-aggregating data from public sources
clearly isn't, both from a legal and moral sense, at least in the US.

The material facts in each ad are public data. The fact that such an ad
appeared on Craigslist with such facts is also public data.

 _> And I join Phil...in suggesting that arguments that the ends justify the
means are suspect._

I'm also suspicious of ends justifying the means, but that isn't the whole
story here.

 _> I’d prefer to see the padmapper folks put up a pirate flag and openly
declare war on Craigslist. Just come right out and call them the evil empire
already._

To me, Craigslist is the one being selfish and shortsighted here. If they back
legislation seeking ownership rights over re-aggregation of data, that would
be an "evil empire" move. It would be better for everyone if they subvert
Padmapper instead of fighting it.

~~~
zem
from a moral standpoint, how does scraping the google cache of craigslist
differ from scraping craigslist directly? it's the same information and the
google cache is clearly filling in some "infrastructure of the interwebs" role
here.

------
raganwald
Cartographers, encyclopedicians, and dictionary aggregators have always
included some false data as a honeypot: An alleyway might not really exist. An
obscure word might have been made up. A famous person might be entirely
fictitious. The goal is to catch people copying their books. If you use
“original research,” you’d never say that Braythwayt Street runs parallel to
Rhodes Avenue in Toronto, you must have copied our map data without
permission.

I’m curious as to how padmapper intendeds to solve this problem. If Craigslist
seed their listings with something false, that something is entirely
craigslist’s copyright. It isn’t a “fact.” They don’t have a right to
reproduce it. Lawyers, please chime in.

~~~
mikeryan
I believe padmapper is kind of relying on 3taps to take the hit. 3taps
apparently is DMCA compliant.

Long story short I'd expect 3taps to be craigslist legal target.

(though actually I guess 3taps doesn't actually _use_ the data so I'm not sure
if what they're doing is technically illegal until padmapper uses it - I'm
going to stop armchair lawyering now)

~~~
ericd
That's not the goal, no. The point there is that neither PadMapper nor 3taps
interact with their servers, so neither has to implicitly agree to their
ridiculous TOU in order to summarize the listings. That's the only reason for
that.

~~~
mikeryan
I don't believe the methodology used to collect the data actually matters does
it? The question is fully around the copyright ownership of the content.

I can record TV content from an over the air broadcast and never touch a thing
and still not have the right to retransmit the content.

~~~
ericd
I should have made it clear in the post (I just updated it), but their
complaint wasn't a copyright one, but a TOU violation one. PadMapper isn't
displaying the listings themselves, it just lists #BR, #BA, Price, and a few
other things, and links you to the original to read more.

------
heyrhett
Can 3taps get me the content of the NY Times website via indirect means, so I
can publish it on my website? I did a back of the envelope calculation that a
lot of people are losing time by having to go somewhere else to get that
content, and it's pissing me off.

~~~
seanieb
Only if the NY Times publishes their content in a fashion that makes it hard
to read, undiscovered and freakishly difficult to use in any useful manner,
and your site makes it readily discoverable and very useful.

~~~
mkr-hn
As a content creator, I would prefer you bring the issues to my attention, not
subvert my protections under copyright law. You don't get to do as you wish
with my creations just because you don't like how they're displayed. It
doesn't matter whether we're talking about the NYT or some random blog.

------
glesica
I don't understand how CL can even exercise this kind of control. I'm not a
lawyer, of course, but it seems reasonable to me that apartment listings would
be considered "facts", just like baseball statistics.

So then the CL terms of use are the only obstacle, but can you really be held
to those if you didn't read them and weren't somehow forced/encouraged to do
so? Perhaps so, but then I would find that a little scary.

Just seems like a bad situation all around, but in my mind Padmapper has the
moral (though perhaps not legal, again, not a lawyer) high ground, at least
relatively speaking.

~~~
therandomguy
Let me try. What you say is correct. The listing is "fact". And CL is not
preventing posting this fact to other services. The user can post the same
listing to CL, Rent.com and dozen other sites. What they are controlling is
access to their servers.

------
colinsidoti
I can't help but think you played this poorly.

It sucks CL isn't going to let you get their data, but I really don't think
this workaround is going to change anything (although kudos for the neat
Google cache hack from 3taps).

I imagine your development efforts would be better spent by leveraging the
small but dedicated following you clearly have. There must be ways to take
advantage of it without opening yourself up to more litigation.

~~~
ericd
You may be right, I'm flying by the seat of my pants here (after having talked
to some lawyers). My goal isn't to supplant Craigslist, which is what you seem
to be suggesting - that's far too big a task to take on, even with my sites'
substantial usage, and not really something I'm personally interested in - I
think that classifieds are a mostly-solved problem, with the exception of
reputation and security. I think CL can get there, they seem to be hiring
devs.

~~~
colinsidoti
Yea it sucks your dealing with CL and not Facebook or you probably would have
been acquired.

I say try and move the traffic to pad. It's certainly a big task, but not "far
too big". Pick the region where you have the most significant traffic and
start there. I haven't looked deeply into your site at all, but I know you
have Garry Tans support, ask him what you need to do to get in YC. Get
"Craigslist _is_ the Killer" as a headline on TechCrunch. Piggyback the
Padlister browser add-on with other common downloads, then push a popup to
users posting apartment listings on craigslist saying "Also list on
Padmapper?" There are all sorts of things you can try.

There are hundreds of people on HN that wish they had the traffic you have.
Use that momentum before it dies off.
<http://siteanalytics.compete.com/padmapper.com/>

~~~
bluesnowmonkey
I think if you look up "network effect" in the dictionary, it's going to talk
about Craigslist. Good luck overcoming that.

A great strategy here would have been to establish a strong relationship with
the people at Craigslist from the beginning. No idea if that was tried or not.
Seriously, was the C&D letter the first communication exchanged between
Padmapper and CL? What a missed opportunity. I wonder if this entire disaster
could have been averted by calling the CL guys up early on and taking them out
to lunch. Make friends, send them Christmas cards, then when this disagreement
comes up you handle it over a round of golf instead of a C&D letter.

------
dpcan
I think everyone should just step back and look at the big picture.

This guy is finding any means necessary to add Craiglist data back into his
application and Craigslist does not want him to. I think the "gray" area is
pretty black and white. Craigslist said no. He's doing it anyway - even if
indirectly.

Here's how I see it:

Me: Don't go in my house.

You: But you left the back door unlocked by accident, so I thought I'd just
come in and eat your food.

In the real world this would not be acceptable.

Even though we are talking apples, bananas, and oranges, the same ethical
perspective should be employed. Craig said no, so we respect his wishes. It's
the RIGHT thing to do. If he adds an API someday, then he's sharing the key to
his house with us, until then. Doors are locked, too bad, move on.

~~~
IanDrake
I think you need to take one more step back. Craigslist doesn't own the
listing data on their site. They even say so and for a VERY important reason.

If they own the listing data, they're responsible for it's content. So, in
some cases, they'd be responsible for selling sex, drugs, and stolen property.
Not to mention responsible for a murder or two.

As soon as they claim they own the user generated content, they'll be opening
themselves to thousands of lawsuits for their complicity in thousands of
illegal acts.

Reference:
[http://en.wikipedia.org/wiki/Section_230_of_the_Communicatio...](http://en.wikipedia.org/wiki/Section_230_of_the_Communications_Decency_Act)

~~~
dpcan
I'm not talking about who owns information.

My point is that company A is asking company B not to use the content that was
submitted directly to company A's site on company B's site.

Company B is finding a round-about way to do it anyway.

In my opinion, this is the wrong thing to do. They should respect the wishes
of company A and just not do it because they were told/asked not to.

I believe that just because you "can" doesn't always mean you "should".

~~~
MichaelGG
So? Microsoft, Autodesk, Dell, EA, etc. would prefer you not be able to sell
used copies of software or hardware. As long as it's legal, what's wrong with
doing so?

I agree that legal is not equivalent to right, but in this case, I don't see
the moral argument against this. Everyone benefits from this; the overall
benefit to society is positive.

------
ilamont
The founder of 3Taps posted this on Quora last year, in response to a question
about disrupting Craigslist:

 _... the postings in question are public facts about exchanges. Just as any
price/supply/demand in a marketplace is open for any and all to notate and
republish, so too is the entire set of Craigslist data -- as these offers
between seekers and providers are clearly in the public domain. Historically,
Craigslist has attempted to block access by others to the comprehensive use of
this data. They block many 3rd parties who try to gain access to the data, and
sometimes threaten to sue and bankrupt others as if they themselves created
the underlying data and hold copyright like property rights over the same.

But public facts are public property. And while some think that predatory
Terms of Use demanding that you hand over the Brooklyn Bridge in liquidated
damages if you don't comply with some obscure (and potentially
constitutionally void) constraint will stand in court -- such absurdities will
break if exposed.

A fear, uncertainty,dread approach over access to data breaks down in a world
where Google already indexes all of Craigslist data and caches that
information all over the internet (for search performance results). If its
possible and legal for Google, then why not for any and everyone else to also
index and offer access to the same data. In short, Google doesn't get special
secondary property rights to privatize public data to the exclusion of anyone
else. Equal access to exchange data and search data is a principle in parallel
to the notions of net neutrality.

The points above are not a theoretical discourse. Look at 3taps.com/developers
to see the execution of this concept. And look at what a 3rd party application
(craiggers.com) can do in recreating the whole of Craigslist in a format that
gives access to data in a way that is not remotely possible in the legacy
Craigslist offering. Craiggers is a perfect example that the function of
displaying Craigslist data (rather than gathering it) is a totally distinct
(and competitive) marketplace, even if there are still huge network effects in
the gathering of Craigslist postings.

Note, Craiggers does NOT disrupt the existing Craigslist revenue model for
Craigslist. It simply opens up the field (along with any other developer
building on 3taps assisted access to Craigslist data) that wants to build on
top of (rather than compete with) the network effects of Craigslist. Think
Kayak and Indeed, but now for the whole body of data covered by Craigslist
accessible, rather than just a single vertical._

[http://www.quora.com/Craigslist/Why-hasnt-another-product-
di...](http://www.quora.com/Craigslist/Why-hasnt-another-product-disrupted-
and-replaced-Craigslist/answer/Greg-Kidd)

~~~
debacle
His post is chock full of BS:

> If its possible and legal for Google, then why not for any and everyone else
> to also index and offer access to the same data.

Because Craigslist isn't suing Google. If they didn't want Google indexing
their stuff, Google would comply.

> Google doesn't get special secondary property rights to privatize public
> data to the exclusion of anyone else.

They can and _do_. That's why it's beneficial to set your user agent to Google
Bot when browsing news sites.

> Equal access to exchange data and search data is a principle in parallel to
> the notions of net neutrality.

Full of shit. It takes effort to provide the service that Craigslist does.
Claiming that you have a right to that data is wrong, and moreover has already
been decided by case law (you can find links in other comments on this page).

> And look at what a 3rd party application (craiggers.com) can do in
> recreating the whole of Craigslist in a format that gives access to data in
> a way that is not remotely possible in the legacy Craigslist offering.

There is a reason Craigslist doesn't make that data available - it would
reduce the utility for sellers of their website, reducing their revenue and
potentially drying up their business. Trying to squeeze it out of Google's
cache is still copyright infringement.

> Note, Craiggers does NOT disrupt the existing Craigslist revenue model for
> Craigslist.

Yes, it does. By scraping craigslist in an attempt to undermine their platform
you are eliminating their site's relatively utility for buyers, which
eliminates the impetus for sellers to list there.

~~~
pdeuchler
All of your points rely upon the assumption that Craisglist "owns" all of the
posts submitted. I'm not saying that's right or wrong, but if that is true
then wouldn't that extend to Facebook owning all content submitted to their
service, Twitter owning all tweets, Flickr owning all hosted photos, and Stack
Overflow owning all submitted answers?

~~~
debacle
Craigslists owns the unique compilation of their listings. That's what is at
stake here.

~~~
ricardobeat
And there's no copyright infringement as long as your compilation, based on
their publicly available data, is also unique, which PadMapper's is. This has
lots of precedents dating back to services derived from phonebooks.

~~~
mthoms
I am interested in this. Can you point me to one or more of these precedents?
Thanks in advance.

~~~
ricardobeat
[http://en.wikipedia.org/wiki/Feist_Publications_v._Rural_Tel...](http://en.wikipedia.org/wiki/Feist_Publications_v._Rural_Telephone_Service)

------
eli
It's funny how many people are demanding that Craigslist sell out its user-
generated content. I understand why people want this, but isn't CL being
incredibly respectful of its users by allowing their data to be used only for
the specific purpose for which it was posted and that's it?

~~~
crazygringo
I don't think so.

Users are on Craigslist to buy/sell, because it's the biggest place for that.
If you post to CL, virtually everyone wants the largest audience possible, so
99.9% of people will be thrilled to have their post copied to other services.

The only annoying thing would be if you take down your post from CL after
you've bought/sold your thing, and you continue to get calls/emails from
people getting it via copied data. But sites that copy just need to stay up-
to-date on whether postings are still up or not...

------
SlyShy
As someone apartment hunting in NYC I appreciate this. If I wasn't my feelings
would be pretty mixed.

~~~
ericd
Yeah, I went back and forth on it quite a lot. That said, the law protects
this sort of thing (facts can't be copyrighted, and that's all PadMapper is
showing from CL), and it seems like a pretty large net loss for the world if
this isn't continued. But I'm obviously biased.

~~~
unreal37
Facts can't be copyrighted. But how is an advertisement a "fact"? Can I create
a website of all the Coca-cola advertisements over the years, and call them
facts?

Ads aren't facts. You're still intentionally copying the CL website without
the right to do that. Good luck to you.

Edited: Not trying to start a "copyright isn't theft" debate.

~~~
Produce
Copying is not stealing! Christ, the music and movie industries have really
fucked up the meaning of that word in recent years.

~~~
jmduke
Craigslist's #1 asset is their gigantic amount of classifieds. There's really
nothing else going for them.

Any service competing with Craigslist that uses those classifieds might not be
_stealing_ them, but its certainly in Craigslist's best interest to keep their
most valuable asset safe.

~~~
Produce
I'm still technically correct, which is the best kind of correct.

~~~
jmduke
I wasn't saying you were incorrect, mainly because I'm not nearly well-versed
in law enough to define 'stealing' in such a nuanced context.

I'm simply saying that Craigslist has a vested interest in keeping their
data/facts/ads/whatever on their platform.

------
mmanfrin
I have a feeling there will be another C+D soon, but since I'm looking for an
apartment at this moment, great.

I am annoyed that CL hasn't uttered a single peep about all this. They killed
a great app, and didn't even respond to the outcry. A simple 'we don't want to
support competitors' would have sufficed, but their silence damns them.

~~~
elsewhen
Padmapper already received a C&D... IANAL, but i think a lawsuit may be the
next step.

~~~
grabeh
If the circumstances have changed, the appropriate course of action would be a
further cease & desist. This would allow Padmapper the opportunity to desist
from their activity which is always the right course of action before
instituting proceedings.

~~~
elsewhen
The C&D mentions not just accessing CL content, but republishing it. That the
content was transmitted through third parties is irrelevant for this copyright
claim.

~~~
grabeh
It may be irrelevant, however you will get penalised on costs (at least in the
UK) if you do not attempt to resolve the dispute prior to issuance of
proceedings.

I personally wouldn't want to risk the ire of a judge in issuing proceedings
when at least on the face of it, PM complied with at least part of the initial
cease & desist sent by CL.

------
mootothemax
Can someone explain to me how this works legally? It seems a bit too
convenient that you can ignore someone's T&Cs and use an unlicensed third-
party service instead.

~~~
Rudism
From what I gather, the Craigslist T&Cs specifically forbid scraping data off
their website. They explicitly call out the fact that they do not own the
actual data/listings/facts on their website, so those are public domain (or
possibly owned by the person who created the listing, who knows). By scraping
the data from Google's cache, you're no longer in violation of the T&Cs
because you're not scraping from Craigslist's website anymore. You are free to
use the data obtained in this method because Craigslist doesn't own the data.
(IANAL)

~~~
slig
What about breaking Google's TOS by using a bot to scrap data from them?

~~~
Rudism
Do they really have a rule against scraping their scraped data? If so, you'd
be in violation of that, but that's your problem with Google now, not
Craigslist.

------
pbreit
For all the people mis-interpreting CL's TOUs: "You automatically grant and
assign to CL, and you represent and warrant that you have the right to grant
and assign to CL, a perpetual, irrevocable, unlimited, fully paid, fully sub-
licensable (through multiple tiers), worldwide license to copy, perform,
display, distribute, prepare derivative works from (including, without
limitation, incorporating into other works) and otherwise use any content that
you post. You also expressly grant and assign to CL all rights and causes of
action to prohibit and enforce against any unauthorized copying, performance,
display, distribution, use or exploitation of, or creation of derivative works
from, any content that you post (including but not limited to any unauthorized
downloading, extraction, harvesting, collection or aggregation of content that
you post)."

------
grabeh
There's obviously a big fat indemnity clause in 3Taps' terms which means
you're on the hook for claims made by third parties against 3Taps for loss
arising as a result of the usage of the service.

Having said that if CL provides access to 3Taps on terms which allow third
parties to take data from 3Taps, then this may be permissible. It of course
depends on the terms of 3Taps' access though (I admit to not being completely
familiar with the ins and outs of 3Taps' service though).

The problem of course is that CL have already shown a dislike of Padmapper's
methods and as a result this workaround is unlikely to meet with CL's
approval. Common sense would suffice to tell you that.

Having said that, it's a real shame CL aren't open to discussions over
licensing data (maybe they are of course but the price is too high).

~~~
ericd
Nah, they weren't open to discussing any sort of deal involving the website
(they were willing to license data for the mobile app).

~~~
stcredzero
Why didn't you go that route? There are so many phones and tablets out there
now, you'd have accomplished almost as much public good with far less
controversy.

~~~
ericd
Good question. Basically, it's not really the best way for people to browse
this kind of stuff, and there are still lots of people without phones/tablets.
You can get a much better picture of things with PM maximized on a 20" screen
than you can with a 4" one. The phone is more of a backup for when you're on
the road or in a pinch.

~~~
stcredzero
_The phone is more of a backup for when you're on the road or in a pinch._

My girlfriend is looking for an apartment in Houston, and sometimes browses
Craiglist with her iPad.

~~~
ericd
Yeah, the iPad is a pretty good substitute for those that have one (and I'm
happier with the PadMapper iPad app than with the phone apps).

------
hornbaker
Bold move. Curious if CL's next step will be to add noarchive tags to prevent
google cache content from being crawled.

~~~
moron
I can't see why they wouldn't do that. It's not like the Google cache offers
much utility for Craigslist posts to begin with.

Anyway, it doesn't seem right for Padmapper to be doing this, but for god's
sake, could people browsing the primary source for residential ads please have
a way of doing that with a map?! That's really all anyone is asking for here.
Sheesh.

~~~
koenigdavidmj
> _I can't see why they wouldn't do that. It's not like the Google cache
> offers much utility for Craigslist posts to begin with._

Then there's no harm in their putting the tags in.

~~~
moron
Uh, okay?

------
lancefisher
I appreciate that Eric is doing this in the open. The moral argument could go
either way, but he is letting everyone know what he is doing and why. He
deserves credit for inviting feedback on it. Personally, I hope padmapper does
well.

------
IanMechura
I see some very interesting comments on this thread. I remember not long ago
lots of people where upset that people where scraping stackoverflow and
getting a higher ranking in google results. This was a threat to S/O's
business model model and it made lots of people on this board uneasy.

How is Pad mapper any different? How does the fact that this site has a better
UI design somehow give them a pass to use data a company has spent millions of
dollars gathering and feels is being used in a way that is a threat to their
business model?

~~~
delackner
Spent millions gathering? Two basic problems with that mindset. First,
spending money to assemble a database of public facts does not magically give
you a right to protect your investment in an uncopyrightable asset. Second, I
don't know what millions you are talking about since craiglist's only asset is
the users, and the users are just a result of tremendous network effects. They
had a great idea and got very lucky, but gathering their massive userbase had
nothing to do with spending money.

~~~
delackner
Could someone explain what they disagree with? Do you disagree that databases
of public facts are not copyrightable? With the idea that craigslist's
investment in their system is irrelevant compared to the value of their
userbase? With the idea that craigslist's main success is just being lucky?

------
kefs
The last few places I've acquired we're through padmapper aggregation of CL
posts, so when PM got the C&D, I felt compelled to email CraigsList asking
them to implement a fee-based API as it would be a win-win-win for everyone
involved. Sadly, no reply.

------
stcredzero
After reading the post and discussion, my conclusion is that Craigslist is
making a policy mistake.

They should make the same calculation of public good that ericd made and
license 3rd party websites that add value to Craigslist, provided the 3rd
parties do not harm the Craigslist infrastructure or business models. Doing
otherwise is just holding up progress, and is a losing game in the long run.
Facilitating value to users would be a win for them, as it was in the past.

~~~
debacle
Creating a site where buyers can browse property listings does harm the
Craigslist business model. To say otherwise expresses a lack of understanding
of online ecosystems.

~~~
stcredzero
As far as I can tell, they aren't posting the full listing. They are only
aggregating the material facts about the listing. As far as I can tell, short
of actually hoofing it to the place, you still have to get on Craigslist to
make contact with the landlord.

~~~
hollerith
What Craigslist is afraid of (probably) is Padmapper becoming the first place
most people turn to when looking for an apartment. Once that happens,
Padmapper can cut Craigslist out of the loop by inviting landlords to make
Padmapper listings instead of Craigslist listings. (Note that once Padmapper
become to "go to" place for apartment hunters, Padmapper's refusing to
continue to host or "index" Craigslist listings _helps_ towards that goal.)

Regardless the morality or legality of Padmapper's latest action, it is pretty
clear to those of us who have spent a lot of time observing online ecosystems
and software ecosystems that Padmapper's latest action is _not in the
interests of Craigslist_ since (even if Craigslist is not out to make money,
but rather out to influence the world for the better) the only important
source of Craigslist's potential for making money and of Craigslist's
influence is Craigslist's being the first place people turn to when apartment
hunting and doing other important things.

That is why Craigslist is against Padmapper's latest action even if Padmapper
never posts the full Craigslist listing.

~~~
stcredzero
And in that, Craigslist is clearly acting in its own interests contrary to the
interests of its users.

~~~
jmduke
Contrary to the interests of Padmapper users (some of whom are also Craigslist
users.)

The vast majority of Craigslist users don't want to use anything else, nor are
aware of anything else.

------
dredmorbius
Eric's confusing method with goal.

Craigslist's past _method_ was to claim TOU violation on the part of
PadMapper, denying PM ready use of its content.

Its _goal_ is to maintain market position as _the_ dominant classifieds
listing service. TOU is just one way for CL to enforce its market position,
and as a commercial entity, it's very likely it will continue to do so.

There's very little reason CL couldn't create an alternate legal claim based
on content (the cartographic watermarking example comes to mind) to make
multiple infringement claims against PM, which would invoke a whole host of
copyright remedies ranging from injunctions and monetary damages to DMCA
takedown notices (which could comprise the entire service depending on how
construed by PM's hosting provider(s)). There's also the notion of a
compilation copyright, in which a collection of CL postings could have
standing. While Feist v. Rural Electric holds that a simple compilation of
facts doesn't meet the minimal authorship requirements of copyright, a
compilation of _postings_ with some level of curation to CL might. I'm not
aware of relevant caselaw here.

My own suggestion would be that PM pursue a dual strategy of coming to a
business agreement with CL (PM's interface _is_ vastly superior) while
curating its own set of rental listings independently (CL is ripe for
disruption here).

Disclaimer: I'm a long-time CL user and fan of much of its philosophy. I'm
also a recent fan of PM. I hope the two will either work something out
cooperatively, or manage to improve apartment and classifieds search by
competitive means. I don't particularly care which.

------
rdl
Thank you for doing this. My only problem with it is that you're rewarding
craigslist for being obnoxious tools, but ultimately it's better for the
users, so that's an acceptable sacrifice.

3taps is pretty awesome. I wish they'd do an AMA or something.

------
jmduke
As a satisfied padmapper user, I'm glad to see this, but somewhat troubled at
the same time.

The original hubbub about Craigslist pulling padmapper's access, regardless of
your stance on Craigslist's actions, was to me a cautionary tale about
building apps around external APIs. (See the Twitter drama of the past weeks.)

What happens next if Craigslist does the same thing to 3taps?

------
csmajorfive
So does 3taps gather data with humans or what?

~~~
keltex
"Any copying, aggregation, display, distribution, performance or derivative
use of craigslist or any content posted on craigslist whether done directly or
through intermediaries (including but not limited to by means of spiders,
robots, crawlers, scrapers, framing, iframes or RSS feeds) is prohibited"

I think this covers Google cache or any other method of gathering data. You're
still in violation of their Terms of Use even if you find a clever means of
getting around it.

~~~
nicholasreed
Simply because it exists in the ToS doesn't mean it is legally enforceable.
Hopefully the "legally kosher" claim means he had a lawyer look over
everything, and conclude that clause in the ToS is for the courts to decide.

------
declan
I posted a look at the legal issues on Google+ here:
[https://plus.google.com/112961607570158342254/posts/HcCuHVG7...](https://plus.google.com/112961607570158342254/posts/HcCuHVG7rLM)

Here's an excerpt from the post: Eric's blog post today says that "using 3taps
just makes it so it’s not a TOU issue." The problem is it doesn't.
Craigslist's lawyers already thought of this, which is why the TOU also
prohibits the "display" and "derivative use" of "any content posted on
craigslist," which PadMapper is clearly doing. As Craiglist's lawyer is likely
to put it in the next C&D, assent to the TOU is still implied. [...]

If 3taps is in fact scraping Google's cache, they're also violating Google's
terms of use, which prohibits any effort "to access [search results] using a
method other than the interface and the instructions that we provide." Anyone
want to bet on how long it will be before Google blocks them through technical
countermeasures, or sends its own C&D? And if 3taps is in the business of the
TOU-prohibited "distribution" of Craigslist data, which it appears to be, it
should expect to get a C&D from Craigslist too.

------
etrain
"Certainly by the time you read this." Code isn't live yet, at least for me,
ericd - is this conversation giving you pause about deploying the new feature,
or is it standard ops issues?

Also, this couldn't come at a better time for me. I love padmapper and am
moving to the bay area in 3 weeks, I need this, man!

~~~
ericd
Sorry, was having some sysops issues, thanks for the heads up.

------
seagreen
What about a service that lets you _post_ to every barter site at the same
time, not just craigslist. Has that been tried? If you could get people to use
it (through better UI and more exposure for what they're selling) you could
break craigslist's hold on the marketplace.

------
jgh
Padmapper is awesome, way way better than using Craiglist to apartment hunt,
even if the data ultimately comes from Craigslist (among others).

Maybe craigslist should buy padmapper instead of getting all lawsuity and take
advantage of just how good their interface is for apartment hunting

------
rjsamson
Not sure if this has been asked, but a philosophical question for Eric: if
someone decided to start a padmapper competitor tomorrow, would you be okay
with them using your data for free?

~~~
huckleberries21
I don't see why not. By being a competitor that scrapes PM, they are accepting
that there will be a delay in their data versus when it's available on PM. So
unless their site was significantly better than PM in some way, in the really
competitive apartment hunting markets, they would be strictly inferior to PM.

------
bambax
How do 3taps do it? Some comments say they "use Google cache", which doesn't
explain much:

\- Google is notorious for fighting bots pretty aggressively, how do 3taps
avoid this?

\- even if 3taps found a special access to Google cache, it would be trivial
for CL to prevent Google or anyone else from caching its listings, thus
cutting off the rest of the food chain that feeds on caches; why don't they do
it?

Is it possible that 3taps use another method, such as "manually" copy-pasting
listings with Mechanical Turk?

------
duked
Really, really stupid. I mean pretending you want to help people find
apartment to justify you're action is ridiculous. You're running a business,
be honest about it. Now craigislist do what they want with their data, you are
just trying to build a service that "adds value" on top of someone else
infrastructure( I mean "add" because I am one of the few who actually like the
way cragislist display posts no bloaty js, no ugly pics no maps ... fast and
simple).

~~~
JackC
I find it difficult to believe that you have searched for an apartment in a
tough rental market on Craigslist and consider their interface "fast and
simple." If you've never had the task of finding an apartment within a mile of
a given spot in a big metro area, you're not looking for what PadMapper
offers. If you _have_ had that problem, and you've never sworn out loud at
Craigslist, you're a saint.

~~~
duked
I was looking for an apartment 6 month ago in Reston, VA and I used craigslist
for that purpose. I looked for a couple of zipcode (that I heard were good,
since I was just moving in the US at that time) and honestly it worked great.
Wouldn't it be the same in a big metro area ? just looking for interesting
zipcodes ?

~~~
drstewart
You realize in some cities a few blocks makes a GIANT difference in
neighborhood, right? Same zip code.

Being able to visually filter those out is a major time saver.

~~~
JackC
The other issue is where I live, people put all kinds of random keywords in
their Craigslist apartment ads to try to get attention. "Just 25 minutes' walk
to Foo Square, Bar Square, and Baz Square! Four hours by horse to Lexington,
and a short plane ride to London!" If I want to know what's within a 10-minute
walk of a given location, I basically have to read every listing with my
town's name in it every day and see if they included an address. Or spend 5
seconds glancing at PadMapper.

------
Zakuzaa
[http://traffic.alexa.com/graph?w=400&h=220&o=f&c...](http://traffic.alexa.com/graph?w=400&h=220&o=f&c=1&y=t&b=ffffff&n=666666&r=3m&u=padmapper.com&);

Traffic took a great nosedive from the day they announced "bye bye" to CL.

~~~
huckleberries21
By your own metric, so did traffic for CL, and CL is the 8th most visited site
in the US, so their Alexa traffic data is probably more stable than traffic
data for PadMapper.
[http://traffic.alexa.com/graph?w=400&h=220&o=f&c...](http://traffic.alexa.com/graph?w=400&h=220&o=f&c=1&y=r&b=ffffff&n=666666&r=3m&u=craigslist.org&);

------
Spooky23
My question is: Why are people using craigslist?

Fortunately for me, I purchased a home before all apartment ads migrated to
CL. For the life of me, I don't get why it's so popular.

The site is awful in at least a half dozen different ways.

~~~
dinedal
It's free and easily accessible. Compare to the costs involved in getting an
ad listed in a dead tree.

------
whyenot
So, instead of directly scraping CL's data, they are now using an API that
scrapes Google's cache of CL's data? That seems kind of sleazy.

------
egillie
Would a client-side padmapper be less morally gray?

------
rprasad
Consider, if the situation was reversed, and Craiglist was taking Padmapper's
data without permission, would everyone on HN would be supporting Padmapper
attempting to block Craiglist from reposting its listings?

In such a scenario, Craiglist reposting the Padmapper listings would be a net
gain for user's and posters (even though the posters never gave Craiglist such
permission) because the listings would be exposed to an exponentially larger
viewership.

Being a startup is not a license (pun intended) to ignore other people's
rights. Padmapper is in the wrong here, and if they proceed down this path
they could face a ruinous (and fully justified) lawsuit.

------
Baba_Chaghaloo
>I've always had a lot of respect for what they've done for the world

Did I miss something? What has Craigslist done for the world besides put
newspapers out of business?

~~~
genwin
Using CL I furnished a whole house with super high-quality stuff for less than
US$2K. Wouldn't have been realistic pre-CL since traditional classified ads
don't have pictures. I also found my house through CL.

------
tubbo
Dear PadMapper,

We hate your app and you can't use our API.

Sincerely, Craigslist

\+ + +

Dear Cragislist,

Fuck you.

Love, PadMapper

