

Mapping CraigsList Housing Ads - andrewljohnson
http://www.trailbehind.com/housing_map

======
coglethorpe
I hate to be a wet blanket, but didn't <http://www.housingmaps.com/> do that
like years ago?

~~~
andrewljohnson
Read my blog post above. I acknowledge them and talk about the improvements we
made.

For example, HousingMaps.com maps a couple dozen cities, and we map all 326 US
cities.

~~~
oldgregg
I thought I read somewhere that housingmaps had a special agreement with
craigslist... But I hope you stick around because the lack of cities on hm is
really annoying.

Craigslist really needs an API... and global search. PLEASE.

------
jonmc12
First, cool. Next, can you share your crawler code?

Lastly, this violates craiglist's usage policy right? Did you research the
implications and what did you find?

~~~
lacker
This does violate craigslist's usage policy.

 _craigslist permits you to display on your website, or create a hyperlink on
your website to, individual postings on the Service so long as such use is for
noncommercial and/or news reporting purposes only (e.g., for use in personal
web blogs or personal online media). If the total number of such postings
displayed or linked to on your website exceeds one hundred (100) postings,
your use will be presumed to be in violation of the TOU, absent express
permission granted by craigslist to do so._

------
andrewljohnson
For those interested in the project, I also wrote up a short blog post about
it at: <http://www.trailbehind.com/trips/view_report/115>

Any feedback or bug reports you have are much appreciated.

------
Cooldude
Good job, I'll be looking for a place soon! I have a few question/comments:

\- Open links in a new window. \- Have a date picker or a dropdown with 1 day
ago, 2 days ago, etc. \- Where are the nearby photos near? \- Show photos
button, why is it different than the checkbox for dogs/cats? Its kind of
confusing. \- Change it so when you click from list of adds, the map moves to
the ad's location on the map. \- When you click on an ad on the map you place
a pin on the map. What can you do with this? I'd rather see pics/summary
information and then I would decide whether I want to save this ad.

~~~
andrewljohnson
All good suggestions

1\. I implemented open in new window 5 minutes ago

2\. I'll add my calendar widget to replace the data selector

3\. I'll overlay the polygon of the photo search area.

4\. Good idea for images for cats/dogs.

5\. I had it set up to pan to the point originally, but now I just open a new
window, so I'm not sure that's useful anymore.

------
rwrice
This is good stuff.I like how the list of ads changes as I pan around the map.
What else does Trailbehind do?

------
mikeyur
Suggestion: Open ads up in a new window, when I click back after viewing an ad
I have to re-select city, etc.

~~~
andrewljohnson
Thanks for the suggestion! I implemented it :)

------
davidaue
It seems like your site is not showing the majority of the adds. If I look at
my neighborhood (El Cerrito, CA) there are hundreds of listings in the last
week on craigslist but setting the cutoff date to 2/20 on your site only
displays 3 or 4 results in the same area.

~~~
andrewljohnson
Sorry, in the future we'll have ads going back further, but we didn't start
crawling CraigsList until the 20th.

Each day the bot runs is another day we have stored.

~~~
davidaue
I think you're missing what I'm saying. If you look at San Francisco/East
Bay/Albany:El Cerrito on craigslist there are 30 listings for 2/27. If I look
in that area on your map I see only 2 listings.

~~~
sam_in_nyc
Same here for my neighborhood. For me, this warrants not using the product at
all... why use something that has an incomplete dataset? I could be missing
out on that perfect listing!

Another thing is that some postings don't contain sufficient information for a
machine to know its location... such as "near the Xyz subway station" or other
atypical types of addresses that the project's parser might miss.

It's a cool mash-up, but I'll pass.

~~~
andrewljohnson
Ok, I see your point.

For some ads, there is no geo-data. To fix that for the next release, we'll
list ALL ads, not just the mappable ones, in the Ad Widget. And we'll indicate
in the table if the ad is on the map.

One good side effect though is the ads on the map tend to be less spammy than
the non-mapped ads. I assume this is because the spam ads often embed their
spam in HTML and don't use CraigsList's structure well. A good landlord is
going to make sure their ad links to a Google Map on CraigsList, which also
means we will index them.

------
sam_in_nyc
Not allowing me to enter my location, and instead having me zoom/wait-to-load
several times makes me wish that closing the tab somehow inflicted pain on the
software.

------
vaksel
how come half the kansas and half of nevada is missing?

~~~
psadauskas
Nobody lives there? There probably isnt a craigslist to cover those areas.

