Hacker News new | past | comments | ask | show | jobs | submit login
Crime Visualization of San Francisco (padmapper.com)
105 points by ericd on July 28, 2010 | hide | past | web | favorite | 63 comments

This is very interesting. A friend of mine who lived on Frederick St, just a few blocks up the hill from Haight, said that crime was much lower on her block. Her theory was that bad guys didn't like to climb hills. And in fact there seems to be a correlation between altitude and crime rate.

There are probably other reasons too. E.g. that the hills are fancier neighborhoods.

But her explanation makes sense. If you're walking around looking for someone to rob, why walk up steep hills?

There's also fewer homeless people the higher the altitude.

If you're walking around outside all day, and have no particular place to go, and are possibly pushing around a shopping cart full of stuff, it makes sense that you would tend to gravitate to areas with lower potential energy.

its true. I've never seen a shopping cart on Half Dome.

I lived on Beulah, one of the flat streets between Cole and Haight. There were countless bums from GGP & haight street milling about my block, which was considered "Cole Valley", a relatively yuppie neighborhood. My apartment was burglarized and my TV was stolen. I moved up to Stanyan and Rivoli, considered to be the same neighborhood, but straight up a very steep hill, and never saw another human soul again.

My parents have lived in the same apartment, above the UCSF Parnassus campus near the end of 5th Ave, for decades. It requires climbing multiple steep hills and/or stairs, it ends in a cul-de-sac, and like many buildings in the city, you have to climb stairs yet again to reach the door - the first floor is ground level in the back, but above the carport in the front.

Since the hillside is forested, we get plenty of traffic from wildlife, but I can only think of maybe once or twice that any homeless person has dared ventured through to collect cans. Despite this I'm told that there WAS a burglary, but it must have been in the 70s, because following that the management put bars over the first floor windows, which have existed for my entire lifetime.

I lived near there - at Clayton and Ashbury - for a year. Not once did I see any homeless people. There is no way of getting up to that location without hitting at least a block of 15% grade. But the hill did not demarcate the homeless population - it appeared to be Frederick, which is a block further south than Beulah away from Haight.

I had two theories: leaving Golden Gate Park, you end up on Frederick street; going north gets you to Haight St which is a wonderful place to panhandle, going south gets you nowhere. The other theory is that it would have gone further south (maybe a few more blocks to Parnassus), but the Cole/Carl intersection was nice enough, and full of enough families, to push people south the second they get off of the N-Judah.

Interesting. Maybe crime actually does flow downhill to some extent? It would be interesting to see if there's a correlation between crime rate and relative elevation.

Completely different city but with lots of hills (seven to be precise): Lisbon. The most dangerous areas are near the top of the hills in places like Bairro Alto, Sé, Alfama, while the main bottom of the hill, Baixa, is much better.

I believe Lisbon has a very low crime rate (for european standards) and my theory for more crime happening on the higher streets is pretty simple: on the bottom the streets are really big and there's lot of space while the streets on the top of the hills are really small so easier to be seen by the police when you're doing something evil.

I like thinking about these kind of relations though!

In Brazil (and probably elsewhere in South America), real estate values go down (and thus crime goes up) the higher you go on the hills. Basically, this is because walking up and down the hills is arduous and sucks, and thus people try to avoid living on the hills if they possibly can. Thus you end up with favelas.

If everyone had cars, or public transit, you would probably see better property values on hills, like you do in SF.

Have you seen the one that plots crimes as if they were elevation? http://dougmccune.com/blog/2010/06/05/if-san-francisco-crime...

Very cool. Apparently prostitution is strikingly localized (which makes sense). It also seems that Hunters Point doesn't have much/any crime represented there either, yet it seems like people think of it as a very high crime area. I wonder why.

> ...yet it seems like people think of it as a very high crime area. I wonder why.

from wikipedia:

"Bayview-Hunters Point is predominantly African-American. The neighborhood has a population of 20,322 and is roughly 60% African-American, the highest concentration of that ethnic group in San Francisco."

The article on the community is quite lengthy and interesting:


I've never lived in San Francisco and don't know whether this community has a bad rep. because it's mostly black or if it really is as crime-ridden as its reputation.

I think we can all agree that this kind of data analysis is really important though, to understand where we live and shape public policy.

Sampling bias. The more people who visit a high-crime area, the more crime is likely to occur.

That and the gentrification resulting from UCSF's push into the area...

Those are awesome pictures. It's interesting to see how crimes "fall" downhill from a center.

Interesting theory. This might also be what gives the Tenderloin its' northern boundary somewhere around Post St, as Nob Hill starts rising more sharply north of there.

Shit rolls downhill.

This is a new visualization layer on PadMapper (an apartment hunting site) which shows the relative frequency of crime in different areas of San Francisco. It's not differentiated between nonviolent crime and violent crime currently (should be at some point). Red is bad, green is good, and nothing means very little crime reported for that area (even better than green within the bounds of SF, simply no data outside).

It would be nice if you could filter by the type of crime, not even necessarily violent crime. I might care if someone's selling heroin outside my door, or cars are being stolen, but probably wouldn't care if prostitutes are running about or dogs are unleashed.

Personally, I'd be ok with pot and coke, but not heroin or meth.

Also, preferably decent-looking prostitutes, not any of those skeezy ones you see outside the 16th street BART.

Yeah, I think this needs a drugs checkbox selector widget, just like padmapper itself.

Good point. It currently needs to be precomputed as images, so it's hard to make it customizable, but it probably makes sense to make a category of unsavory crimes. I would assume that most people dislike living near the same types of crime.

How are you precomputing the images? I thought GMaps provided a way to input data and get a heatmap, but apparently I'm mistaken.

I couldn't find anything to that effect, so I wrote custom scripts. Basically, it creates an image pixel by pixel, with the pixel value corresponding to the crime density of that chunk of the map, blows it up, smooths it with a Gaussian kernel, and then uses some GIS tools to warp it and chop it up into tiles for different zoom levels.

That's an interesting feature for an apartment hunting site. However I think you have to add some information like the time-interval this goes back to. If it's only for a couple of days the statistic might be full of relative false-positives and false-negatives.

Ah yeah, I should specify that. The data window is the past 58 weeks or so.

Unrelatedly, PadMapper is excellent (better than other Craigslist mappers I could find): it has a great filtering system, seems to fetch fresh results more often, and can alert you about new places. I found my current place using it.

I concur. Padmapper gives me a quality of life advantage over my peers since I can find exactly what I want and monitor for deals while they're getting the luck of the draw off CList.

Thanks :-) Glad it was useful!

Wow, this is excellent. I'm looking at moving to San Francisco later this year and this is such important information. I knew that places like the Tenderloin were dangerous, but there's an unexpected hotspot in the middle of the Mission that I know to avoid now. Thanks.

Cool. It's available on PadMapper itself as an overlay, so you can see where the apartment listings sit on the crime map. It's a bit buried - have to maximize Filters and then Super Secret Advanced Features.

that is excellent. There's a 20 block area that is the reddest zone in the city and my guess is that's the Tenderloin.

I wonder what impact an intense cleaning would have on the crime rate there. And enforcement of simple things like public urination laws. And maybe some decent bathrooms.

I recall the kind of things New York did to restore a sense of order. Malcolm Gladwell talked about them in the Tipping Point. Surely similar measures would improve the crime situation in SF.

Most definitely, but you'll never get a Rudy Giuliani figure elected as mayor of San Francisco, at least not in the foreseeable future.

My prediction: sometime in the next 30 years the tech industry will have drifted elsewhere, and San Franscisco will be the new Detroit. Right now it's a mixture of the sublime and the awful, but the sublime can always pack up and move to Portland/Austin/Seattle.

Most of what is sublime about San Francisco can't move.

But you're right. People in San Francisco don't seem to value public order enough to take the actions required to establish it.

About 10 years ago I had to go to SF for a week at short notice and left the booking of the hotel to our office manager - big mistake.

It was on Geary, and a bit of a dive. First thing that surprised me was when I got into the taxi at the airport and the driver asked "Do you really want to go there?".

Next sign that things were not too good was when the chap at reception said - "Do not walk along Geary to get to Union square, walk two blocks up the hill, along and back down to Union square.

Of course, late one night I decided to walk directly back, after all how can it be (feeling reasonably safe going out in Glasgow - which is practically a war zone late at night). I only did this once - getting a taxi back after that!

Turns out the hotel I was staying in is right in the orange zone there.

I stayed in a Youth Hostel right in Tenderloin. The crack dealers I met were pretty much only interested in you if you were either interested in buying crack, or were obviously scared of them (at which point they'd make fun of you - hey I would too).

The big surprise for me was how dirty downtown San Francisco was compared to LA. I'd never seen someone actually take a dump in the middle of a city sidewalk, or piss a level down into a train station before SF, and I haven't seen it since. My Full House illusions were well and truly shattered.

I asked the taxi driver on the way to the airport if things were always like that and he gave an explanation about adjacent police districts taking it in turns to move these folks from one district to another - apparently this had just happened so things were a bit more lively than normal.

I learned the lesson and booked my own hotel the nice time I visited - stayed somewhere very nice right next to Union Square. :-)

I made CrimeDeskSF (http://itunes.apple.com/us/app/crimedesksf/id339706696?mt=8) to visualize some of this data on the iPhone, including a mostly-gimmicky-but-kinda-fun augmented reality view (drug bust 100ft away!). I'm using the datasf.org datasets for mine (not sure how they compare to the crimespotting ones, or if they're the same).

I can't be the only one who took one look at this and thought: Hey, that's SimCity!


Looks like the City of San Francisco and I employ similar strategies for fighting crime:


That was my thought too - and I'd be curious to see something like SimCity's police station coverage map overlaid on this.

It amuses me to imagine the mayor's desk with a selection of these maps, plus a little R/C/I readout.

Is this normalized by the population density or foot traffic in an area? If not, it's going to be hard to use to estimate personal risk.

It isn't. My theory is that crime density by land area alone probably gives a decent estimate of how the area feels to live in. IE: if there's an assault or robbery every week in a one block area where you live, it's going to feel like a seedy place to live regardless of your personal risk of being involved in a crime.

I doubt there's an objectively right answer as to how to best normalize this - I think it just matters that it be understood by the user what it means. There are a number of ways to normalize it depending on what you care about.

Very interesting. I would love to see someone build something similar for other cities, perhaps overlaid with Points of Interest and mass transit lines. I think it could be made to be useful for tourists and people who are new to an area.

However even if it can't be made into a usable product it'd be very interesting to see.

I used to have markers representing high-rated yelp locations on PadMapper, but they weren't too happy about having their data cached (which is necessary in order to make it smooth), so now it has to be pulled manually by the user.

Transit lines are something that should go on PadMapper at some point. I hope Google opens up their transit layer sooner rather than later.

All I knew about SF crime was that the Tenderloin felt quite dodgy when I visited. So I opened the map ready to be suprised. Instead it turns out my feelings were right. Looks like the crime rate in SF is an inverse function of the distance from the Tenderloin.

The LAPD has a much more informative interface


which separates out crime by types. You can clearly see patterns such as GTA's (car theft) along major streets in Hollywood.

If you want more details, sanfrancisco.crimespotting.org has them, or you can download the raw data from datasf.org. The hot spot at 16th Street BART is drug dealers and in practice there is no reason not to go there.

I would say the off streets right near the 16th street bart are pretty sketchy. I've seen lots of tweakers and crackheads fighting on Hoff and Capp. A crazy guy smacked me in the head "hit and run" style over there, and a roommate was mugged by 3 guys waiting for him in a car.

I'm going to have to disagree. Its accessibility via the muni and the bart, combined with the proximity to the hard-drug areas of town means that there are a number of quick and dirty smash-n-grab robberies of transient ne'er-do-wells. Admittedly I'm biased because I experienced a single one of those datapoints first-hand.

OK, I shouldn't have dismissed it so quickly... there are some real issues there too. But I have spent a lot of time there without worrying about my safety.

Yeah, it's definitely not all dangerous crime. I still wouldn't want to live in a drug dealer hotspot, though...

So most of Hunter's Point is crime free? Is this data correct?

Hunter's Point is actually very low population density, especially due to large properties like Hunters Point Naval Station and other industrial sites which are fenced and controlled.

Bayview is higher crime, and shows moderate numbers on most of those maps (for violent crime). Murder isn't on that map. For gun-related crime, it is the top place in San Francisco, which is why the SFPD deployed the "ShotSpotter" system there -- basically a microphone array which can localize gunshots and notify the police.

Probably the issue is that Hunter's Point has a lower population density than many other parts of the city. The Tenderloin is chock full of huge apartment buildings, SROs, etc... so even if the per-capita crime rate is the same (or lower), the total number of crimes can be much higher.

This is why I'd like to do violent/nonviolent versions. I've heard it described that the Tenderloin has a lot of non violent crime, and that Hunter's Point has much more violent crime. Ultimately, I'd rather be in the Tenderloin if that's the case.

I believe the data is correct, but I'm not sure what the SF police's data policies are. There could be some withholding, etc. The data comes from http://sanfrancisco.crimespotting.org/.

The San Francisco Police link to http://www.crimemapping.com/map/ca/sanfrancisco from their page at http://sf-police.org/index.aspx?page=1618 which has some discussion of their data policies.

I would guess that people just don't call the cops as much in bayview/Hunter's Point.

No luck getting the presidio data?

The presidio is actually a government property, so any crime within the presidio is actually a federal crime and is reported differently. Beware of committing any crime there, the consequences are far more drastic.

Thanks for clearing that up.

That's very likely just a function of population density - this is raw crime density by land area.

EDIT: There's a crime density cutoff below which areas are blank. Within the bounds of SF, blank means very low crime density.

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact