There is no rent bubble. Bubbles are caused by speculation. There isn't a large amount of people renting or owning speculatively in San Francisco. In fact, pro-tenant regulations in San Francisco make it an especially bad place to speculate. I have heard of the city forcing landlords to renew leases for suspected drug users that leave spent needles all over the porch.
High rents in San Francisco are caused purely by scarcity. That is, everyone wants to live there, but the city refuses to allow more units to be built or make it easier to rent. This damage is self-inflicted; California liberals hate density and gentrification although such things are green, decrease broad-market housing costs thereby serving the "displaced" poor, increase urban vibrancy, and reduce traffic congestion compared to the status quo.
I saw a study that suggested the economic cost for permission to build a unit in SF (calculated by subtracting a unit's cost from the value of the land it sits on plus the cost of building it) is $600k. By contrast, it's $100k in Chicago and five figures in any major city in Texas.
I would argue that there indeed is quite a bit of speculation going on. Most of my friends are spending an unsustainable % of their income on rent, betting that their stock options will pay off. Of course for most, they won't, and once that sinks in the market will adjust.
I don't think it's correct to imply that the city refusing to build more units is why SF is expensive. There's only so much space in SF, regardless of how high, fast, or dense you want to build.
SF isn't particularly dense. It's twice as big as Manhattan,[1] but has only half as many people. It's the second-densest major city on paper, but it seems to me that this is because the city itself doesn't have the "long tail" of less-dense urban area merging into the suburbs that other cities have.
If you look at the downtown areas of Manhattan, places like Midtown are over 100,000 people per square mile. Neighborhoods like Rittenhouse Square in Philadelphia top out over 60,000 per square mile, and neighborhoods like Gold Coast in Chicago are around 50,000. The only neighborhoods that dense in San Francisco are Chinatown (75,000/square mile) and Nob Hill (50,000 per square mile).
There are a lot of neighborhoods in San Francisco like the Mission District that are in the 20,000-30,000 range. The near north side in Chicago is about 30,000, and there are vast swaths of low-rise areas there that could be built up (the whole west side of the near north side is semi-industrial). If built with high-rises with generous street set-backs, 50,000 people per square mile is very easy to achieve without feeling cramped.[2]
[1] And frankly, even Manhattan as room to grow. If upper manhattan were as densely built as lower manhattan, you could probably fit another half a San Francisco onto the island easily.
[2] When people think of 50,000+/square mile density, they think of Manhattan, but that's a bit misleading. Manhattan's paper density is understated because it has an absolutely enormous daytime population (almost 4 million people, or 180,000 per square mile). This is the result of everyone commuting in from NJ, CT, etc. Manhattan also has relatively poor set back and air and light requirements, no alleys (so people share street space with trash bags), tons of tourists, etc, which all make the place feel more cramped than it would otherwise be. My neighborhood in Chicago is 50,000 residential per square mile, and it can feel pretty empty if you stay away from the tourist areas. That's because land area is saved by having residential high-rises, which allows very generous set-backs for air and light and broad sidewalks.
The Gold Coast in Chicago is a particularly good example.
It's Chicago's most prime real estate, preferred by celebrities and high powered executives. Yet it's cheap enough that most people here could live there in a studio or 1br on a code monkey salary and have plenty to spend left over. This is because Chicago allows supply to meet demand, and has the Manhattan-like neighborhood of the Gold Coast in a city without nearly as much as housing demand as the real Manhattan. (Also, the Gold Coast is way nicer for reasons the parent post talks about.)
The thing I love about Chicago is its design reflects a deeply midwestern sensibility. Illinois is plain, flat land. Chicago isn't an island (MFH), or on a peninsula (SF), or sandwiched between two rivers (PHL). They could've sprawled, but instead they built this compact downtown, because it was orderly.
The town my wife grew up in in rural Iowa has less than 2,000 people. Yet, it's got this perfect 10x8 little street grid. Why? The town founders didn't want the layout to be disorderly in case the town ever became a huge city.
Grid layouts were very common from the mid-19th century to about World War II. For example, Boston's colonial-era city center is a mess, but Back Bay and Cambridgeport are an orderly grid, and the postwar suburbs outside Route 128 are sprawl. A city like Chicago did not exist in 1776, and so it has a more orderly downtown.
There's also a certain path-dependence at work. Western states were also laid out in a rectangular grid by federal land surveys, and thus arterial roads often fell on the boundaries. Subdivisions, however, can be much messier, depending on when they were developed and what the developer felt like doing. Compare Seattle (prewar) to its outer suburbs across the lake (postwar). Or inner Phoenix (prewar) to outer Phoenix (postwar).
While I admit I don't know a lot about SF, I have a hard time seeing that large scale redevelopment would make sense once an area is already gentrified, not least from an infrastructure perspective. I also think gentrification works best when it "moves around" to new places, something that seems to have happened successfully in NY.
It's not a matter of "gentrifying." It's a matter of tearing down all the stupid low-rise buildings and putting up skyscrapers. You know, civilization.
What I find deeply ironic about the Bay Area is the conflict between the whole "high tech" thing and the pathologically suburban upbringings of most of the people who work in the sector. It's people who talk about 21st century technology who aren't comfortable with solidly 20th century technology.
What I'm saying is that once something becomes gentrified (higher real estate prices, condo conversions, changes in demographics etc.) it gets a lot harder to just tear down buildings. Both from a economic, political and practical perspective.
I do agree with the sentiment in you second paragraph though.
Amazing, 21st-century technologies like "elevators" and "steel" can be used to make very high buildings: per this: http://www.slate.com/articles/business/moneybox/2012/05/face... , which I posted elsewhere in this thread, it would be very easy for developers in SF to deploy these technologies in order to provide more living space.
Good, I'm not the only one that is dumbfounded by the abysmal quality of apartments in SF and the Bay area. If you start checking out reviews for apartments, 99% are complaints about noise. Well, yes, that's because 99% of the apartments are shoddy two-story wood constructions, built decades ago. There's some modern apartment buildings up in the city, and precious few on the peninsula, but the overall level is ridiculous.
And the worst part is that not even if you have lots of money can you easily get a modern apartment, there just isn't any supply. It drives me nuts.
I think that what they mean is that the height of buildings in SF is limited in most places (except the financial district).
So while the land is mostly occupied, the buildings could be 10x bigger (each of them).
So yes.. there is some "artificial" scarcity. Yes but.
If the city was a manhattan with only big buildings, nobody would want to live there, anyway. The reason why people like SF is that it does NOT feel too much like a big city IMO
High rents in San Francisco are caused purely by scarcity. That is, everyone wants to live there, but the city refuses to allow more units to be built or make it easier to rent. This damage is self-inflicted; California liberals hate density and gentrification although such things are green, decrease broad-market housing costs thereby serving the "displaced" poor, increase urban vibrancy, and reduce traffic congestion compared to the status quo.
I saw a study that suggested the economic cost for permission to build a unit in SF (calculated by subtracting a unit's cost from the value of the land it sits on plus the cost of building it) is $600k. By contrast, it's $100k in Chicago and five figures in any major city in Texas.