1. Space The rents are actually in many cases less here than in other parts of the valley. (My office is $1 per s.f. per month!) A huge building boom is finishing up in the SOMA area of the City with hundreds (maybe 1000+) new apartments entering the market. This is good news for downward pressure on rental prices.
2. Convenience: Within walking distance of both my office and my apartment are countless restaurants, markets, office supplies, banks, etc etc. You can simply walk to what you need saving so much time over driving around on errands in rush hour traffic in the South Bay. There are countless take-out food restaurants open until late at night to keep you coding with a reasonable diet, with many delivering to your door. (Chinese takeout costs about $7 for a main item - roughly $10 for a meal)
3. Coolness: There are a lot of web companies in the city because it is such a great place to live. Attracting hackers to work for you may be easier here.
Even the Large Corporate Web companies are setting up major operations in the City:
4. Life Style: For those moments when you're not all-nightering on a problem, you have many options here:
a. Sailing in the Bay
b. Biking - across the Golden Gate and you are into nature in no time.
c. Beach - Many beaches in the area
d. Skiing - As close as 2.5 - 3 hours away for some great slopes.
d. Symphony/Opera/Theater/Movies galore, etc etc etc
e. Night Life - Live music, etc etc etc.
f. Wine Country - Only an hour drive to Napa County/Sonoma County - countless vineyards - great wine.
5. Variety: San Francisco has many different neighborhoods that have entirely different cultures. A few blocks in different directions will let you experience the unexpected. (Usually good, ocassionally bad)
6. Ok, you're right, I'm biased. I love living in the City.
7. Downside: The VC's and many angels are in the South Bay, but thats only a 30-40 min. drive away - or Caltrain ride.
Regarding SF vs Palo Alto. SF is way more fun. But keep in mind historical patterns. How many successful startups have been started in SF? How many have been started up in the valley?
Valley based startups are more successful, simply because there is nothing else to do down there.
I much prefer the mission for nightlife and neighborhoods like Cole Valley for living.
Regarding "where to live": I currently live in Mountain View, but I would move to SF if I could afford (although MV is not that cheap either!). SF is an awesome city.
Rent is a factor of course. I'm currently in a nice 2-bed town-house close to downtown mv, for $1700/month.
San Franciso is a much cooler place to be than Palo Alto/Menlo Park/Mountain View. YScraper's probably a good bet, I imagine it's getting pretty popular though.
The downside is that the Muni rail system is not entirely reliable. Normally it takes me 30 minutes from door to door via Muni (N line - when it's raining which isn't that often), but it can take up to 50 if there is a problem which is about 20% of the time. (Biking takes me 20 minutes or less). Of course if you work out of your apartment, commuting is a non issue.