Aside from English, I speak Spanish fluently, Japanese conversationally, Tagalog passably, and I'm learning Portuguese and Italian. I know enough German to sing 99 Luftballons, and enough Latin to translate some of the classics. I wouldn't say any of these have been at all useful in my programming career. I've worked with offshore programming teams in Malaysia, the Philippines, India, Belarus, Mexico, and Pakistan, and they all spoke English better than I could ever hope to speak their native language (except Spanish, but even then the guys in Mexico I work with were schooled in the US).
You could try learning something really obscure. As far as I know I'm the only hacker within 1000 miles who speaks any Mongolian as well as fluent English. Someday, two buyers are gonna show up in that market and keep bidding against each other till my rates go to the moon! Dreaming about that day is much more fun than trying to compete with tens of thousands of (French|Chinese|Spanish|Russian)-English bilingual programmers. In the mean time, I try to hang out at a webforum from time to time and answer random tech questions in Mongolian, which makes me feel all warm and fuzzy. (Or less charitably: I can't sell my skills, so I give them away for free).
To be perfectly serious, the recommendation for what language is "useful" to learn really depends on your own language learning skill --- which is a combination of inborn talent, knowing what learning methodologies suit you, and experience with compensating for a bad environment (e.g. I assume you live in the U.S. and attend an Anglophone workplace everyday).
Spanish is a pretty typical answer to this question not because it's the "most useful" but because it's not that hard for English speakers to learn, it's more widespread in the U.S. than other foreign languages, and it even has some applicability in tech (a lot of people I've seen suggesting moving to Latin America to cut your cost of living while you bootstrap your startup). Chinese/Japanese is somewhat harder to speak, a lot harder to read, a lot harder to get a visa to the country as a one-man startup, etc.
My favourite quote from Eat, Pray, Love was (roughly) "Why would you learn Italian? So that if they ever successfully invade Ethiopia again, you'll know a language which is spoken in two countries?"
With that in mind, it makes sense to learn Spanish, which is spoken through much of Latin and South America. Chinese (Mandarin) has replaced Japanese as the 'next big economic language', so if you're looking to converse with a large new market, then perhaps Mandarin.
But for me, I dream of taking up French again. Limited global reach. No economic benefit. But I love the way it sounds coming out of my mouth. Does a form over function answer negate the 'useful for a programmer' part of your question?
Italian is spoken in the canton of Ticino in Switzerland, and is also an official language in the Croatian region of Istria, which was long a possession of the Republic of Venice. That said, yeah, it's not terribly useful in the world at large, but the food is good, the girls are beautiful, and the countryside is stunning in its variety.
French is still spoken in many African countries although its not likely to translate into economic benefit anytime soon. It can also be useful if you want to do business in Montreal, Canada, which is relatively big in the video game industry. De mon côté, j'aimerais bien apprendre le Mandarin :)
Lots of important products are designed and developed in Japan and usually are translated into foreign languages quickly but you could get a jump on developing for whatever Sony or Nintendo is doing with Japanese. Japan has a big internal market you might want to do l10n for it sometime, also.
The products designed in Japan (or California) are likely to be manufactured in China. If you want to participate in the final production and delivery of products you program, you'll want to consider Chinese. China's internal market is a desert of failure for software sales, though.
So on a technical basis I'd suggest 汉语 (Chinese) or 日本語 (Japanese). Too bad they're among the hardest written languages in all human history to learn. Save yourself the trouble and pick Spanish, the Internet's third or fourth most popular language. It's not a hotbed of technology but if you're living in the USA, you'll need it soon enough if demographic trends continue.
After 4 years of studying Japanese, I can speak enough to carry on a conversation with strangers in Tokyo, order food in a restaurant, and flirt with girls. Writing is a different story, though. I learned hiragana and katakana, but nowhere near the 2000 kanji that a high-schooler knows. I doubt most people would learn enough Japanese to be able to work in a highly technical field where Japanese is spoken and written.
I'd vote Spanish (lots of countries speak it, which makes it a good language). Or Chinese - more people speaking it than any other language, I think.
BTW, I started learning Spanish using the "Pimsleur method". It's an amazingly easy and quick way to learn a language, If you're looking for a way to learn a new language, I recommend looking it up.
I speak English and Spanish and was using the Pimsleur CDs to learn Japanese. I can't recommend it enough. I gave up when I had no one to talk to and yet after several years, I still remember quite a bit.
I know a tiny bit of German and I'm learning Japanese (in Tokyo) right now. I can say that one of the advantages of a language like Japanese is that it really gets you thinking about languages in general. You have to learn to think a different way which is quite interesting.
I guess its sort of like if you learned C++ first, then your next language to learn should be something totally different like Scheme, Scala, Prolog or Erlang - not C or Java.
Lots of 'free software' and hacking stuff goes on in Germany. Of course, the level of English in Germany is superb, so you can still probably talk to them there.
