My wife is the same way: she works best in a coffee shop, with background noise. My hypothesis is that people who are able to multitask (ie. not me) need a certain level of stimulation to absorb the attention that would otherwise crystallize into distractions.
For me, when I'm able to concentrate, the background noise is irrelevant. I simply do not hear it.
That's exactly what it is, for me at least. The slightest noise of something other than nature is enough to throw me completely, the best I can do living in a city is to work in an environment where the background noise has a masking effect. It's either no noise or a constant background drone loud enough to mask any single interrupting sound - anything in between is unworkable for me.
As much as I love the mountains and forests, I've come to realise that I like the idea of solitude a lot more than the reality of it. And so I find myself living in large cities, constantly seeking noise because it's the only way I can find peace.
When the ambient sound level gets too low, I get a strange feeling in my ear -- kinda like a hand or leg "falling asleep". It's really distracting.
White noise, and ambient "study" music is great -- it's like a null placeholder, I can focus and I remember hearing nothing instead of hearing silence.
For me, when I'm able to concentrate, the background noise is irrelevant. I simply do not hear it.