My senior project was creating a business plan for India. I guess the professor was farming for ideas. The one thing he berated over and over was that due to the multiple municipalities and mass corruption, conducting business between their states is not a trivial matter. This means that there is no group cohesiveness. Pollution knows no borders and I do not think this will be an easy problem for the Indian people to solve due to said internal struggle.
If I had to guess, the cause could be due to how much focus is spent on non-secular issues. There is little time left for issues like better air, water, roads, etc... If this hurdle is cleared, you then have to get past the selfishness and corruption built into the system. Set your gods, ideology and your selfishness aside for half a minute so you can at least breath a little easier. It's going to take some work and sacrifice, but in reality it's not that bad.
In India, a vast majority of the population doesn't realize the growth by letting others grow concept. The predominant sentiment is to grow at someone else's expense, meaning I have to grow and others shouldn't or even if they did they have to grow less than me. This manifests in every step and walk of life. This leads to all sorts of corruption. Some guy who owns a key piece of land won't sell that land for a public project, just because a politician who doesn't want that project to succeed will back him up and prevent him from selling. This happened in Bangalore when a key arterial road was being developed. As soon as word got out, some local politician flooded a key government land with slum dwellers and overnight they occupied that land and prevented the road construction. Stories like this are the norm in any land acquisition.
Of course there are exceptions, and that's how this country is seeing modest growths of 6-8%. I say modest because, our baseline is fairly low, as compared to other economies. I feel so bad, we have the highest inhabitable land mass of any country, some of the most fertile regions, no adverse weather in most parts, bio-diversity to die for. It's like someone handed this climate on a platter, yet we don't realize that. We are just destroying this paradise day by day.
How does a local government do something if their constituents flat out refuse it?
Secondly they grabbed all the land around NCR(National Capital Region) which resulted in prices of properties reaching simply affordable levels. This only increased the pressure on the core part of Delhi.
It is an example of evil Governance.
New Delhi isn't even in the forty most densely-populated cities . The city feels crowded because infrastructure has not developed with the population. If you want to see what happens when a government artificially cap housing supply, visit San Francisco.
An interesting thing to notice from your link, Union City, Guttenburg, and West New York (all in New Jersey) make the list while New York City does not. This is only because of how the lines are drawn.
There is a lot of land around Delhi to develop the city laterally instead of allowing 4 floor construction on a 50 square meter plots on a narrow street. How to you allow for electricity, water, and commute without creating bottlenecks. Road are already conjested and new ones cannot be constructed easily. You can go underground and that's happening (Delhi Metro Project) but it's neither easy nor cheap to build an underground transportation system under a crowded city. It all comes down to appeasement and corruption going hand in hand.
SFO might be on the other side of the spectrum with its draconian zoning laws though.