Prestige is not something you decide you want in the morning and have by evening. A lot of the most prestigious universities are very old, and so it is with the most prestigious journals.
The whole academic culture needs to be changed. People like Gowers who already earned their stripes can begin to change attitudes. 99% of mathematicians can do nothing except hope they can get a permanent job one day. And even then their institutions are going to force them to undergo yearly evaluations where they have to give an account of what papers they wrote that year (too bad if you are working on a multi-year breakthrough) and whether the papers were published in journals they deemed respectable (universities are also competing with each other for prestige).
Another problem is that too many mathematicians are devoutly religious when it comes to speaking out. Any kind of rocking the boat is likely to affect your karma (chance of getting tenure/promotion/funding).