Mathematics grants are relatively small, on the order of $20-60k a year. But senior mathematicians (of which Gowers is most certainly one) typically have larger grants. In mathematics, $1-3k (the publishing fee for PLoS) seems like a rather high barrier -- but I hope the coming years prove me wrong here!
Libraries also have infrastructure in place - they already host their own websites, so running a journal just requires adding another page to it.
The most annoying part would be coordinating the different reviewers, but I think that job would be done by the editorial boards, composed of researchers in the specific fields.