I'm going to run some quick numbers for B2C software, based on the 1k downloads a day estimate and a few guesttimates from my experience:
Page views: ~4k
CPM rate: ~$2.50 ~ $5 (CPM rates for OSS sites that target businesses which spend money on big business stuff can be fairly high. For consumer OSS, not so much. The salient feature of consumer OSS users is that they like getting things for free -- how much would YOU pay to advertise your software to them?)
Ads: $600 per month on the high end
Donations: 10 a month or so, average will be for below $5.
Paid Support: For B2C software? I consider myself an incurable optimist. I am feeling cured now.
My suggestion: if you want to make money selling software, then you should try selling software. You're worried about your community forking it. Your community is not capable of forking it, nor do they have the inclination to do the amount of work required to do so. They are here for the free stuff. It is highly likely that there is no copy of your source code anywhere except on your committers' computers and whatever your centrally accessible repository is. If you were to turn that off one day, the source code would no longer be available. And if you were to disable download of the free version, somebody might actually notice.
You are worried a pro edition will make you your own biggest competitors. Welcome to every shareware business ever. We all compete with ourselves, effectively. It isn't trivial but it isn't impossible, either. (Hint: feature limitations on behavior which identifies the people who most need your software work great.)
[Edit to add: You may consider white-label versions to customize to the requirements of studios&etc needing marketing devices. Take a look at http://www.mediabrowser.tv/images/gallery/g5.jpg for a second. I wouldn't want to have to be the guy who made the sale, but I could sure see Paramount or whoever deciding "What the heck, that would be worth one hundred thousandth of the marketing budget for Ironman" ]
Essentially, he has a really good product that sees 1000 downloads a day. It's open source, and he can't afford to keep maintaining it for free. He's reluctant to charge money for it, and gives a good explanation as to why.
It sounds like he's so worried about disappointing people that he'd rather abandon the project entirely than do the logical thing: sell it as a product.
Is it an art thing? Something psychological? Is he also experiencing this effect?
We tend to think of Commercial Software as this Big thing that Big teams from Big companies spend years to develop. Windows took 400 man-years or whatever to develop and it only costs $79 a copy. Why should my thing that I spent less than 500 hours on be worth that much?
From the outside though, nobody else can tell how hard it was to build. They only see that it solves their problem and that it's valuable.
It's a false dichotomy (hard to build vs. valuable). We all have stories of the 3-day hack we built that made the company $400K, and we've all seen marketing guys go nuts thinking that our little internal tool could be re-packaged as the Next Big Thing. If only we could muster that sort of enthusiasm about our own stuff, we'd be a lot more likely to put a price tag on it.
That's the post where Jeff Atwood explains the 90% software piracy rate. I think it's one of the most illuminating posts he's ever made. If you close the source, you will probably immediately begin to experience piracy, and it will probably be at a rate around 90%. Make sure you have a solution for that problem in your monetization scheme. (This also disproves that notion that most people are willing to pay for software. I myself haven't paid for a piece of software in years [though I don't pirate ANYTHING either].)
Consider also the GPL, which clearly states that the rights it grants are irrevocable (section 2). Even if the source were closed for new versions (and if the source were closed, and this is absolutely vital, the GPLed parts would have to be rewritten since the GPL prevents inclusion of GPLed code in closed-source projects [section 5]), the old versions would still be open source. If the software is popular, I do not find it hard to imagine someone would fork it... or just keep using the old version.
Keep in mind the author's comment that the software has benefited from being open. Closing the source would also mean decreasing quality.
Finally, the goal of keeping your software hackable is noble. Hackable software is a gift to programmers everywhere. It's not unreasonable to want to continue offering this.
You will be charging for the installers and branch management / stabilisation.
Those who want your software for free will be able to build it. Some will probably publish their own binaries, but I bet they won't maintain them. In the best case they will just build off the master branch from time to time.
Many people want software to just work and are ready to pay for the convenience.
So it may or may not work out how you hope.