1) Don't build a company around your FLOSS project, accept it'll be free in all senses forever
2) Start with a customer that will finance you and build slowly (focus on profit not revenue)
That's obviously not very helpful as finding that one customer is the really hard part so it's usually a hybrid of expecting what you build to be free forever and then finding that one customer.
Quote/Charge a lot. Nope more than that. Now double the price and you're roughly there.
On the plus side you'll get very enthusiastic developers that tend to be intrinsically motivated which is a huge plus once you can build that company. I'd argue that this is one of the key competitive advantages of a "FLOSS company"
When it comes to new customers...Always be aware that you're not selling your cool tech but rather selling the solution to a specific problem (basically you're selling pain-easing medicine). I feel like it is beneficial to visualize a somewhat mean guy at the other end of the table who is only thinking this one line over and over "enough with the nerd speak, how does it help me and does it roughly cost what I expect it to cost". Yes we like to imagine that we can impress people with nifty tech details and shudder that the competition might be "that horrible Oracle database every developer hates".
Hand in hand with this...charge a lot. The price serves as a signal. Just assume that evil guy you're talking to only cares about getting back to that golf course and won't even look at the number that closely as long as it's fitting his expectation. Yes that means charging too little can kill your deal.