Stuff I have read indicates that, ideally, you should be able to convey it verbally without a lot of explaining. If it isn't clear whether it is "4" or "for" or "fore" in the domain name, you have a problem. I ran into that issue (of not being able to readily convey it verbally) with my first domain name so firsthand experience backs that up (though the issue wasn't a number).
