It's incredibly difficult and time consuming. I've found that a good compromise is to buy a domain name (or use one you already have), and pay for a reseller hosting account with a reputable service provider. You'll get a level of customer support that is far above a standard hosting account, full control over your email settings, and you'll have full root/WHM access to your account. As a bonus you can spin up a website whenever you want for testing or production, and make back your monthly fees by selling webspace to a single client. It sounds like a lot to just have good managed email, but I only pay about $15/month for the reseller account, barely more than a regular hosted account.
I went with A Small Orange, they have incredible support and they stay on top of abuse reports so their emails servers never get blacklisted. I've also used Tiger Technologies in the past and they are superb as well.
With that said, I'm still considering replacing Gmail with something like Fastmail even though I'm satisfied with the email service from ASO. Having a web-based account outside the reseller hosting umbrella makes two-factor auth a lot easier to manage.
You can't run an email server from a residential address b/c most ISP's block port 25 and 587. That being said, there a project called mail in a box, that attempts to be a plug and play solution. You'll have to host it on a VPS provider (Digital Ocean, for example) and even then you'll have a hard time keeping your email from being black holed by major email service providers like Google.
