(2) set up mail server, create account(s)
(3) reset public email account with domain registrar + hosting facility to newly set-up email addresses
(4) kill public email account
You only need one for a very short while to bootstrap your own email server, you don't need it forever.
But I'll bite, if your mail server goes down - and it will - you now have no email. You can't email your hosting support. You try to log in to your host control panel to do a reboot or raise a ticket but forgot your password. You can't get the reset email. Somehow you manage to login and raise a ticket but won't get any email notifications when the technician responds asking for your server id number...