Have you looked into what SES or other email services provide? Sending emails is easy, while actually getting them delivered is harder. You have to make sure you're not getting flagged as spam, can handle bouncebacks, etc.
Not getting flagged as spam isn't actually that hard, though. Besides, if you're using SES or some other hosted SMTP service, you still have to set up SPF for your domain, so you haven't even really gained much comfort. The only really useful thing is to gain an ip address with a high reputation, but you can generally get those at any reputable hosting provider as well. Just don't try sending emails from your residential internet connection.
Here's one discussion: https://stackoverflow.com/questions/371/how-do-you-make-sure...