The main benefits of Amazon is that it:
a) allows you to scale down i.e. buy services in smaller portions than complete physical servers
c) integrated features
You could probably pay for one devops position once your infrastructure gets to 10 physical servers.
The best resource to research different providers are the webhostingtalk.com forums. You can also contact me and I will do my best to advise you based on your desired criteria.
*Full Disclosure: I'm the founder/owner of a dedicated hosting company
I could go on and on about those, but other options were even more painful.
I've been running three 32GB servers (each with 3TB storage) with them for 2+ years now and the only outage I've experienced is the switch (5 port GBit) dying once. Hetzner tech replaced it in under an hour.
These three servers cost me €263/month (that's total, not each). Included in that monthly price is an additional IPv4 for each server, a private 5port Gbit switch, remote console access and 300GB of DC backup space.
There are probably better deals available now (i.e. more RAM at the same price) than the one I'm on since it's old and not offered on their site any more (/makes note to self to call Hetzner sales)
I wouldn't want to do it, which is why I'd rather work for somebody who'd pay for AWS, but I think there's a thing in there somewhere for those who want to dig.