I'd say it makes no sense to buy your 32gb, 16-core single point of fail, waste $40/mo and half a day setting it up and then have to keep it running yourself when you can easily spin up an API in API Gateway/Lambda that dumps data into dynamo/simpledb and front it with a static site in S3. That setup scales well enough to be mentioned on HN without getting hugged to death and is kept running by someone else. And if it is, literally, free for the first year, how is that not a no brainer?
You make a valid point about credits, but after one year you are hit with a huge bill. Hiding the costs of your infrastructure isn't always a good idea.
Why not buy 3 servers in different data centers for $120 a month. Is that enough redundancy for your infant startup? Why do you need to pay $100 month for a single 4gb, 4 core VPS from Amazon when you can get 38 cores, 120gb and 16tb disk dedicated servers for the same price?
That's the thing, it's really not much of a bill if you do it right. Aside from possibly the S3 bandwidth bill, which could easily be avoided by using something like Netlify, nothing in that setup I described will get prohibitively expensive until you reach significant scale. Yes, EC2 is expensive compared to other providers. So don't use EC2. You can build a startup on Amazon's managed services for a lot less and you can build in a way that's pretty easy to get off when the time comes. Just make sure to build in the proper adapters around your business logic. It's not hard to write code that works equally well under Lambda and Express and I assume you can do likewise for Java, C# and Python if you'd prefer to use those runtimes.
There are certain classes of startup that don't make sense in the cloud...as others have mentioned, if you've got heavy storage or egress requirements, cloud quickly gets very expensive. But for the majority of startups, particularly B2B, that's not the case and there's ways to build in the cloud that are damn close to free, both in dollars and ops resources. There's a group of technically-minded people who I believe are addicted to managing their own boxes. They got comfortable with doing it back when it was required and now don't want to learn a new way of doing things. But that new way really is faster and cheaper for most small businesses. And if people take some time to experiment with Serverless, Apex or some of the other tools that have sprung up around Lambda/Cloud Functions/Azure Functions, I believe most will see the virtue in moving on from self-managed servers for most workloads.