EBS performance is an issue - Yes it is, don't use EBS. Use instance store and push to S3 for persistence. If you need performance I/O for something, there is likely a separate service available that pushes your I/O bottleneck further away (RDS, ElastiCache, SQS, etc.).
AWS costs more for weaker CPU - Indeed, this can be the case. But, it's often cheaper (but not much) to put up an Elastic Load Balancer with an Auto Scaling Group and dynamically support your peak traffic than it is to pay for an enormous VPS that sits idle 60% of the time.
As these benchmarks suggest, I agree that if you're only using one EC2 instance (and you can't get away with a micro), you should probably be investigating other solutions. If you want to architect your app/project/service/whatever to be more distributed and fault-tolerant, AWS can probably make that easier (not necessarily cheaper).
On a side note - From a marketing perspective, Amazon probably should have launched PIOPS as a separate product from EBS. The idea that "EBS sucks" is pretty firmly entrenched, so EBS+PIOPS is fighting an uphill battle.