It's certainly feasible. Your containers run on a public cloud, just as Heroku does, which enables access to things like logging-as-a-service. But you'll soon find that such services are just as horrifically marked up as Heroku, because they too are middlemen who need to make money. (I'm referring to services like Loggly or Logentries, to continue with the logging example.)
We've been really enjoying AWS's hosted ElasticSearch, PostgreSQL, Elastic Load Balancer, etc. Most of the benefits of a hosted service at the cost of an EC2 instance.
Definitely agree that the other services are marked-up too! The issue is one of leveraging developer time at a dynamic startup. At some scale, it becomes cost-effective to manage your own application cluster. At some (usually larger?) scale, it becomes a good idea to administrate your own database cluster. We're definitely not big enough to consider managing our own RabbitMQ or log aggregator or other ancillary services from a developer-time opportunity cost/benefit perspective, though!
We've been really enjoying AWS's hosted ElasticSearch, PostgreSQL, Elastic Load Balancer, etc. Most of the benefits of a hosted service at the cost of an EC2 instance.