Most of the CakePHP are slow complaints are more issues of not knowing how to use it. Suggest if you really think CakePHP is being slow set $recursive = -1; in all of your models and use Containable to only fetch related model data when needed. You should be able to hit tens of thousands of unique visitors per day without caching no problem.
Remember that you will need to add indexes to the database manually. Cake won't screw with your database by adding them itself.
I've done xdebug profiling and cake does add some non-negligiable overhead. However, 90% of the issue here is that the host is really crappy, and the latency on the db is bad, so view caching is helping. It'll be moved to a new host in 48-72 hours.