In my experience, the DB layer is always the hardest to scale. At a certain point, vertically scaling system resources starts producing ever diminishing returns. At that point, the natural progression is to move to a clustered setup with multiple DB nodes, etc, but that introduces a whole new universe of complexity and failure points.
My takeaway is that proper data modeling at the initial stages of designing a software architecture is absolutely essential to long-term scalability. You really need to consider the type of data you are storing, its shape, and the access patterns involved with reading/writing it. Of course, since it is basically impossible to get this correct the first time, it is even more important to have a good abstraction layer between your business logic and your database to make it possible to change your data store in the future.
Spot on. You can only go so far with vertical scaling. While you can't just Willy nilly shard and hope to scale, modelling is key as well as introducing and embracing eventually consistency (and read your writes) all the way across stack especially on the front end experience.
Sometimes vertical scaling is counterproductive because of NUMA memory bottlenecks. In my experience the STREAM benchmark correlates pretty well with RDBMS performance.
My takeaway is that proper data modeling at the initial stages of designing a software architecture is absolutely essential to long-term scalability. You really need to consider the type of data you are storing, its shape, and the access patterns involved with reading/writing it. Of course, since it is basically impossible to get this correct the first time, it is even more important to have a good abstraction layer between your business logic and your database to make it possible to change your data store in the future.