Hacker News new | past | comments | ask | show | jobs | submit login

I've spent my career answering this question at myriad companies. In a few cases, the database was actually a bottleneck. In the vast majority of cases, the database was just being used very poorly and was no longer a bottleneck once used correctly. The exceptions to this usually involve exotic data models, which isn't most web apps.

Of the few cases where the database was the bottleneck in practice, I cannot think of a single cases where it should have been the bottleneck in theory. The limitations were always a property of the specific database implementation being used, not intrinsic to databases as a class of software. For example, most open source databases have notoriously poor storage throughput as a product of their architecture; faster storage definitely helps but it often solves a problem that may not exist in other database implementations. In these cases the problem is an architectural mismatch.

Sometimes architectural mismatch is an engineering problem. A handful of workloads and data models are currently only possible with custom database engines e.g. high-throughput mixed workloads or data models that require continuous load shifting. As a practical matter, the exotic cases that require a custom database engine are also the cases where no company should try to build it themselves without a deep expert -- they are considered technically exotic for good reasons.

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact