Hacker News new | past | comments | ask | show | jobs | submit login

It works great at a small scale. It's really convenient to have all the data at hand and be able to manipulate it directly in your programming language without having to go through SQL, which, no matter how comfortable you may get with it over years of experience and/or no matter how much library/ORM/etc. you throw at it, is always a foreign imposition into your programming language.

If you know beyond a shadow of a doubt that you will never leave "small scale", it's honestly not a terrible approach. It's also easy to define how to backup "the one file that is everything". You also ought to be confident that the worst case of the One File becoming corrupted is something you can deal with. (Although this is an often-overlooked issue in systems design anyhow.)

I've got some systems that have bits and pieces of this idea... but I'm really, really confident they'll never scale any bigger because for the systems in question it's simply impossible they ever would. I've got one that is, for instance, bounded on the set of wiki pages of a certain type that for it to grow into the 10,000s of pages where the approach would finally start to creak (from the "dozens" it has been in for the last several years and to all appearances will remain indefinitely) would require so much human effort to create those 10,000s of pages that I'd certainly have the budget to upgrade the approach anyhow as it would be negligible compared to the rest of the effort.

It also helps if this not something you're loading per HTTP request; in my case it's one-time on startup of a persistent service.

But you need to know you understand the relevant scales, because this is an immense trap if you don't do the math (it's not hard math, but it's math). Your prototypes work well and are easy to write and then are just total trash when they go to production, and presumably, if the engineer didn't do the math in the first place, it will emerge that they don't understand why the system is crashing in production either. Huge, huge trap. I've not quite seen this exact one with JSON files but I've definitely witnessed the "prototype that any experienced engineer could have absolutely guaranteed would not scale in about 5 minutes of examination".




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: