Hacker News new | past | comments | ask | show | jobs | submit login

The author of this essay is beating a dead horse. "NoSQL" (I really dislike the term) has proven its value over the years and deserves a seat the table. People will always misuse technology or implement it poorly, but I don't think it warrants yet another oversimplified "SQL vs NoSQL" rant.





Genuinely curious since I have only seen bad things said about NoSQL in most places over the years, what benefit does it provide other than scaling?

It's unlikely that a company needs to invest in data scientists or even thinking very hard about their data organization until the scale of their data is already pushing the bounds of what most RDMMSs can handle. NoSQL is nice if you are planning for scale, since it will seamlessly get big without much thought or any significant changes to the performance. There are no "gotchas" that will cause very long-running queries or that will lock a huge number of rows, so performance is very good and most importantly very stable. This is largely due to the fact that you have to think more deeply about your data access plan up front, since almost any query is a table scan (which, ideally, never happens). I think that this is secretly a benefit in that it forces people not to perform ad-hoc queries on databases, and to think of their databases in terms of the APIs that they have built over their databases, because the databases are not going to efficiently support any other sort of access than the access plans included to support those APIs. I would lean in favor of a NoSQL database to back a production networked service because the upsides are helpful in this case (stable performance, easy to scale) and the downsides are not significant (have to plan your APIs up front -> going to do that already, no ad-hoc queries -> not going to run ad-hoc queries on a production database anyway).

> It's unlikely that a company needs to invest in data scientists or even thinking very hard about their data organization until the scale of their data is already pushing the bounds of what most RDMMSs can handle.

I work in an engineering data science consulting group for a major UK PLC. Almost all of our clients have mainly SQL based data sources, and none of them really require cloud scale. In fact I don't think I have come across a client whose data I can't handle in SQLite on my laptop.

However, the problems we solve do require data scientists - mainly STEM MScs with a smattering of PhDs who are capable of combining statistics, domain knowledge and the ability to work within a variety of organisations and present.


Seconding that. Several ‘AI’ startups I worked with, do not require a massive amount of data to create MVPs/first releases.

The do however invest early in sufficient data science functions to have a good liaison between the researchers and data acquisition.


> It's unlikely that a company needs to invest in data scientists or even thinking very hard about their data organization until the scale of their data is already pushing the bounds of what most RDMMSs can handle.

I disagree. Data science is largely about applying and interpreting data, and the value of that isn't really dependent on having data that is problematic for RDBMSs due to scale.


If you think RDBMs can't horizontally scale you have not kept up with the developments :) Look into https://docs.citusdata.com/en/v8.3/get_started/what_is_citus...

Immutable/stream databases are a fundamental innovation, even if everything else turns out to have been unsustainable performance hack.

I never really understood how anyone expected to have a sensible debate on "thing" vs "anything that isn't thing" (I realize "NoSQL" is somewhat better defined than that, but not by much).

Agreed. The more meaningful debate would be "relational" v. "not relational", albeit marginally (swapping "not relational" for something more specific like "document" or "key-value" would be even more meaningful). There are indeed relational databases that are technically "NoSQL", like Mnesia.

well said



Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: