Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You can just shove json or XML you receive into a SQL database, too.

That can be good strategy when dealing with changing data formats:

1) Import the data "as is" into the database.

2) Process the data, extracting the data you need for querying.

3) Whenever parsing fails, figure out why and fix it (update your source code, throw away bad data, whatever), then rerun the 'process' step for that data.

Reparsing can be made reliable if you use transactions in the step 2. With a NoSQL solution, it may be hard to guarantee that you don't lose a few records ('may' depends on the specific NoSQL solution and the amount of manual work you are willing to do to restart your pipeline). (You can also postpone discovering your data problems, but that's delaying the inevitable)

What NoSQL is particularly good at is running on multiple machines. It achieves that by giving up some letters of ACID.

It also typically makes it easier to store and query unstructured data than SQL databases do, but SQL databases are catching up.



Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: