
MongoDB gets support for multi-document ACID transactions - uptown
https://techcrunch.com/2018/02/15/mongodb-gets-support-for-multi-document-acid-transactions/
======
adwhit
MongoDB has successfully played the 'hype first, features later' strategy. Now
it is well on the way to being a decent swiss-army-knife database.

The RethinkDB retrospective[0] contains a lot of insight into how MongoDB has
succeeded despite being vastly inferior on a technical level back when it
first launched. I have to admit them a certain respect for executing their
strategy so successfully.

Choice quote:

 _Every time MongoDB shipped a new release and people congratulated them on
making improvements, I felt pangs of resentment. They’d announce they fixed
the BKL, but really they’d get the granularity level down from a database to a
collection. They’d add more operations, but instead of a composable interface
that fits with the rest of the system, they’d simply bolt on one-off commands.
They’d make sharding improvements, but it was obvious they were unwilling or
unable to make even rudimentary data consistency guarantees.

But over time I learned to appreciate the wisdom of the crowds. MongoDB turned
regular developers into heroes when people needed it, not years after the
fact. It made data storage fast, and let people ship products quickly. And
over time, MongoDB grew up. One by one, they fixed the issues with the
architecture, and now it is an excellent product. It may not be as beautiful
as we would have wanted, but it does the job, and it does it well._

[0] [http://www.defmacro.org/2017/01/18/why-rethinkdb-
failed.html](http://www.defmacro.org/2017/01/18/why-rethinkdb-failed.html)

~~~
SCdF
> MongoDB has successfully played the 'hype first, features later' strategy.
> Now it is well on the way to being a decent swiss-army-knife database.

I have no idea how capable MongoDB is these days, as I haven't used Mongo in
years (and even then it was not for long).

However, I do not know any developers who, after living through the "hype
first, features later" strategy, have been left with a positive enough opinion
of MongoDB to ever want to use it again.

~~~
stevenwoo
Epic had a post mortem blog post here that mentioned in passing they had
stumped all the experts they could find to look at unsolveable issues they had
with MongoDB.
[https://news.ycombinator.com/item?id=16340462](https://news.ycombinator.com/item?id=16340462)
I kind of assumed the fix is going to be a rewrite with Postgres or MySQL.

~~~
Thaxll
\- That's not what they said.

\- You think people replace a MongoDB cluster by a single Posgres instance?
You guys should really use HA, cluster in real life and stop reading reddit /
HN and the hype behind PG, with 3.5M+ CCU no one would use an architecture
with a single master / slave ( that's what pg is ).

MongoDB / MySQL have bad press by people that never used it in real life and
just repeat what they read online.

I could tell you horror story about pg not have an official replication system
until 2011 when pg 9.0 landed.

~~~
woolvalley
You can cluster with postgres & mysql, but you have to implement the
clustering / sharding logic yourself.

Nowadays I would just use redis & cassandra if you need something beyond a
collection of postgres instances. Most projects do not.

~~~
dankohn1
For clustering MySQL, take a look at Vitess. It was developed at YouTube and
recently adopted into CNCF:

[https://www.cncf.io/blog/2018/02/05/cncf-host-
vitess/](https://www.cncf.io/blog/2018/02/05/cncf-host-vitess/)

(Disclosure: I'm executive director of CNCF.)

------
lkrubner
Most software developers have a negative impression of MongoDB, based on the
many flaws that it had back in 2010. Among the people who did the best job of
documenting those flaws, was Kyle Kingsbury, in his Jespen series:

[https://aphyr.com/posts/284-jepsen-
mongodb](https://aphyr.com/posts/284-jepsen-mongodb)

But it is important to realize that the team at MongoDB has actually been
working with Kingsbury, for several years now, and they have slowly and
patiently fixed the problems he identified. Consider how the situation had
evolved by 2017:

MongoDB 3.4 Passes Jepsen – The Industry’s Toughest Database Test

Jepsen Evaluation Demonstrates MongoDB Data Safety, Correctness & Consistency

On February 7th 2017, Kyle Kingsbury, creator of Jepsen, published the results
of his tests against MongoDB 3.41. His conclusions:

"MongoDB has devoted significant resources to improved safety in the past two
years, and much of that ground-work is paying off in 3.2 and 3.4. MongoDB
3.4.1 (and the current development release, 3.5.1) currently pass all MongoDB
Jepsen tests….These results hold during general network partitions, and the
isolated & clock-skewed primary scenario."

[https://www.mongodb.com/mongodb-3.4-passes-jepsen-
test](https://www.mongodb.com/mongodb-3.4-passes-jepsen-test)

MongoDB has become an excellent document-store database. If you are still
repeating FUD from 2010, then you are simply out of date. It's time to come up
to speed on the reality of 2018.

~~~
gshulegaard
Well...to the best of my knowledge the Mongo DB BI connector in 2018 is still
Postgres:

[https://www.linkedin.com/pulse/mongodb-32-now-powered-
postgr...](https://www.linkedin.com/pulse/mongodb-32-now-powered-postgresql-
john-de-goes/)

Has this changed?

~~~
kstirman
Yes this has changed. The BI Connector 2.0 is written in Go and uses the MySQL
wire protocol: [https://docs.mongodb.com/bi-connector/master/release-
notes/](https://docs.mongodb.com/bi-connector/master/release-notes/)

------
btown
Lest we forget, we should wait for an Aphyr/Jepsen analysis before jumping to
use this in production: it could go either the way of
[https://aphyr.com/posts/284-jepsen-
mongodb](https://aphyr.com/posts/284-jepsen-mongodb) or
[https://jepsen.io/analyses/mongodb-3-4-0-rc3](https://jepsen.io/analyses/mongodb-3-4-0-rc3)

~~~
amelius
Good point!

The code is almost completely new, so definitely a thorough testing is
required.

------
overcast
Next up will be SQL compliance, and we'll be back to a relational database.
I'm curious as to what the impact to speed will be, and what the use cases for
these types of databases is now that the major SQL players support JSON.

~~~
drmirror
I beg to differ. (Disclosure: I work for MongoDB.) Using JSON as your data
model, rather than relational tables, lets you build different applications
that don't need multi-document transactions as often, because the data is
already together in a single document. But when you do need multi-document
transactions (a small percentage of applications do, and only few use cases
inside those applications), they are now available. There is no speed impact
on cases when you don't use them. And most of the time, you shouldn't use
them, otherwise you wouldn't be capitalizing on the advantages of JSON. I
think that's a game changer, but then again: I do work for MongoDB.

~~~
jeremiep
Its usually only after a while you realize almost every piece of meaningful
data is relational. It just didn't look that way when the project started. But
now you're committed on the wrong database and its very costly to switch back
to SQL.

Literally every project I saw using MongoDB ended up going back to SQL within
the first 2 years after realizing the data is indeed very much relational and
theres no clean way to model it using documents.

You always end up with either tons of duplication across documents, which is
hell to maintain, or tons of multi-document queries with hacks to look ACID,
which is also hell to maintain.

Sure Mongo makes it easy to prototype applications, but it makes it very
complex to build robust and maintainable software. Its especially bad if you
think your data isn't relational, because it almost certainly is.

Disclaimer: I believe Datomic to be the game changing database; because it
values simplicity and composition and these attributes drive the entire
design.

~~~
fnayr
Video games storing player data are a _great_ example of nonrelational data.
I'm intending on writing a blog post after I finish my game detailing the
structure of data I store and why it was so perfect for mongodb.

~~~
mustardo
On the surface it sounds like you might have a case for Mongo, lookout for
scenarios like...

* Trading in game items between two users (needs multi document atomic locks if you don't want duplicate or lost items) assuming your "schema" is a document per user

* You want to rename or restructure an attribute in the future, with no schema it's not possible change migrate data easily without writing ad hoc code (maybe you can use third party tools) or changing queries to expect data in multiple "schemas" which quickly gets painful

Good Luck!

~~~
mygo
> You want to rename or restructure an attribute in the future, with no schema
> it's not possible change migrate data easily without writing ad hoc code
> (maybe you can use third party tools) or changing queries to expect data in
> multiple "schemas" which quickly gets painful

You can have schemas with MongoDB. There are various libraries to facilitate
database design by schema specification.

Also renaming or restructuring your data is not necessarily an easy task with
SQL. The nature of a database dictates that how good it works for your
application depends on up to how well thought-out your schema is. Having to
change your schema around is tasking. One of the reported advantages of
document stores when they were becoming trendy was that it was easy to change
your schema since your schema is essentially determined and regulated at the
application layer.

Also MongoDB has ACIDic transactions now (freaking finally) so if it’s as-
advertised then I feel like half of your argument is not really a strong one
any more.

~~~
chris_wot
If you need a schema, then go with an RDBMS.

------
alyson-cabral
I'm the Product Manager on the Core Server responsible for the multi-document
transactions project. For those of you interested in learning more about how
we're building transactions in MongoDB, I suggest checking out this video that
discusses creating WiredTiger timestamps to enforce correctness in operation
ordering across the storage layer. The description is presented by Dr. Michael
Cahill, the co-founder of the WiredTiger storage engine aquired by MongoDB.
[https://www.mongodb.com/presentations/wiredtiger-
timestamps-...](https://www.mongodb.com/presentations/wiredtiger-timestamps-
enforcing-correctness-in-operation-ordering-across-the-distributed-storage-
layer)

~~~
jbellis
Hi Alyson,

Thanks for the link! It looks like the video covers using timestamps to
implement replicated mvcc correctly from primary to other replicas.

The interesting next step is how you go from there to multi-document across
different primaries. Or is this limited to documents in a single shard?

~~~
alyson-cabral
In 4.0, transactions will just be across replica-sets. The following release
will have transactions across the entire sharded cluster (across multiple
primaries).

------
jaequery
Mongodb can be quite a nightmare once you start requiring anything more than a
1:1 relationship, which is pretty much any kind of app that is doing anything
meaningful. Having to resort to doing things like map/reduce for a simple
group by / order is not the way to go IMO. I think you later truly realize the
beauty of SQL once you get far down that rabbit hole.

Initially I rode on the Mongo's NoSQL bandwagon when I saw that you can just
save a JSON hash and thought that's the coolest thing in the world. But ever
since I tried out Postgres's JSONB, I just can't go back to Mongo anymore.
With Postgres, I have the best of both worlds, performance, relational data,
and reliability. I don't have to sacrifice any of it. Also, I don't know who
codes using raw SQL, it's been years languages have had ORMs that made queries
look just like a Mongo query.

Also, for anything else, like a super simple requirement of saving data
(JSON), Firebase have fit that role perfectly.

Mongo is starting to look like it's out of place in the eco system.

~~~
metheus
> Having to resort to doing things like map/reduce for a simple group by /
> order is not the way to go IMO.

Perhaps you are unaware, but that is a straw-man. That’s not how you’re
supposed to with MongoDB. You’d use the aggregation framework:

db.sales.aggregate( [ { $group : { _id : null, totalPrice: { $sum: {
$multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity"
}, count: { $sum: 1 } } } ] )

[https://docs.mongodb.com/manual/reference/operator/aggregati...](https://docs.mongodb.com/manual/reference/operator/aggregation/group/)

[https://docs.mongodb.com/manual/reference/operator/aggregati...](https://docs.mongodb.com/manual/reference/operator/aggregation/sort/#pipe._S_sort)

This has been the case since MongoDB 2.0, in 2012.

~~~
spamizbad
Aggregates carry with them a memory limit.

We currently use MongoDB 3.4. It's definitely much improved. The replication
protocol that came along with 3.4 has been very reassuring. But aggregate
queries really do not give you the power of SQL. They are great for
transforming stuff for a report, but I would avoid using them for anything
else unless 1) it can be cached (and therefor properly invalidated) and 2)
doesn't need to be "real time"

Being a document, schema-less database, your "quality of life" as you scale
with MongoDB is going to be heavily dependent on how you structure your
documents and the types of their fields. Are you treating your collections
like SQL tables? Have any many-to-many relationships in hot code paths?
Welcome to hell. Its type system is also limited compared to modern SQL DBs.
Storing IP addresses, and want to query them based on a given CIDR range?
Postgres makes this eas, MongoDB has you writing code that does sub-queries.

~~~
metheus
>Aggregates carry with them a memory limit.

I _think_ you are referring to the 100MB RAM limit, but that’s not a hard
limit, it’s more of a bad default. The `allowDiskUse` option lets MongoDB
write intermediate results to the disk (which is exactly what SQL databases
are doing).

> But aggregate queries really do not give you the power of SQL. They are
> great for transforming stuff for a report, but I would avoid using them for
> anything else unless 1) it can be cached (and therefor properly invalidated)
> and 2) doesn't need to be "real time"

I really don’t see a difference between what you can do with MongoDB and SQL.
I can’t say much more without knowing specifically what impediments you have
in mind, but I would certainly like to hear more. For example, why do you cite
results caching and lack of real-time requirements?

> Being a document, schema-less database,

I guess if you’re on 3.4 you can’t take advantage of JSON Schema yet, but keep
that in mind as a part of your upgrade plans. In the meanwhile you can still
use document validation?

> your "quality of life" as you scale with MongoDB is going to be heavily
> dependent on how you structure your documents and the types of their fields.

This is completely true, but couldn’t we say that just as much about any
database? I’d put money on there being way more grief out there over bad
tabular schema than over bad document schema. I mean, who’s worked on large-
scale systems that hasn’t put off implementing great ideas, or had to hack up
app code to compensate for a restrictive schema, because you can’t take the
pain of ALTER TABLE?

> Are you treating your collections like SQL tables?

Ouch. Please don't!

> Have any many-to-many relationships in hot code paths? Welcome to hell.

That's probably fair, but if you're in hell to a _vastly_ greater than you
would be with Postgres, I'm pretty sure that's a modeling problem. Again, can
you tell me more about the particular example?

> Its type system is also limited compared to modern SQL DBs. Storing IP
> addresses, and want to query them based on a given CIDR range? Postgres
> makes this eas, MongoDB has you writing code that does sub-queries

That's 100% legit. MongoDB needs to do a ton better with that... types rule.

~~~
spamizbad
> I think you are referring to the 100MB RAM limit, but that’s not a hard
> limit, it’s more of a bad default. The `allowDiskUse` option lets MongoDB
> write intermediate results to the disk (which is exactly what SQL databases
> are doing).

While technically true, this isn't an apples-to-apples comparison. MongoDB's
memory limit, as best I can tell -- and please correct me if I'm wrong -- is
based on the contents of all documents and operations in the pipeline.
Documents in MongoDB tend to be larger, so you can run into that limit faster
than one might anticipate.

In Postgres, you have the equivalent of "work_mem". It defaults to 4MB, but
most production installation will bump this up. Regardless, this limit is _per
operation_ (a join, a sort, etc), not per query. And often times the operation
is against a specific fields, as opposed to the entirety of the record
contents.

> I really don’t see a difference between what you can do with MongoDB and
> SQL. I can’t say much more without knowing specifically what impediments you
> have in mind, but I would certainly like to hear more. For example, why do
> you cite results caching and lack of real-time requirements?

This might be my own hangups or just us fighting our own specific problems,
but I've never been happy with the latency I see come out of the aggregate
pipelines we've created. I also don't like what they do to the server's
memory.

------
egeozcan
> At its core, MongoDB is a document database and — almost by default — these
> kind of databases aren’t ACID compliant, especially when it comes to multi-
> document transactions. For the most part, that’s not a big deal for
> companies that use database systems like MongoDB because they are not trying
> to write to multiple documents at the same time.

No. At least in the open-source world, you can see many applications make
multi-document "transactions". I don't see how it would be different in
companies using MongoDB.

> Because of this, though, many MongoDB users still run relational databases
> in parallel with their document database.

No. For the most part, they do write to multiple documents and not think about
consistency.

At least the situation seems to be getting remedied. Better late than never.

------
ralmidani
Every time I hear about some NoSQL "breakthrough" that existed a while ago in
SQL databases, I can't help but feel underwhelmed.

In general, I'm convinced SQL is like Constitutional Democracy; it's not
perfect, but it's better than any alternative humans have come up with so far.

~~~
munk-a
The consistency is really nice too. I've always hated the fact that column
declarations are the first portion of a query and wanted them at the end
but... I'd rather be slightly disappointed all the time than occasionally need
to rewrite huge swathes of queries if we're changing DBMSs

------
mathieuuu
We have been using mongo over the last three years on our project and it has
been pretty smooth so far. However, lately I gave some thoughts into what our
project would look like if we used PostgreSQL instead. I tried to figure out
what problems Mongo solves that PostgreSQL doesn't.

I am far from being a database expert, I just know enough basics to query what
I need, so feel free to correct/complete the following:

\- Mongo has been built to store json objects -> Yes, but from what I
understand benchmarks indicate that PostgreSQL is faster at
reading/storing/indexing json/jsonb content. I don't think that it is good
reason to use it.

\- Mongo is schemaless -> There might be some usecases, but I bet in most
cases this problem can be worked around. Especially in a database with JSONB
support.

\- MongoDB horizontal scaling is way easier than PostgreSQL. Yes, it seems
that scaling horizontally Mongo is extremely easy compared to any other
relational database.

And ... that's it. But there is probably more.

At the moment, here's how I would summarize MongoDB benefits if asked my
opinion when starting a project:

\- For a small projects or a prototype: ease of use, ease of configuration,
don't require too much thinking into my data model while I am experimenting

\- For a bigger project: horizontal scaling should be easier

Does that sound accurate to you? Am I missing anything important?

~~~
Pxtl
Honestly, the existence of Mongo is mostly an indictment about how user-
hostile conventional RDBMS is. The fact that other DBs _can_ do what Mongo
does is not helpful when there is no easy workflow _to_ do what Mongo does.

The fact that I theoretically implement a web-based CMS in C and it could be
more performant than all these web-language CMS products doesn't mean that C
is better for making a CMS.

~~~
ZenoArrow
> "Honestly, the existence of Mongo is mostly an indictment about how user-
> hostile conventional RDBMS is."

That's nonsense. I've taught people to use SQL before, even people with little
to no programming experience. After the initial concepts were understood it
was fairly easy to gradually expand knowledge over time.

The basic SQL keywords to start getting useful information out of a RDBMS are:

SELECT

FROM

WHERE

INNER JOIN

LEFT JOIN

ON

AS

AND

OR

Those 9 keywords give you a good starting point for exploring SQL. It's really
not hard. You could probably learn enough to get started in a couple of hours,
and it's easy to expand your knowledge as and when you need to once you've got
the basics sorted.

~~~
Pxtl
That's after you set up users, schemas, tables, columns, oddly-named datatypes
with unexpected behaviors, etc.

The RDBMS is a lie. It's a beautiful kernel of relational theory wrapped in a
60-foot ball of hacks, tweaks, duct-tape, bubble-gum, and hate. The document
store doesn't lie. It doesn't pretend. It's honest that it's stupid and it's a
glorified hashtable with a string in it. It doesn't make any ridiculous
pretenses of having a Sufficiently Smart Query Optimizer that will inevitably
let you down and leave you pulling your hair out trying to figure out why on
earth a simple, straightforward query is running so goddamned slow.

Then you build a complicated model and have to figure out from the query plan
why your query is slow and deal with indices and foreign keys and all that
nonsense.

Meanwhile an object DB may be inefficent and clumsy, but it gets all of that
stuff out of the way. Also, if you don't want to join, you can work around
that by duplicating the data all over the place. Something you can't do with
an RDBMS because tables are fundamentally flat and so you can't stuff a
parent-child relationship into a single table.

~~~
takeda
> That's after you set up users, schemas, tables, columns

You have software engineer you similarly should have a data engineer (i.e. a
DBA). Nearly 50 years has passed and we still didn't find a better way to
represent data, so perhaps it is the right model. The only difficult part is
to bother enough to learn it.

> oddly-named datatypes with unexpected behaviors [...] 60-foot ball of hacks,
> tweaks, duct-tape, bubble-gum, and hate

That's only when using MySQL

> Meanwhile an object DB may be inefficent and clumsy, but it gets all of that
> stuff out of the way. Also, if you don't want to join, you can work around
> that by duplicating the data all over the place. Something you can't do with
> an RDBMS because tables are fundamentally flat and so you can't stuff a
> parent-child relationship into a single table.

You absolutely can store data inefficiently in RDBMS for example in Postgres
you can create a table with two columns one named key, and another data. The
data type for the second column can be JSONB. End you're essentially have
equivalent of Mongo's collection.

But in that case you store data inefficiently and if your application starts
evolving and you need to make different queries things will get more complex
quickly.

------
chasingthewind
I literally just finished making some code changes minutes ago to very
carefully sequence a set of changes to some related documents to make sure
that if write failures occur they'll have the least impact on our system. I've
been very happy with our decision to use MongoDB because in the vast majority
of cases I just don't need transactions, but there's that one place where
using them will be a big win.

------
krylon
I looked into MongoDB a couple of years back, because it was the hot thing at
the time. About fifteen minutes in, I try to find out how to do transactions.
That is strange, I thought, the manual says nothing about transactions. I
asked a popular search engine and was a little shocked to find out there were
no transactions.

That was a dealbreaker for me. If MongoDB has now grown support for
transactions, that changes things. I think I am going to look at it again
sometime.

EDIT: Typo

~~~
metheus
While doing so, I recommend you think some about the meaning of embedding
relationships vs. referencing them in other collections. MongoDB can do both
(the aggregation framework provides joins in the form of the $lookup stage).

If you have a “one-to-many(some)” kind of relationship, embedding is a good
option, and gets you ACID semantics without even resorting to multi-statement
transactions.

If you have a “one-to-many(thousands or more)” kind of relationship, objectID
references (a la foreign key) is more likely what you want.

~~~
krylon
It will be interesting to learn about database design in a non-relational
environment. I guess you can replicate a relational structure in MongoDB, but
then why not use an RDBMS in the first place?

As far as I understand it, MongoDB's claim to fame is a) handling huge amounts
of data and b) clustering. Neither of these apply to me, so the only reason to
use Mongo would be data that does not match the relational model well. I am
still looking for a use case, any use case, that might make a valid excuse to
learn it, but so far I have come up empty.

I am a little worried that I am facing a situation somewhat like the one when
I tried to learn Lisp. Learning Lisp was very hard for me because I had
absorbed the structured programming approach so deeply that the functional
part of Lisp programming seemed downright alien to me. At first, at least. So
for the time being, I cannot tell with any certainty if problems that are a
good match for a document store are just so rare, or if I am just incapable of
modeling my data in ways other than the relational model.

~~~
metheus
Think about it less in terms of relational/non-relational and more in terms of
tabular/document. Tables can _only_ model things in terms of relations.
Documents can model them either as embedding or as external references,
depending on access patterns.

A decent example is a person record with their email addresses and phones. In
a relational DB, you would always and only model those as three sepearate
tables, and you would quite frequently need a three way join to assemble that
person again.

In MongoDB, you would definitely have an array of emails and an array of phone
numbers embedded in that person document, sparing the join on those queries.
But in an ecommerce context, you would likely _not_ embed an array of all past
orders with all the line items into that person, instead you’d have an array
of previous order numbers.

But an order document would have an embedded array of line items, sparing the
DB a bunch more joins (and the attendant indices you’d need for joining line
items to orders efficiently).

Getting the 10 most recent orders from a customer would involve joining the
customers collection with the orders collection (MongoDB’s join is the
`$lookup` aggregation stage), but it wouldn’t involve joining the line items
to the order.

Is that replicating a relational structure? A little column A, a little column
B.

------
callumjones
I find it telling of MongoDB's sales strategy that this is being covered by
Techcrunch. I don't think you'd get the same coverage on Postgres (granted
MongoDB is a company, vs PG is a project).

------
majidazimi
Call me when they added Triggers, Stored Procedures, Common Table Expressions,
Window Functions,...

~~~
jasondc
Triggers?? Where you add some application logic to the database, 10 years go
by, and no one has any idea how the triggers work? Or even how to test them?
I've never seen triggers used successfully in any production application
(maybe they work at first, but give them time, and a few code changes).

~~~
ZenoArrow
> "I've never seen triggers used successfully in any production application"

Triggers have one or two good use cases, and plenty of bad use cases.

I would suggest the strongest use case is for data validation. Databases don't
have very sophisticated type systems, and custom data types can cause
headaches. Database triggers allow you to ensure that the data stored in
fields matches a set of criteria. To give a basic example, if you had a
customer table with an email address field, you could validate that the email
address had an @ symbol using a code in an insert and/or update trigger. Of
course, you should have similar checks within the software that sits on top of
a database, but by putting the validation at a low level by using database
triggers you can be more confident that the data integrity will be protected.

As a second use case, if the triggers are being used to maintain data for
audit or reporting purposes, that can be fine.

However, aside from the audit/reporting use case, I would recommend avoiding
using triggers which span multiple tables (unless there are some exceptional
circumstances where it's the best option). Things get messy when you have a
chain of tables, each with their own triggers that can update other tables
within that chain. If you spot lots of code like that, run to the hills!

~~~
majidazimi
We are using it for building aggregate tables from raw table. We insert data
into raw table. And triggers propagate running Min/Max/Avg for
hourly/daily/monthly tables. After couple of months we truncate our raw tables
but keep aggregate tables. Duplicates are easily removed based on insert. And
we get instantaneous result on our aggregate tables (No hourly batch job)

------
neovive
Has anyone worked with MySQL JSON data types in production? For most projects,
I prefer working in SQL via a query builder or ORM for abstraction, but find a
few features that would benefit from denormalized JSON storage.

~~~
furicane
I've been using MySQL 5.7 since its release in production. I used to store
JSON data in a blob, but when they released JSON support - I just couldn't
wait to use it. It's working really, really good. JSON support solves a ton of
problems we used to have, and we used EAV model to tackle those problems (I
won't get into details, the discussion will go the other way). I deal with a
few hundred MySQL deployments ranging from 1GB dataset size to several
terabytes, with multimaster and MySQL cluster setup to in-app handled
sharding. There are several notable pieces of software that never caused any
problems in our use scenario, and MySQL is one of them. The other is nginx.
Can't really remember the third. I often wondered why anyone would use MongoDB
or similar, but I always keep forgetting that developers are usually
inexperienced hipsters who are looking for magic unicorn to solve their lack
of knowledge / experience.

~~~
pizza234
> I often wondered why anyone would use MongoDB or similar, but I always keep
> forgetting that developers are usually inexperienced hipsters

You can do a basic google search, and you will find out use cases where
document-based data storages are advantageous compare to relational ones.

~~~
furicane
So what you're saying that if I find something on google, it's undoubtedly
true and I should blindly believe it? For every one of those use cases, I can
provide a counter-argument. And there are many more who can do the same.

~~~
scarface74
I used RDMSs for 15 years before ever using Mongo. I fell in love with Mongo
from the first time I used it with a pre-existing system. The beauty of just
being able to insert and retrieve my whole C# object graph with one statement
was a thing of beauty.

I'm a Domain Driven Design true believer. The thought of just being able to
store my entire domain model without the machinations of the object relational
impedance mismatches was glorious.

I use C# so my document collections are strongly typed C# objects and thanks
to the Mongo C# Linq provider, my queries are also strongly typed.

------
outside1234
"Ship and iterate" almost always wins over "perfect and ship"

------
jonjojr
That changes nothing for ArangoDB
([https://docs.arangodb.com/3.3/Manual/Transactions/](https://docs.arangodb.com/3.3/Manual/Transactions/))
as they have had this feature long before MongoDB. But MongoDB gets the
headlines for the hype and being light years late to the party.

------
unclebucknasty
This is funny. "NoSQL for the win! RDBMSs/SQL suck! Who needs ACID?"

Years later: "MongoDB has added ACID transactions."

"Awesome!"

------
khrm
"Every NoSQL attempts to add features until it can do relational ops and then
ends with a sql layer plugin on top. Those who can't are replaced by ones
which can." \- Rehash of Zawinski's Law by me

------
truth_seeker
Wow. In v3.6 they started supporting complex joins and in next major release
ACID is also coming. It would make it most flexible database ever.

I would love to see RocksDB as storage engine instead of WiredTiger in their
future releases.

~~~
nemo44x
You might be interested in this:

[http://mongorocks.org/](http://mongorocks.org/)

MongoDB has plug-able storage engines.

~~~
truth_seeker
I think Percona's MongoDB version also supports RocksDB as storage engine but
they are not keeping up with latest releases and hence new features offerings.
Correct me if my assumption is outdated as of today.

------
santidh
In general, the content that can be sent is defined as "anything that
satisfies one's intellectual curiosity.

------
scarface74
I keep hearing about how people have been burned by Mongo but nothing
specific.

Does anyone care to give any specifics?

~~~
riku_iki
I never used MognoDB, but read many stories that few years ago MognoDB default
configuration permitted data loss: server returned write success response to
client without syncing data to replica or disk, and if your server died then
your data gone.

Additionally some time ago MongoDB had single-threaded writes with collection
level lock, which could cause poor write performance.

~~~
scarface74
That's one thing I keep "hearing". The default configuration now is a
WriteConcern of 1 - meaning at least the primary has to have written it
successfully. You can choose 2 or majority among others.

~~~
takeda
That won't protect you, you can read the analysis for MongoDB from here:
[http://jepsen.io/analyses](http://jepsen.io/analyses)

The last analysis looks good, but you need to note that this is only true with
the strongest settings (this means that Mongo will be configured with slowest
settings) also this analysis did not include node crashes or restarts.

~~~
scarface74
I read it. From his analysis it seems like the loss of data will rarely happen
in the real world if you have a WriteConcern of Majority and you don't have a
large latency between nodes.

One of the comments said that theoretically, that could also happen with any
system if something happened between the time that it wrote to disk and the
time it acknowledged it, you may have extra data.

There are times though that you care more about speed than reliability - ie
capturing string data for an IOT device, logging etc.

There are even times that eventual consistentency is good enough. But
definitely choose the right tool for the job. I wouldn't trust Mongo for
financial data where transactions are a must.

And in my preferred language - C#, if you write your code correctly, you can
switch out your Linq provider from Entity Framework to the Mongo driver for
instance without any major code rewrite so you aren't stuck with your choice.

------
ChicagoDave
This is interesting, especially for micro-service architectures (one table,
one service) that need to modify multiple tables.

~~~
BillinghamJ
No, it doesn’t work for that use-case. It would all need to happen within a
single service in a single invocation.

To enable transactions across network boundaries, you need to enable the
transactional safety at the application/API level.

And, for the record, one service per table is a terrible idea. It would have
none of the benefits of a service-oriented architecture, while adding all the
complexity and caveats of a distributed architecture.

~~~
ChicagoDave
Except I've implemented it that way in highly visible, high traffic internal
web applications that were lauded as some of the most successful transitions
in company history in a very large corporation. So your argument is really
legacy opinion and not relevant to micro-service architectures.

~~~
BillinghamJ
None of the things you've just said make any difference to the fact that it
doesn't make sense and is a terrible idea.

I fail to see in what way my statements are "legacy" or not relevant to micro-
services. My company (Cuvva, insurance in the UK) runs a "micro" service
architecture and deals with these problems every day.

It is not possible to use DB-level locking/transactions across network
boundaries. It's just as simple as that. It's not an opinion - it's a fact.

~~~
ChicagoDave
I wasn't responding to the distributed transaction argument, though nothing in
the MongoDB blog says it won't support it.

I was specifically responding to "one table one service" which is the
foundation of domain driven micro service architecture. To have a single
domain silo'd and tested within its own boundaries.

Whether you're firing events between services or just calling other services
is a design decision, but the encapsulation of a specific domain is the point
and the primary benefit.

I have no idea what you're doing, but this is what I worked on:

[https://www.accenture.com/us-en/success-performance-
manageme...](https://www.accenture.com/us-en/success-performance-management-
achievement)

Which proves that silo'd micro-services are cost-effective and produce real
world results in a complex environment.

Feel free to knock more points off my karma. :)

~~~
Fins
Just because something _can_ be done, including the fashionable microservices
"architecture", it does not mean that it should be done, or that it is a good
idea.

~~~
ChicagoDave
Application architecture is inherently malleable, yes.

But when properly implemented, micro service architecture is actually a great
idea.

I can’t help if your experience has not been the same as mine.

~~~
Fins
Certainly, one can architect anything, use NoSQL for critical data storage, or
use Kafka for synchronizing two SQL databases with the same schema and living
on the same netowrk(an actual proposal from a fan of microservices, btw). But
whether it should be done is another question, and I have yet to see a single
example of microservices being a good idea. Not "it works" \-- obviously, you
can get it to work, but being better than alternatives. Come to think of it,
Mongo is the same...

~~~
ChicagoDave
The Accenture link I posted is a living example of a highly successful micro-
services architecture in a critical internal application in a large
corporation.

What more do you want?

------
Bitcoin_McPonzi
What I like most about MongoDB is that it's "web scale"

[http://www.mongodb-is-web-scale.com/](http://www.mongodb-is-web-scale.com/)

It's because of shards. Shards are the secret ingredient in the web scale
sauce. They just work.

~~~
NeoBasilisk
I will never stop loving this. Here is a youtube link in case anyone
can't/won't run flash on that site.

[https://www.youtube.com/watch?v=b2F-DItXtZs](https://www.youtube.com/watch?v=b2F-DItXtZs)

