
The MongoDB hack and the importance of secure defaults - tkadlec
https://snyk.io/blog/mongodb-hack-and-secure-defaults/
======
mike-cardwell
When our sysadmin set up our Mongo cluster, he firewalled out all IPs except
our production systems, turned on authentication, set things up to ensure we
used SSL, and configured backups.

He didn't do this because he's an amazing sysadmin. He did it because he's
competent, knows how to put a service on the Internet, and RTFM.

I mean. If you can connect to something and use it without having to
authenticate yourself, wouldn't it naturally cross your mind to check that
others can't do the same? It's just common sense.

~~~
peterwwillis
The article's argument is that users have no common sense and don't think
about what they're doing, so we should reinforce this same behavior.

~~~
acobster
> _you may decide that not binding a database to local connections would be a
> reasonable default...But assumptions like that are dangerous because when
> not if someone does something that breaks that assumption, they’re now
> vulnerable to attacks—and they may not even know it. Understanding of the
> potential security risks presented by these defaults is not exactly
> mainstream knowledge._

I wouldn't call this reinforcing behavior...I'd call it protecting your users.
Common sense is something that comes with experience, and MongoDB is popular
enough that it's likely to be in many web devs' first projects in production.
Is it their fault they haven't developed this "common sense"? Yes and no. On
the other hand, is vulnerable-by-default _ever_ a good idea?

------
51Cards
I have never used MongoDB so I admit I'm talking blind here, but can someone
explain how/why a piece of highly popular software gets to version 2.6
allowing unsecured remote connections by default? Further to that is that type
of thinking you want in the development process of something as critical as a
database engine? It just seems amazing to me that it got so far before the
community in general pushed back that this was really bad design? Hopefully
someone with MongoDB history can explain if this was a long term sticking
point, etc. Thanks.

~~~
richardwhiuk
In production deployments, unsecured remote connections might be fine, because
security would be handled at the network layer (e.g. by deploying MongoDB on a
VLAN which only was accessible by your MongoDB clients which were on separate
servers).

~~~
raverbashing
> In production deployments, unsecured remote connections might be fine

No. Absolutely not

Unsecured local connections yes, anything from outside 127.0.0.1 should be
explicitly allowed and secured (like port 80/443). Rate-limited, fail2banned,
fuzzed during testing, etc

~~~
Pxtl
"I opened the ports on the firewall and gave the user access, why doesn't it
work!? This system is stupid."

"I changed my password on one system and then on the other, but somehow my
account got locked up in that brief time between changeover, how the hell do
you unlock an account? This system is stupid."

"We're scaling up and we're running into outdated rate-limiting settings on
every service we use, some of them even having multiple layers of rate-
limiting within a single service, and each setting requires hours of debugging
to figure out which setting is taking down the whole project. This system is
stupid."

There's a reason Mongo got adoption while "secure" systems did not.

~~~
raverbashing
To be honest, the issue is that a lot of secure systems make things more
convoluted, unintuitive and/or frustrating than it needs to be, then calls the
user stupid when they can't set it up

------
Pxtl
Realistically, insecure defaults are part of the reason Mongo was adopted in
the first place. Other databases are a hellscape of configuration, meanwhile
Mongo is starting to get work done from the moment it's installed.

Security and usability are always at odds. If you make security a pain, people
will give up on security. Mongo is the ultimate manifestation of that
principle. Developers were _driven_ to Mongo by opaque and painful admin
tasks.

~~~
scott_karana
A "hellscape of configuration"?

`apt-get install mysql-server` makes you enter a "root" password out of the
box.

`apt-get install postgresql` create a local user `postgres` with access the
local database.

Both bind to only localhost by default.

Those sound like _easy, reasonable_ defaults to me. (And this isn't exclusive
to Debian-likes: RHEL almost certainly shares the exact same benefits)

~~~
lacampbell
_`apt-get install postgresql` create a local user `postgres` with access the
local database._

Does it!? I'm sure I've had to do that myself.

~~~
fulafel
Yes.

------
mikestew
Microsoft took a rash of shit some time ago (15 years?) for shipping MS Proxy
Server with every port open by default. From the POV of employee-at-the-time,
it took them a disappointingly long time for them to not do that anymore.

Since then, I've learned to not assume that products are secure-by-default. At
the same time, I kind of thought we learned our lesson and cut that shit out
low these many years later. Add a line to a text config file that's probably
buried eight directories down in a hierarchy that's owned by _root_? (I'm just
hyperbolically guessing for effect; I generally avoid Mongo.) Do it, or you're
hacked? And it's been this way for years? Come _on_.

~~~
debacle
Microsoft is a different beast. They ship things based on the principal of
least surprise. There are incredible features in newer versions of SQL Server
that aren't turned on by default when upgrading, even though no one would want
to not have them on.

~~~
orf
Can you give some examples?

~~~
debacle
Read isolation is probably the biggest stand-out in terms of "Wow this is way
faster why would anyone* not use this?"

* Almost anyone, there are definitely potential pitfalls.

------
andreyf
No matter what your defaults are, if a sysadmin doesn't understand that some
software is OK to connect to a public-internet addressable interface while
other software is not, their shit is going to get hacked (or, as in this case,
accessed).

This is the equivalent of leaving your data in a file cabinet in your lobby
and being surprised someone took it. The answer isn't to add self-locking
little office locks to the file cabinets by default, it's to educate folks
that this is a file cabinet that's designed to be inside the office, not in a
publicly accessible place.

~~~
jldugger
Network oriented software created after 2000 has zero excuse for allowing
unauthenticated writes from the general internet by default.

~~~
stouset
Tell that to redis.

No, seriously. Please. I've been banging that drum for years.

~~~
andreyf
Also, MySQL [1]. Are you sure that you're right and all these major db
projects are wrong?

1\. note the default value of "bind to *":
[http://dev.mysql.com/doc/refman/5.7/en/server-
options.html#o...](http://dev.mysql.com/doc/refman/5.7/en/server-
options.html#option_mysqld_bind-address)

~~~
jldugger
Well, mysql was started before 2000, so it gets a pass. But I'm also pretty
sure mysql doesn't allow unauthenticated writes, so it doesn't need one?

------
achillean
The most common MongoDB versions that are being exploited with the ransomware
come with secure defaults!

[https://www.shodan.io/report/kjVzjr3O](https://www.shodan.io/report/kjVzjr3O)

Originally, I thought the problem would disappear once MongoDB changed the
default from "0.0.0.0" to "localhost". However, that has had no impact on the
number of exposed MongoDB instances on the Internet. Bad defaults are
definitely a problem but in this case that isn't what's happened.

Also note that the most popular location where these instances are hosted is
on AWS where you explicitly have to open up the firewall.

~~~
exogen
Just a guess, but those people probably updated their installation with an
existing (insecure-by-default) configuration in place. Very little software
like this will mess with an existing configuration, so that upgrading doesn't
appear to break anything. That's how they'd be on a secure-by-default version
but still be insecure.

~~~
achillean
Yeah, I could definitely see that happening though the overall number of
exposed MongoDBs has also increased. And there are Docker images that come w/
insecure defaults ala:

[https://hub.docker.com/r/swcc/docker-
mongodb/~/dockerfile/](https://hub.docker.com/r/swcc/docker-
mongodb/~/dockerfile/)

One of my concerns is that people are piling onto MongoDB because "it's web
scale" and ignoring potentially systemic security issues that aren't specific
to MongoDB. The same issue exists for Riak, Cassandra, Memcache etc.:

[https://blog.shodan.io/memory-as-a-service/](https://blog.shodan.io/memory-
as-a-service/)

------
koolba
While we should hope that server software is secure by default, you shouldn't
rely on it either. A good rule of thumb is: _Distrust until verified_

If you're running any servers yourself, there's no reason to open up network
access to the world. At most the specific ports you want to expose, to the
specific IP ranges you want to allow, should be white listed. A default of
everything to everywhere is insane.

Even with SSH key based authentication (v.s. say passwords) I'd consider it
inept to expose all your infrastructure publicly. Set up a VPC or whatever the
local equivalent is for your hosting environment and proxy SSH access via a
bastion host. If you whitelist the inbound addresses to the bastion host
(let's say to restrict it to just your office) then you've also eliminated the
vast majority of auth log spam too.

Unfortunately the people that would understand and implement things like this
are the same set of people that wouldn't have publicly exposed MongoDB
instances in the first place.

------
discreditable
I find it interesting that there's no firewall with a default deny rule
between these exposed mongodb installs and the Internet. All I can think is
that most of them are on cloud services which are directly exposed. It reminds
me of the fiasco with all of the directly-connected vulnerable network cameras
on the Internet.

~~~
00deadbeef
I think the problem is developers running apt-get install mongodb and assuming
all other considerations, like a firewall, are somehow magically taken care
of, then patting themselves on the back for not needing a sysadmin.

~~~
pavel_lishin
I think it's more of a case of assuming that the database wouldn't expose
itself to the internet at large, because why would it? It's like buying a new
car, and checking for holes in the gas tank before rolling off the lot.

~~~
00deadbeef
Your car analogy fails because safety is highly regulated by the government,
and the manufacturer can be sued for their failings.

When it comes to server security, you are responsible, and should never make
assumptions. If you don't know how, or simply can't be bothered to check, then
you shouldn't be administering a server.

Also a firewall is a most basic requirement, even something simple like UFW
would do.

------
orblivion
I don't mean to defend Mongo here, but as a counter argument to the broad
principle: Ubuntu doesn't ship with iptables blocking all incoming connections
by default. Should it?

~~~
kalleboo
Honestly, in this day and age, it probably should. As well as whitelisting
software that is allowed to make outgoing connections. At least for server
installations. Network security is still a complete joke.

------
jbverschoor
MongoDB has a history of using the insane defaults - not secure, and willing
to loose your data.

------
nailer
Relevant laptop sticker:

[https://twitter.com/ag_dubs/status/603273801783234560](https://twitter.com/ag_dubs/status/603273801783234560)

Source code to print your own:

[https://github.com/mikemaccana/stickers](https://github.com/mikemaccana/stickers)

------
sfilargi
I wouldn't blame MongoDB.

Any machine connected to the Internet should start with a configuration
equivalent to:

    
    
        ufw default deny incoming
        ufw allow ssh
        ufw enable
    

And then start punching very specific holes as needed.

------
gator-io
I had a situation where their cloud backup product was not backing up data and
not issuing alerts when it wasn't working.

I gave them full access to logs, etc. and the support response was basically
that they couldn't determine why it wasn't working or alerting, but it it
works for them, so I should migrate to their hosted solution.

I sometimes think of the potential company-ending cluster-fudge that could
have caused, while I stare wide-eyed at the ceiling at night.

------
Pxtl
To play devil's advocate for a second, isn't Mongo following the unix
philosophy here? Small, composable tools instead of bolting on redundant
features to everything?

Opening/closing ports is the firewall's job. Mongo's job is storing and
retrieving data (now, how poorly it does at _that_ is another conversation).

Having both Mongo and the Firewall handle ports means you do everything twice,
which violates OnceAndOnlyOnce.

~~~
scott_karana
If that's the case, it should be behind something like inetd, instead of being
able to daemonize itself ;)

------
jstoja
I stumbled upon this (awesome) article that made me realize tons of things:
[http://www.ranum.com/security/computer_security/editorials/d...](http://www.ranum.com/security/computer_security/editorials/dumb/index.html)

I'm not a security expert (far from it) but I hope that I understand enough
the importance of security to learn a bit about it and implement it as much as
I can.

Secure defaults is now maybe the first concept I'm trying to explain to people
in my company.

------
peterwwillis
Defaults are stupid and dangerous.

First off, there's the constant problem of default passwords. The big IoT
compromises lately were due to default passwords, and ever since they became
popular they've been a bane to security everywhere. Wifi APs used to use
defaults, but luckily most come with a randomly assigned password now (except
those provided by an ISP, occasionally). All software that matters should not
use default passwords.

Second, a "default" is not intended to be an operational setting. When
developing software, a default is literally a placeholder setting so that your
software won't immediately crash if a configuration value is missing. Some
software is effectively meaningless without a configuration, but if they are
obscenely permissive they can allow a user to still take advantage of it,
which is how MongoDB is set up.

Finally, there is effectively no such thing as a secure default, because
something isn't really secure until it's been vetted through a process.
Pretending your software is secure when you have literally not given it a
second thought is just setting you up for eventual disaster. People should be
actively thinking about their security if they care about it; defaults just
leave them blissfully ignorant.

In my ideal world we would require software to no longer have defaults. Demand
that users be aware of the configuration and running of their software so they
understand what's involved with how it runs. This would teach them more about
the software they're using, which would enable them to take better advantage
of it, as well as make it more secure. If you don't have time to do that you
probably shouldn't be using it.

------
BillFinchDba
It's pretty simple folks, RTFM. If you are running MonogoDB on your laptop to
build an MVP, sure you can run it unsecured no worries. When you go to
production, you go secure. My hope is that no DBA worth his under-appreciated
skills would drop a totally open DB on the public internet. That said, it
obviously happens... Read the directions, have a plan, and follow the
checklist: [https://docs.mongodb.com/manual/administration/security-
chec...](https://docs.mongodb.com/manual/administration/security-
checklist/?jmp=community-hub)

Everyone has a different security model. In my case all my DB servers live
behind and API layer on the internal network, and the DMZ web layer talks to
the API. That makes keeping things secure MUCH easier...

------
dhatch387
The v2.6 open access issue in MongoDB is no new revelation, and has been
rehashed over and over again. It's worth noting that this post comes from a
company that profits from convincing its customers that the products they are
using are insecure.

------
1_2__3
I'll say this is even worse because whether or not to bind to localhost, all
interfaces, or something in between is often highly dependent on how the
software works - which you're less likely to know when just trying it for the
first time. Even someone security conscious would have to pause and do some
investigations to find out what mongo expected in terms of port and interface
bindings. If the default was to bind to 0.0.0.0 my first assumption would be
they require that.

------
jlaustill
This was 100% preventable, MongoDB comes with great resources for securing
your data. I pity whomever puts ANY database on the internet without taking
proper security measures.

Here's some documentation to get you started.

[https://www.mongodb.com/blog/post/how-to-avoid-a-
malicious-a...](https://www.mongodb.com/blog/post/how-to-avoid-a-malicious-
attack-that-ransoms-your-data?jmp=community-hub)

[https://docs.mongodb.com/manual/administration/security-
chec...](https://docs.mongodb.com/manual/administration/security-
checklist/?jmp=community-hub)

[https://www.mongodb.com/collateral/mongodb-security-
architec...](https://www.mongodb.com/collateral/mongodb-security-
architecture?jmp=community-hub)

This really should have been titled "the importance of good DBA's..."

------
balgan
This isnt a mongodb problem. redis has already been targeted and memcached and
elastic will. e next. for more info: ise.binaryedge.io or
[http://blog.binaryedge.io/2015/08/10/data-technologies-
and-s...](http://blog.binaryedge.io/2015/08/10/data-technologies-and-security-
part-1/)

------
virtuabhi
I just had to deal with an insecure default in our Hadoop cluster.

Hadoop, by default, enables WebHDFS, which provides read+write access to HDFS
over HTTP. The service runs on name node using the same port which displays
the Hadoop Jobs Web UI.

Now, it is very easy to not know anything about WebHDFS and every day
authenticate/login to server, submit jobs, and track the status on WebUI
(which is public). But then anyone can send a POST request pretending to be
any user and Hadoop will happily execute the command (including deleting all
files).

Maybe Hadoop developers believe that all Hadoop installations are behind a
firewall (ours was, but an exception was made for Jobs Web UI as it seemed to
be a read-only status page for running jobs) and/or have Kerberos integrated
with Hadoop for authentication.

But that being said, enabling HDFS access over HTTP by default seems to a
debatable choice. Also, if you have a small Hadoop cluster running, do set
"dfs.webhdfs.enabled" property to False in hdfs-site.xml.

------
UlrichC
I cannot believe, that so many users install MongoDB, only using default
settings and installed on servers, which are accessible via the internet.

That is fundamentally wrong application design. The application connects to
the database, so the database must not be public accessible via internet.

I mean, if you push something into a production environment, you must think a
bit about the tools you are using and how they are configured for security.

On a development system, I don't want to setup users or auth-mechanisms.

There are so many resources from MongoDB, how you can secure your MongoDB
database: \- Security Architecture White Paper:
[https://www.mongodb.com/collateral/mongodb-security-
architec...](https://www.mongodb.com/collateral/mongodb-security-architecture)
\- Security Best Practices blog post: [https://www.mongodb.com/blog/post/how-
to-avoid-a-malicious-a...](https://www.mongodb.com/blog/post/how-to-avoid-a-
malicious-attack-that-ransoms-your-data)

There is even a checklist which you can work through:
[https://docs.mongodb.com/manual/administration/security-
chec...](https://docs.mongodb.com/manual/administration/security-checklist)

And for those, who don't want to read anything ;-), there is a comprehensive
Video-Course on the MongoDB-university:
[https://university.mongodb.com/courses/M310/about](https://university.mongodb.com/courses/M310/about)

If I enter a car and don't use the seat-belt or switch on the light at night
(defaults are off), then I cannot blame the car-manufacturer. If I use
something important (here the database in a production environment), then I
have to learn the ropes and read the material specifically on security topics.

------
mkagenius
While we are at this, this checklist is worth looking at:
[https://github.com/FallibleInc/security-guide-for-
developers...](https://github.com/FallibleInc/security-guide-for-
developers/blob/master/security-checklist.md)

------
hashkb
Postgres' defaults barely let you connect to the DB. You don't hear stories
like this about Postgres.

~~~
cpuguy83
Well, sure but then people just open the flood gates because it's such a pain
in the ass to change postgres access policies.

I guess secure by default means there aren't likely to be massive hacks like
this, but it does not mean people are running securely.

~~~
Kudos
Is it just that no one's running exploit scanners for Postgres then?

------
ns8sl
Speaking of defaults, if you install Mongo on EC2 using a default instance of
Ubuntu using the default engine, Wired Tiger, you will get ceaseless server
crashes. Wired Tiger is NOT STABLE on the default volume - ext4. You have to
use xfs.

This was only recently added as a startup check.

------
nuha
Education, experience, diligence. I'd expect that of anyone basing their
livelihood on delivering production systems.

Secure out of the box is nice, except that when the boxing is done by others,
and delivered to under-informed, unexperienced people. It's too easy to npm
yourself a "full stack" or just pick a Docker container with full stack on it,
then place it on public inter-web... there are dangers to be had, and others
can save you. It's up to you.

And yeah, the click-bait titles including the word "hack" are laughable.

------
gitignore0
I have recently been working with mongodb wanted to start the service with
auth enabled with defined users and security. Here is the way I did.
[https://medium.com/gitignore/automatically-start-mongodb-
in-...](https://medium.com/gitignore/automatically-start-mongodb-in-
authentication-mode-using-predefined-admin-users-48dca5d1f075)

------
satokano
I found another piece of useful information from mlab, so I will share it.

[http://blog.mlab.com/2016/07/mongodb-tips-tricks-
collection-...](http://blog.mlab.com/2016/07/mongodb-tips-tricks-collection-
level-access-control/)

------
nirajadhikary
The news is too late for me. Two of our development servers are already
compromised. Anyway I added those servers under NAT so those servers are
inaccessible from internet. Added admin access credentials also. The news
should have been spread without any delay.

------
guypod
It's worth noting this isn't unique to MongoDB. The "Marked" npm package, with
it's 2 million downloads, doesn't sanitize input by default. "st", another
popular package, allows directory listing by default. Quite a few of those...

------
bjt2n3904
If I were a mongo dev, I wouldn't want to write authentication and have to
manage crypto.

Default bind to localhost. If you want to handle external connections, the
network layer provides your security.

------
unquietcode
The MongoDB hack and the importance of hiring competent people.

------
ggregoire
A bit off-topic but related: how secure are the defaults of an AWS RDS MySQL
(created by a BeanStalk environment, if it matters)?

------
mark_l_watson
Not defaulting to 'localhost' (or 127.0.0.1) is pretty bad in general.

------
fdim
Why would someone still use v2.6? There isn't much trouble to upgrade.

~~~
mtmail
Based on the article at least 30.000 internet-accessible installations.

------
tormeh
Who, when wanting to write a secure piece of software, thinks: "I know! I'll
use JavaScript!"? Security was always going to be an afterthought at best.
There are legitimate reasons for writing things in JavaScript, and none of
them apply to a database.

~~~
golergka
What does this have to do with Javascript?

~~~
tormeh
MongoDB is written in Javascript.

~~~
grzm
In part. According to the Github repo, it's 75% C++, 18% JS.

[https://github.com/mongodb/mongo](https://github.com/mongodb/mongo)

~~~
aioprisan
The JS is from docs and examples, not from core MongoDB code.

~~~
tormeh
So
[https://en.wikipedia.org/wiki/MongoDB](https://en.wikipedia.org/wiki/MongoDB)
is wrong? Well, guess I trusted Wikipedia a bit too much.

~~~
aioprisan
I'm confused, what about that page are you claiming to be right or wrong? The
code is here for all to see:
[https://github.com/mongodb/mongo/tree/master/src/mongo](https://github.com/mongodb/mongo/tree/master/src/mongo)

