
Kubernetes clusters being hijacked to mine cryptocurrencies - igama
https://blog.binaryedge.io/2018/12/06/kubernetes-being-hijacked-worldwide/
======
tetha
Ugh. I mean, I recently got in an argument if anything but a hard firewall
could or should be exposed to a WAN interface on the internet and we kinda
agreed to not agree for now.

But, popular services, on default ports, with default APIs enabled, without
hard authentication on a WAN interface? That should be a paddling. That
doesn't fly. Or, well it does, except not for the guy paying the power.

~~~
kevin_nisbet
To be fair, kubernetes itself and most distributions are quite secure by
default. So with kubernetes it's not the same as it was NoSQL databases that
didn't have authentication that were bound to the internet.

I'm not familiar with enough distributions to know if there is a popular
distribution that totally disabled authentication by default, but in my
companies distribution, kubeadm clusters, and I suspect all managed clusters
(GKE/EKS/AKS/etc), the vector outlined in the article would only work if an
admin specifically disabled the authentication.

In gravity (my companies distribution), we even disable anonymous-auth, so
someone would have to do real work to allow API access to the internet.

~~~
ryacko
You will have to patch a critical vulnerability every year on production
systems, no matter what language or who develops it.

Secure defaults are irrevalent if you pay attention to the news.

~~~
TheDong
> You will have to patch a critical vulnerability every year on production
> systems, no matter what language or who develops it.

Interesting. I've got a few openbsd boxes that do not have vulnerabilities
that impact them nearly so often.

It turns out that if you practice defence in depth, the majority of security
vulnerabilities in the news have no impact on you.

For example, on my openbsd boxes I have only a single user. I do not run any
untrusted code. That means spectre/meltdown doesn't actually impact me because
no one can run code which will perform such a timing attack.

There was a recent openbsd/Xorg security issue. I didn't have X installed, and
even if I did since it's only a single-user server, it again wouldn't have
impacted me (privilege escalation means nothing when everyone is effectively
root in my threat model).

All vulnerabilities are not created equal, and with enough good practices it's
possible to have boxes that are secure for years and years with no need for
patches.

~~~
clarry
Yep. The bloated, overhyped and chruny mainstream crap needs updating all the
time (and it kinda tends to break too). Build on that, and hope the updates
get done before someone fires and exploit. That's defense in "pray and hope
we're faster."

I'm not so concerned about my OpenBSD box with >800 days of uptime, which runs
very limited and carefully selected services.

------
whalesalad
Got in a pretty heated debate with a colleague once about this. We had a
really great infrastructure setup with a VPN bastion host that would get you
into our VPC. You couldn't reach any of our kube nodes externally. Your Google
account was your VPN account. It was pretty solid.

When this engineer redid things they opted to go the public internet route
where the master runs a public api and auth is done via a certificate. The
logic here was so that external 3rd party stuff (CI) could control our master.

To my knowledge this setup is still running and chances are these machines are
vulnerable to this issue.

Contrast to the prior setup where, immediately upon being offboarded from the
company your VPN access became automatically terminated (thank you LDAP and
Foxpass!)

~~~
closeparen
Why is a web server's demand for a certificate different from a VPN server's
demand for a certificate?

~~~
hueving
Complexity and publicity.

Complexity: Single purpose apps built with a very specific threat model in
mind for a boring, established usecase tend to be more secure. K8s is a fast
evolving labyrinth of complexity with contributions from thousands of people,
very few of whom have a grasp on the whole codebase.

Publicity: the general Internet doesn't find your VPN server just by using
your API.

------
voltagex_
At least cryptocurrency has removed most of the creativity from script kiddies
- there's so many more interesting things you could do than just mine coins.

~~~
blazespin
Yeah, exactly. It’s almost like a bounty for find a vuln. It seems to be a
mostly harmless attack that doesn’t cause global internet grief like a DDoS or
something.

~~~
gammateam
WannaCry’s very public global ransom brought attention to a Monero mining
botnet which was using the same exploit for weeks beforehand, it was making
$40,000 per day. It made much more than WannaCry and its operators are still
unknown and would have been able to cash out

Script kiddies are just annoying and their actions resulted in the patch
killing that silent mining botnet as well.

------
nineteen999
This is one of the side-effects of products having enormous hype in this
industry.

Far too many people are adopting Docker/Kubernetes as they have been the hot
new product for the last couple of years, often regardless of whether they are
actually the best or most appropriate tool for the job.

A lot of the people who get sucked into the hype are often inexperienced
programmers, devops or admin types who are in positions of power or influence
in companies that they probably shouldn't be, IMHO.

As a result, they don't have the Linux or networking experience to be able to
know when they are deploying these complex products securely or not, and they
are putting their employers businesses at risk.

~~~
ram_rar
>A lot of the people who get sucked into the hype are often inexperienced
programmers, devops or admin types who are in positions of power or influence
in companies that they probably shouldn't be, IMHO.

I cannot agree more. Many times, I feel you da easily do away with ansible and
terraform to setup VMs / docker. you dont quite need k8s. Just cuz K8s are
cool.. people feel the need to use it.

~~~
ownagefool
It's more complicated that that. Whilst some people are probably jumping on
kubernetes for the hype, there's a lot of things it makes really easy,
especially for less experience teams.

For example:

\- You want to spin up ephemeral environments to test PRs end2end. Sure,
create a namespace, deploy your charts and run your tests. You want to do that
with ansible, sure you can, but it's harder.

\- You org is running apps via a multi-cloud and on-prem strategy? Okay, lets
just write lots of tooling per cloud and another for on-prem, or we could
abstract that away via kubernetes and only worry about tooling for kube
itself.

\- You want to do have rolling-upgrades. Sure, you build them with ansible
then, or you could just use kubes.

Further to that, kubernetes is guiding reasonble abstractions, seperating
infrastructure from code. Sure, it comes with complexity, but so does most
things when you start throwing in scaling and auto-recovery.

For example, deploy terraform from your laptop? The device you probably browse
porn on has becomes an attack surface. Move this to Jenkins, the CI is the
attack surface. Put your code on Bitbucket? Bitbucket and the Jenkinsfile
becomes the attack surface. Pretty much everything we do has complexity and
attack surface _problems_ and using a managed k8s service will allow you some
easy wins so you can actually think about those other problems, and those
solutions will work on all platforms you can run k8s on.

~~~
nineteen999
Can I ask because I'm genuinely interested - what on earth do you do for
third-party applications (for eg. closed source) that have to be integrated
into your environment that don't come pre-packaged in a convenient container?

Do you containerize these yourselves, whether or not the vendor says that will
support that? Or does it get pushed to some other team that manages whole
VM's/AWS instances that are not container hosts.

Or is this a scenario that just doesn't happen in your environment?

Genuinely curious.

Also:

> using a managed k8s service will allow you some easy wins so you can
> actually think about those other problems, and those solutions will work on
> all platforms you can run k8s on

None of which matters one jot, if one cannot properly manage ingress/egress
filtering on one's API endpoints, or a reasonable level of password/credential
security. One will be used for cryptomining or worse, as per the fine article.

In that instance, one needs to go back and get some basic UNIX/Linux/network
and security training before one starts playing with complicated software on
publicly connected clouds. Or hire some people who actually know what they are
doing with respect to that.

~~~
ownagefool
> Can I ask because I'm genuinely interested - what on earth do you do for
> third-party applications (for eg. closed source) that have to be integrated
> into your environment that don't come pre-packaged in a convenient
> container?

Depends what it is. I've taken a number of apps and wrapped them into docker
containers and then written a helm chart. Some orgs get a bit skittish over
"vendor support" but this usually only matters when they think it's a key
product.

The point is, once you have a fleet, you should manage everything the same. If
you're off building other pet services, you're going to have capacity
problems.

> None of which matters one jot, if one cannot properly manage ingress/egress
> filtering on one's API endpoints, or a reasonable level of
> password/credential security. One will be used for cryptomining or worse, as
> per the fine article.

I mean sure, but I did say use a managed service, which will come with auth.
Similarly I wouldn't recommend you host services on any cloud or network
facing the public, without a professional involved.

For example AWS is easy to get wrong all the same. One of my current client is
busy hiring developers with no experience to put services on AWS, and they
came up with no encryption, no auth, no monitoring, misconfigured IAM. What's
really the difference between that and kube?

------
igama
CTO Binaryedge here. For those wondering, We have detected more than 15k
Kubernetes APIs with Auth. This post focuses on ~1.5k found without Auth, that
are fully open.

It's not just a Kubernetes Problem. Like many have posted, many databases,
other types of clusters, shares, are accessible without Auth for those that
know how to look for them (not that hard now days), mainly malicious actors.

------
WrtCdEvrydy
JSON file is still available
([http://192.99.142.232:8220/222.json](http://192.99.142.232:8220/222.json))

~~~
gammateam
> "algo": "cryptonight",

Nice, Monero mining

Cryptocurrencies makes the bug bounty market A LOT more efficient than
companies, legislation or HackerOne ever could.

------
unstatusthequo
Heading continued: “... thieves make off with $4.50”

------
clubm8
Is anyone else a little tired of "X used to mine crypto" stories?

Yes - if it has a CPU and access to the public internet, someone will hack it
and make it mine "cypto". Let's stop pretending we aren't aware that the
internet of things exists and writing breathless stories every time a toaster,
router, or adult toy starts churning out Monero.

~~~
roguecoder
This is an important vulnerability in widely-used software. Crypto is relevant
because the inherent design of crypto makes hacks like this more profitable,
but it's not the main thing about the article.

~~~
igama
Exactly, the main story is Kubernetes being exploited in the wild and in large
numbers, Crypto mining is just one of the "attacks" tacking place.

------
conanthe
Is kubernetes a mongodb of orchestrators?

------
gipmon
These guys are amazing. They have a lot of data and an excellent app with a
lot of potential!

