
Major Remote SSH Security Issue in CoreOS Linux Alpha - jonaslejon
https://coreos.com/blog/alpha-security-incident-subset-of-users-affected.html
======
jo909
For those who are interested in the actual technical error:
[https://github.com/coreos/coreos-
overlay/pull/1965](https://github.com/coreos/coreos-overlay/pull/1965)

Very easy to make such a mistake, and it never made it into a stable release.

~~~
zymhan
> it never made it into a stable release.

True, but...

> Based on log data from the CoreOS Linux Update Service roughly 3% of online,
> auto-upgrading, hosts were affected.

If hosts are configured to autoupdate to a release, you're treading a fine
line if that release could have issues like wide-open SSH access. Though I'd
hope that no one is hosting sensitive data on an OS in alpha, that compromise
may be used as a jumping-off point for a more serious attack.

~~~
jo909
I guess having a lot of machines using the newest alpha releases is a double
edge sword. It is great to have many users that are able to report bugs early,
but then again a security issue like this is actually a somewhat serious
issue.

I really hope people that use anything of alpha status know what they are in
for. Maybe a bug like this every now or then is required to remind people of
the inevitable consequences of doing so.

According to
[https://twitter.com/mjg59/status/732221095051296770](https://twitter.com/mjg59/status/732221095051296770)
it affected about 1400 machines.

------
0x0
Reminds me about the 2002 bug in Debian Unstable where something in PAM
incorrectly allowed password-less ssh logins for all the system accounts,
because "*" in /etc/passwd was parsed as "empty" instead of "no valid
password", so you could ssh in as "nobody" and be greeted with a shell :)

[https://www.debian.org/security/2002/dsa-177](https://www.debian.org/security/2002/dsa-177)

------
siliconc0w
We use CoreOS but I'm still not sold on automatic updates. We've seen too many
issues running disparate versions of etcd or docker. To their credit - it
seems to be getting better but still not prime time.

Plus, we really want to push for an 'immutable' production model which is
counter to automatic updates.

~~~
digi_owl
And then we run into the xscreensaver issue, where upstream gets fed up about
redundant bug reports.

------
therealmarv
I still wonder (and this one of many reason I'm not using CoreOS, because I
don't now how to secure it as good as Ubuntu/Debian) how do you install a
simple Fail2Ban (ssh) implementation on CoreOS? Are there any practices how to
make the server which runs all containers more secure? Or do I simply rely on
CoreOS team for all that ssh securing and firewall stuff (it seams it is this
way now!) ? I have the feeling that hardening a CoreOS server is not the main
priority for CoreOS.

~~~
lima
There's no point in using fail2ban if you use public key authentication. It's
a kludge which helps against brute force attacks, which are limited to
password login.

Since CoreOS enforces public key authentication, and the SSH server is the
only external attack surface, they're doing just fine. Security is
_definitely_ a top priority for CoreOS, and they're doing a great job at it -
image signing, Docker container signing, ASLR binaries, a clean build system
(based on Gentoo)...

~~~
vidarh
I tend to agree. One reason to use something like it, though, is that it clogs
up logs. But I'd tend to want to tie it down with iptables and limited IP
ranges and/or just move ssh to another port (the latter as a quick and dirty
way to avoid the steady annoying log nonsense if you for whatever reason can't
limit ip ranges, not because I have any illusions it does much for security)

~~~
ultramancool
When filtering your logs, filter for only successful attempts. You really
don't care how many people failed to login on a server with only pubkey
enabled. It's the ones who succeeded you want to know about.

~~~
vidarh
Exactly, but it's nice to be able to follow your logs without all that
clutter. Though with journald I mostly look at logs on a per-unit basis anyway
(or via e.g. an ELK setup), so it's a slight nuisance rather than something
essential.

------
schmichael
99% of systems patched in under 12 hours. I know CoreOS doesn't have a high
number of instances in existence, but that's still an insanely fast
turnaround.

------
jjawssd
"Major bug found in alpha release" ... SURPRISE!

------
otterley
In other news, it looks like CoreOS is finally adding PAM support, which means
we might be able to use it in an LDAP-enabled environment soon. That's great
news!

~~~
dozzie
You realize that you didn't need to wait for PAM? That you could read NSS
`shadow' table from LDAP in the same way as `passwd' is?

~~~
otterley
On all the Linux distributions I've used in the last 15 years, NSS has never
been responsible for authentication, though. It's only been responsible for
_resolution actions_ like getpw* and getgr* (i.e. uid/gid to name), whereas
PAM has been responsible for authentication and auditing. You have to
configure both to support LDAP, not just one.

------
robszumski
A post-mortem has been posted regarding this issue:
[https://coreos.com/blog/security-brief-coreos-linux-alpha-
re...](https://coreos.com/blog/security-brief-coreos-linux-alpha-remote-ssh-
issue.html)

------
dbalan
I get that coreos wants to be transparent, but isn't this the point of having
an alpha release?

------
carrja99
I feel like this being #1 on hacker news is someone's way of trying to say
it's damning evidence against CoreOS' security model and why you should steer
clear of it.

Of course they found issues in an alpha release. Happens all the time.

------
zxcvcxz
I quit using coreOS a while a go when I realized how unprofessional some of
the top devs were. I don't want to name names but one in particular seems to
spend all their time writing blog posts bashing any and all competition, no
matter how disingenuous they have to be to get their point across.

Most people seem to use coreOS for security. When the devs seem more concerned
about slinging shit at Canonical and Red Hat it just makes the CoreOS team
seem unprofessional and when it comes to security I demand professionals.

~~~
throwanem
That's a pretty bold allegation to make without support.

~~~
jldugger
Really, the dev in question has been a vocal critic Canonical and others for a
while. It's much of a reflection upon CoreOS beyond their decision to ignore
it when hiring the person.

