
HackedThat: Breaking in to a hardened server via the back door - bradleybuda
http://polynome.co/infosec/inversoft/elasticsearch/linode/penetration-testing/2016/08/16/hack-that-inversoft.html
======
snomad
Money line:

> It’s an example of a utility box that runs various random services - maybe
> acts as a bastion host or testing ground - and nobody quite manages it or
> knows what it is used for. This server is as weak as its weakest service;
> and because it is not purpose-managed, it can be difficult to keep track of
> what is running on it and ensure all services are patched and secured. If
> you have one of these servers floating around somewhere, you might want to
> think twice about keeping it - it may very well be the chink in your armor.

True for applications AND servers.

~~~
spc476
But the weakest point is always human:
[http://boston.conman.org/2004/09/19.1](http://boston.conman.org/2004/09/19.1)
(true story---happened to me)

------
djhworld
That was a delightful read.

Wasn't entirely sure how they knew where to look for the Passport credentials
though, unless they'd gleaned that information earlier. From the article the
author makes it sound like they had a limited window to mount the disk, pull
off what they wanted and shut everything down, did they already know where to
look?

~~~
bradleybuda
Thanks, and great question. During our exploration of the first box we popped,
we noticed that most of the services on the host had their code and
configuration in /usr/inversoft (or /usr/local/inversoft, I can't remember
which exactly). So when we got our shell on the app server, that was the first
place we looked, and luckily it didn't take long to find a config file
containing Passport credentials.

------
danielcid
Very fun read. I love following the train of thought and seeing where they
"failed".

Also, this Elasticsearch RCE has been patched a while ago and we still see a
lot of servers hacked because of it. In fact, there is a DDoS botnet made of
only ES servers that we have been tracking.

<unrelated>If you are using Elasticsearch, please patch it!</unrelated>

------
blincoln
For aspiring pen testers who read this article, there's an important catch: if
the rules of the challenge (or other Inversoft policies, like any bug bounty
programmes they may have) didn't specifically allow targeting their other
systems, then the author of the blog post could have been prosecuted under
e.g. the CFAA.

 _Always_ read the list of in-scope systems and rules of engagement before
starting. If that information hasn't been provided, then don't start until it
has.

------
shawkinaw
Two main lessons for me:

1\. Always run services (e.g. ElasticSearch) with a unique user dedicated to
that service and nothing else.

2\. You're never as secure as you think.

~~~
voidlogic
>Always run services (e.g. ElasticSearch) with a unique user dedicated to that
service and nothing else.

Quick tip; If you do this, you can also make your iptables rules be per user.
For example, "webserverUser" can only accept inbound connections on 80/443 and
only have outbound connections that are related to established inbound ones.
If an attacker gains execution as this user, they cannot download new code,
etc or even do DNS lookups for that matter.

~~~
breakingcups
Wow, that's great, I never considered doing that. Thanks!

~~~
startling
It's only marginally helpful: it doesn't actually prevent attackers from
uploading code to your server.

Instead of having 'nc -l 8080 | bash' or whatever as your payload, an attacker
can just run code instead. "pwd > /var/www/html/exfiltration.html". If they
absolutely need a shell, they could e.g. alter nginx or its config files to
run `bash` on POSTs to a hidden route.

This does make it a little trickier, and potentially a little easier to
detect. But it certainly doesn't make it so that "they cannot download new
code".

~~~
voidlogic
>This does make it a little trickier, and potentially a little easier to
detect.

Correct, I should have said, it eliminates many easy ways to download code.
Defense is depth is all about making the attackers job harder and increasing
their likelihood of being detected.

------
nexxer
Interesting, but I had to disable images to read through it. Too many
distracting and unecessary gifs.

~~~
stephengillie
The gifs are how the author expressed emotion. I felt they added to the
experience of reading the article.

------
ryanlol
Would've been much easier to just actually hack Linode.

:)

~~~
mjmasn
This has always been my concern with Linode and other cloud VPS providers. You
can secure the server all you want, but you can always get in via a
vulnerability in their API or out of band console access. 2FA helps a lot to
prevent access to Lish/Glish/Rescue mode console but as shown the API is
powerful enough to still be a potential problem if your key gets leaked
somehow.

~~~
ryanlol
Oh, but last I checked lish was an INCREDIBLY (can't emphasise this enough)
insecure hack around screen(1)[1] 2FA doesn't help you a bit.

Moral of the story is not to trust your provider. Don't use cloud if you don't
need it, don't let your host have your ipmi crdentials, run FDE (this has
saved several bitcoin exchanges from losing millions).

[1]. Here's one particularly funny thread from their support forum that
perfectly captures exactly how much of a mess this setup was (is? I haven't
hacked linode lately):
[https://forum.linode.com/viewtopic.php?t=3231%3E](https://forum.linode.com/viewtopic.php?t=3231%3E)

~~~
nickpsecurity
"I put in a support ticket about this back in December 2007." (source: comment
in mid-2008)

Usually a bad sign by itself lol...

------
toyg
So Inversoft got a full pentest for the price of a MacBook. Not bad ;)

Nice read anyway, some important lessons there.

------
kstra
Awesome read - Congrats to Polynome for a hack well done.

Here is Inversoft's account of the events leading up to the hack and the
lessons learned.

HackedThat: Mind the backdoor
[https://news.ycombinator.com/item?id=12390936](https://news.ycombinator.com/item?id=12390936)

------
fapjacks
I enjoyed the article. It inspired me to participate. However, the Iversoft
challenge site has a stupid password policy. Password policies are basically
the same thing as just storing them locally in plaintext. Requiring people to
perform password gymnastics is the most surefire way to get them to write it
down.

~~~
robotdan
There is probably some truth to this, but I don't know the answer is to allow
weak passwords. Every company I've ever worked for enforced some sort of
minimum password requirements.

What I do find to be a PITA is when you attempt to create a password that does
not meet the minimum rules and the error message gives you no indication of
what you need to change to meet the requirement.

~~~
rocqua
Sure, block the stupidly weak passwords.

But don't block a 6 word diceware phrase because it has not numbers (or
because it is to long.... looking at you PAYPAL).

Meanwhile, I'd venture that P@ssword1 meets their requirements...

~~~
robotdan
fair point.

------
jwcrux
Wow - fantastic work! I'm glad the ES article came in handy for you ;)

I can almost promise you that if you could reach the external ES instance and
exploit it - it was already exploited. They need to wipe that box and rebuild.

Overall, great writeup - thanks for sharing!

------
dizrupt
I have not read any hacked that blogs more intently than this one... wow.
_Slow Claps_

------
maristotle
I would love to see a screen share video of this. Can you make that happen
possibly?

------
YPCrumble
Could two factor authentication have stopped the hacker from exploiting the
Elasticsearch vulnerability or from implementing the final "Smash and Grab"
strategy?

------
haser_au
Really interesting read, well written. Congrats to you and your team.

------
throwanem
This is a great article. Unfortunately, it's littered with a bunch of animated
garbage that's super distracting and adds absolutely nothing of value. Please
fix.

~~~
robotdan
Even Blue Steel?! :-)

~~~
throwanem
Even Blue Steel. (What's a Blue Steel?)

~~~
robotdan
The first gif on that page, Derek Zoolander's famous look.
[http://www.urbandictionary.com/define.php?term=blue%20steel](http://www.urbandictionary.com/define.php?term=blue%20steel)

~~~
coredog64
Maybe I'm taking crazy pills, but that looked more like a Ferrari or LeTigre
to me.

~~~
Intermernet
You may be loco, but probably LeTigre. Definitely not Magnum though.

