

Linode Manager Two-Step Authentication - timewasted
https://blog.linode.com/2013/05/02/linode-manager-two-step-auth/

======
threeseed
After being bitten the first time with Linode I don't care what technical
measures they are taking. I want to know what process and policy changes have
been made.

Do they still store public/private keys on the same server ? How often are
they doing security audits (which clearly never happened before) ? Are they
still going to be dodgy and withhold key information from their users ? Are
users still going to find out hackings from IRC/Reddit rather than Linode
itself ?

Two factor authentication would have done NOTHING to prevent both hacking
attempts.

~~~
fiatpandas
I also find it really troubling they haven't released a "Here's what we're
doing different" blog post in response to the attack. Their only blog post on
the matter came a week (2 weeks?) after the intrusion, which they were of
course pressured to release after everyone found out via a pastebin IRC
transcript... By chance I happened to sign up for my first Linode account the
day before that hit HN.

I hope their silence on the aftermath is due to an ongoing investigation with
feds, or something, where they can't talk about it yet. Do they think their
customers are stupid and will forget the incident?

Imagine if AWS had a security breach of that magnitude. They would release an
initial 4000 word blog post in grave technical detail, and then follow up with
a 25 page white paper, or whatever.

Oh, and to stay on topic, I tried Linode's 2-factor with Google Authenticator
and it works well.

~~~
threeseed

       Do they think their customers are stupid and will forget the incident?
    

Yes. They have done it before and people on here still recommend them with a
straight face. It honestly confuses me that people care so little about
security.

~~~
hkmurakami
I'm one of those people who have a slight interest in the security but don't
know enough about it to be properly informed about my own decisions.

For people like me who basically can't make my own decisions properly, where
should I switch to? Is DigitalOcean better in this regard?

~~~
Kudos
Digital Ocean is largely untested in this regard.

~~~
raverbashing
And that's the real issue

Two factor auth addresses the user password as being a weak link, and this is
a nice step

Oh and btw, yes, the private keys were on the server, with a passphrase

------
nullrouted
I left Linode after 5 years of being a customer because I can no longer trust
them. I let the first issue slide as I thought they would learn and
communicate better to their customer base but the second incident has shown
they learned nothing.

Security issues will happen with any provider it is all in how a provider
communicates and remediates those issues. Linode has shown it will not
communicate thoroughly and does not talk about any remediation so why would
you trust a company like that with your data?

The last incident was extremely sad for me because I thought I was using a
company that I had a good relationship with. I could care less that CC details
were lost as CCs are easily replaceable and protected against fraudulent use.
What they lost was my trust which is far more valuable that my credit card
number.

~~~
agwa
Out of curiosity, who did you switch to? I would leave too (after being a
customer for nearly 8 years) but I'm having trouble finding other providers
which don't have their own set of issues.

~~~
nullrouted
I have been testing/working with a few different providers. The three I'm
currently working with the most are Ramnode, Gigenet and DigitalOcean.

Ramnode's panel is SolusVM which isn't as good as Linode but their performance
blows Linode out of the water. They have ipv4/ipv6, multiple locations
(Atlanta and Seattle) and a good owner who seems very open/honest with
customers. I expect we'll see feature enhancements as they grow bigger.

Gigenet Cloud has multiple locations (Chicago and Los Angeles), ipv4/ipv6,
good performance, good custom panel and a company that has been around for a
long time. They use a SAN for all their nodes. Overall one of the most
underrated cloud providers out there. (Note: I got free credits for beta
testing their cloud)

DigitalOcean has multiple locations (San Francisco, New York, Amsterdam), a
decent custom panel (would like to see more statistics and it seems a good
staff. They did have a security issues that they seemed very open about
([https://www.digitalocean.com/blog_posts/resolved-lvm-data-
is...](https://www.digitalocean.com/blog_posts/resolved-lvm-data-issue))

Other hosts I have tested/used but did not choose:

Rackspace - Excellent panel, ho-hum support/performance. My biggest issue is
they lock instance throughput and refuse to change that. If you have a 512
instance you are locked to 20 mbit which doesn't make sense as you are billed
per GB. I asked to have this unlocked as my instances push more and they
refused.

Amazon AWS - Great interface but the lack of ipv6 (unless you buy ELB) and
poor performance had me look elsewhere.

Others tested/used:

Joyentcloud, Terremark, Zerigo (Was a long time customer but they went
downhill when 8x8 bought them), Voxcloud, Cloudsigma , Azure, HP Cloud,
Stormondemand (Another good cloud provider that just didn't fit with me),
VPS.net, Gandi

~~~
kyrra
Great post. I'm interested to know which you end up ultimately choosing. I
just worry that ramnode and digital ocean won't be able to keep their current
price model and still maintain quality service in the years to come.

------
epo
I find all the whining of all these armchair security experts a bit wearing.
No one outside of Linode and the alleged hacker knows exactly why and how
Linode was hacked so quit speculating, you are simply making things up. "Ooh,
this is a bad thing, I bet Linode did this bad thing". For example people are
suggesting that 2FA is useless if outsiders had free run of Linode's
infrastructure, and so it would be. BUT, is there any evidence whatsoever that
this was the case? If not then STFU and stop spreading lies. 2FA is useless
against a nuclear strike, what exactly is the point of saying so? Anyone can
fantasise disasters.

To get some positive content out of this thread. Is there a VM provider with a
provably better security record than Linode?

If you are going to stay with Linode then 2FA seems like a no brainer. So, is
there a simple way to get the 2FA iDevice systems (Google, Duo) to work on
multiple devices, say to allow an iPad or an iPhone to be used
interchangeably?

------
rdl
Wow, this is great.

It would be nice if they let you set up SSL cert + MFA + password. I am kind
of angry that modern desktop browsers continue to make SSL certs suck so much,
but they're decent on mobile. I hope a future version of OSX builds in great
cert management and UI/UX with local biometrics or something.

------
luser001
I was hoping for Yubikey support. But I'll take this for now.

I'll have to see if the Google Authenticator app shows up on all of my
iDevices linked to my Apple account and whether the code from any of them will
work (from the setup process, I don't see why not). Does anybody know?

If the app will work from any of iDevices, it would not be secure enough for a
service storing bitcoins :) because the second factor should be hard to copy
(which a real hardware token is, while a software token isn't).

~~~
threeseed

       it would not be secure enough for a service storing bitcoins
    

Linode was hacked twice (once where Bitcoins were stolen) in recent times and
was shown to have the worst security practices I've ever seen. They have never
been secure enough for storing Bitcoins.

~~~
sithlord2
Are you serious??

What you were doing is the equivalent of living your wallet in a public place
unattended, and then shouting and screaming it got stolen. You are putting
your bitcoin wallet on a public accessible server, you should know the risks
of this by now.

Don't leave your wallet in a public place unattended, that includes your
bitcoin wallet.

Let me guess, you didn't bother to encrypt your wallet either, didn't you?

Don't blame others for lack of security, if you can't even figure out your own
security best practices...

~~~
threeseed
Are you replying to the right person ?

I don't own Bitcoins. And if I did I would never, ever host them on a Linode
server.

------
oinksoft
Not everybody owns or wants a smartphone. Linode needs to extend this to some
non-smartphone device, like YubiKey, or offer SMS codes, like Google
Authenticator. This is a step in the right direction, but is ultimately
disappointing for me.

~~~
madsushi
I have to imagine the overlap between Linode customers and smart phone owners
was so large (and the cost of implementation so low) that leaving out hardware
authenticators makes sense for v1.

~~~
rdl
One area where hardware authenticators work really well is where you want to
split access to an account, or have some accountable/logged procedure for it.
You put the physical token in an envelope and in a safe/put it in the control
of a finance person. Tech people have the password, but need to request the
token to do logins.

This also requires having role accounts which aren't able to reset
authentication settings when logged in, though, to really be good (or else you
just disable tokens on first successful login).

Also works well for paranoid people who don't trust their phone, or people who
log in _only_ from a phone/tablet and thus where MFA is really one-device-
authentication.

~~~
alanctgardner2
I see what you mean about losing the phone, but unless you're saving your
password locally it still satisfies the old "Something you have, and something
you know" rule. If you lose your phone, the attacker won't know your password.
And an attacker without your phone won't have your OTP.

These physically secure OTP techniques are interesting, but shouldn't you have
accountability at the system level anyways? If everyone has a two-factor
device and a password, it's pretty tough to plausibly deny that you logged
into a server. Someone would have to guessed your password and stolen your
device.

------
nilved
This will do absolutely nothing if Linode themselves are hacked, which is what
happened the past two (100% of the) times.

~~~
pionar
Ok, so they get hacked and passwords are stolen and those are cracked. Guess
what? They're useless. With 2FA, the attackers still won't be able to get in.

~~~
bndr
The problem wasn't in passwords being stolen. CC information was allegedly
leaked.

~~~
nwh
Allegedly? They admitted it was.

~~~
bndr
They admitted that the encrypted CC numbers were leaked, they didn't mention
if the encryption keys were stored on the same machine. The alleged hacker
said that the encryption keys were stored on the same machine, making the
encryption useless.

~~~
astrodust
It was also made clear that the encryption key was protected by a passphrase
which was not stored on the machine.

~~~
sendob
"which was not stored on the machine", like they should be commended ( Reminds
me of exams where you received some credit for including your name... ).

I am sorry, them confirming this fact, and even if I recall adding a smiley in
the tweet they did it, just cemented that they do not understand their
business.

They clearly wish to give the impression that they are "secure". They need
more lock icons...they are almost as effective as the racing stickers on my
car!

~~~
astrodust
The real problem here is that PCI certification is an absolute joke.

There should be several classes of certification, from "I want to sell a few
pet rocks" to "I'm Apple with 150,000,000 credit cards on file". Right now
there's basically two.

------
funkaster
just a warning: I just enabled it and it wasn't working with my account &
google authenticator for android. I had to call customer support in order to
disable the feature so I could login into my account again.

~~~
sendob
Would you mind sharing details ( process wise) of what they did to validate
that it was a legitimate request from the account holder?

------
nivla
Regardless of Linode's previous screwups, I welcome this change and I hope
more hosting companies to offer 2-factor authentication soon.

For anyone installing the Linode's recommended Windows App "Authenticator",
WARNING, it does not work! I was locked out! I then used the Microsoft's
Authenticator app to find the right token.

Do not logout without verifying it works first in an incognito mode. Better
yet, save the secret key temporarily to your PC.

------
jro7
Doesn't work for the Linode Manager iPhone app. After enabling 2FA on Linode,
the iPhone app still uses only user/pass..... (app last updated, jan 17th,
2011)

------
peddamat
Ok, so can anyone recommend an established VPS provider, with a comparable
management interface, and a track-record of excellent security practices?

~~~
rdl
Amazon AWS has the best security of any virtual server provider I've seen, by
miles. There might be specialty providers (e.g. FireHost) which are good
dedicated server offerings, too, but I haven't evaluated them -- it usually is
"AWS, is it good enough?" and then if no, directly to a cage, do not pass go,
do not collect $50k.

AWS also has the best first and second derivative on everything related to
product; they were essentially crippled crap in 2006, and have turned into a
viable option over the past years, without slowing down. Compared to the level
of innovation in colo/dedicated hosting (~zero per year) and openstack, AWS is
amazing.

It's still inferior to a good on-premises or colocated environment (mainly due
to technical limitations in the virtualized environment; AWS's policy is top-
notch commercial standard), but that may not matter for you. AWS pricing and
performance is also worse in a lot of ways than dedicated hardware, but may
also not matter to you.

A lot of the big cloud/dedicated hosting companies have decent security
(SoftLayer, Rackspace), but aren't as good at AWS at policy or technical
security. The sketchy VPS providers are miles below the middling standard set
by companies like Rackspace.

Linode is solidly in the "sketchy VPS provider" realm. A bit better for
availability, and not likely to actually be attacking you themselves, but not
a responsible choice for anyone who cares about security from everything I've
seen.

PaaS, in practice, is also a good solution if you care about security but have
no skills or budget. While Heroku has its own set of problems around price,
performance, and availability, it's more secure out of the box than a badly
configured/maintained AWS deployment of your own, or a badly configured on-
premises/colocated cage or dedicated servers.

~~~
kyrra
How about for those that just want a small instance system. I just do the
smallest linode setup for some personal projects. From my understanding, AWS
is expensive for that type of use case?

~~~
rdl
AWS Free Tier is...free.

I'd probably go with <http://prgmr.com/xen/> for low end above that.

~~~
jnw2
How do people expect prgmr.com compares with linode in terms of security?

------
rmason
Glad to see Ann Arbor's Duo Security listed as a choice. Know the team and if
you chose the Duo option you won't regret it.

------
yogo
Ah, there's a bright side to the recent breach after all.

------
thezach
Linode is no longer worth paying any attention too....

~~~
dllthomas
Because obviously nothing ever improves.

~~~
seany
They certainly aren't improving their transparency.

~~~
dllthomas
Their transparency hasn't been awful; it should be better. I am for now giving
the benefit of the doubt that they'll be releasing further details as they
become more certain of them and/or investigations conclude.

------
crashbunny
it's security theater. if someone gets in again, they can download the api
keys and own you that way.

------
drivebyacct2
Is this likely to actually fix anything? Were the past intrusions via the
manager? Or via a compromise of the login to the manager via individual user
accounts?

Or is this just a show? Either way, this question itself reflects the fact
that they refuse to give proper information and postmortems.

