Hacker News new | past | comments | ask | show | jobs | submit login
Linode hacked, CCs and passwords leaked (slashdot.org)
753 points by DiabloD3 on Apr 15, 2013 | hide | past | web | favorite | 404 comments

Ah this is so shit. I want to support Linode, I've had nothing but a good experience. But I just had to check my credit card to be sure they hadn't lost my details. I've NEVER had to do that before with anyone - they've got to respond fast here because if I don't trust them with my CC then I can't leave five-figure contracts at jeopardy hosted on their servers.

I've been living comfortably on Linode servers for over three years. This is like suddenly being evicted and having to pack my stuff up and find another apartment.

I have to wait for some sort of verification for this but if true then I have to leave Linode. I have client sites hosted here - not for cost reasons, just because I like Linode.

For the sake of $5 a month I can't even take the slightest risk of being criticised for using Linode. And this lack of transparency could be a nail in the coffin here.

I don't want to waste a couple of days on this but that's what's going to be involved if this is true.

I've now heard from a number of people using Linode that have suspicious activities on the cc which they used with Linode.

I just called up my bank to tell them to 'block' it as a precaution (I will now have to give them a visit later today to get a new card). I encourage all other Linode customers to do the same, because it'll be easier to just spend half an hour doing this instead of spending hours upon hours disputing specific transactions.

Linode customer support keeps saying they have "no comment" on this issue (which I suppose does make sense -- I'm assuming they've been ordered by law enforcement persons to not share details), so as we're not being given much information to work with... just treat this as a worst-case scenario (all names, addresses, credit card numbers, etc. have been compromised). Do operate now with the assumption that all of this data has been compromised and may very well be public soon.

"because it'll be easier to just spend half an hour doing this instead of spending hours upon hours disputing specific transactions."

I live on the internet. Put my credit card out on many services. Over the last 5 to 8 years I've had my credit card numbers taken I believe 4 times.

Never had to dispute it once. These Credit Card companies and Banks have a stake in not allowing your account to be drained.

I think it would be a waste of time to go out and cancel our CC until hearing from Linode that yes, CC information was taken.

It depends on how sophisticated the identify theft is. I had a good friend who was taken for about $9000 in credit card fraud in 1998/1999, with Well Fargo. It took him the better part of six months, and endless correspondence with WF to prove all of the purchases were not his. There are lots of stories of people who were financially wiped out, to the point of bankruptcy, because of Credit Card/Identify fraud.

With that said - almost everyone seems to feel comfortable handing out their credit card to random taxi drivers, waiters, sales staff - with no idea whether a copy of their information is being taken down. Heck - if you give them the Credit Card, they even get your CCV as well.

That makes me wonder why the credit card system is so insecure in the first place. Why are credit card systems not secured with a password that the merchant never gets to see? Yet at the same time credit card suppliers keep bragging about how "secure" their cards are.

Funny that you mention that. After a few bouts with fraud (because as i mention I have my CC out to many services), I was wanting a way to track down the offending service.. I was thinking something along the lines of a vendor-specific set of numbers to be run through.

I even wrote a blog post about it. I probably don't know what I'm talking about, but these were my thoughts at the time:

Wishlist – A Method to Pre-Approve and Track Credit Card Transactions

The issue:

A business using a credit card doing business with a relatively small number of vendors wanting to first avoid credit card fraud (stolen numbers) and secondly wanting to easily track down the offending business.

The idea:

The business would like to approve particular vendors to use the credit card with number 0000-0000-0000-0000 with each individual business pre-approved to run the transaction with a 5th set of identifiable numbers, so something like 0000-0000-0000-0000-0001.

If the credit card is used to make a fraudulent transaction, then ideally, they would have had to have used the 5th set of identifying numbers. This 5th set of identifiable numbers would then allow for easy tracking of the offending vendor, which would allow the business to either re-think doing business with them, or to serve as a starting point discuss security issues with the vendor’s credit card transaction processes.


Basically, I believe there may be a need for a new or value added credit card type service. This transaction type would require a 5th set of numbers which have been assigned to pre-approved vendors. This 5 number set (ie. 0000-0000-0000-0000-0002) credit card transaction would most likely prevent theft right off (because the vendor is pre-approved and should provide their own private key (ie. CCV) to put through the transaction). Secondly, if and when the credit card number is stolen and used to make a fraudulent purchase, then, at least with the 5th number set a vendor can be identified and security policy with them can be re-evaluated.


> Why are credit card systems not secured with a password that the merchant never gets to see?

My bank in Sweden requires MasterCard SecureCode for all online transactions on their debit cards. Stores that don't support it simply won't work with the card.

So, it's up to the bank how secure they want it to be. The technology is there.

Of course, the main difference being signed up with such a bank makes is it's now harder to legally dispute fraudulent transactions on your card.

Yep. I believe that's also true of Chip-and-PIN transactions. The liability is shifted onto the consumer to keep their PIN secret.

I did it, not worth the worrying. But then again I only bill two things to this CC, one of which is annual.

I've had roughly the same experience: in the last 8 years, I've had suspicious activity on my CC about 5 times. Each time, the bank caught and trapped it before I noticed and issued me a new card quickly. I've only had to fill out paperwork for a disputed charge once, and it was a 2-page, 2 question, sign-and-mail-it-in deal.

My advice is different, though. I notice that I tend to get lucky in places where people can have very aggravating experiences. I'd say that if you've had problems before, then anticipate problems this time around too. If you haven't, then don't bother.

Or most people are lucky, and you hear about those who have aggravating experiences.

Wow, four times? You should probably be more careful about who you give your number to. Personally, I usually get a new card every 3-5 months. If someone ever sat on my card number, it's useless to them now. Never had any issues either.

@kansface It's not bad for your credit rating. A number is simply a representation of the account. The account doesn't change. It's not like getting a whole new item of credit issued. Just the means to access it.

Also, great idea. But a pain, because most of my bills - cell, internet, insurance(s), etc all go through my credit cards. Is a gigantic pain to change the numbers.

Even old card numbers can be used for transactions in some cases.

Sure, but that isn't the former owners problem.

Are you thinking of expired cards? That is different.

I can be in some cases. I got mugged and my card was used to pay for parking garages for 1.5 years until it expired even though it was canceled and blocked by the issuing bank. They said that for some transactions, the blocking mechanisms are so expensive its more economically sane to them to refund whatever was drawn.

Ugh. Did that require you to protest each charge? Or did you get a charge and credit on every bill?

I also do that, but it's because I'm scared of recurring subscriptions that I've forgotten about, especially those that decide to sneak into my pocket after I've deliberately canceled them.

This is bad for your credit report I believe.

He's not tearing down and setting back up the entire credit line, just the card number associated with it. It won't be reflected on any credit reports.

I would be utterly shocked if nobody using Linode had suspicious activity on their CC. Linode has lots of customers, and at any given time, some of them probably have suspicious activity going on.

There's baseless speculation and then there's I have some information speculation. I'm operating on heuristics which rely on information that is handily available. Yes, in the end you're right, I'm just speculating. But hey, it's better to err on the side of caution.

Credit card numbers are of pretty low value. Like way less than a buck in medium volume and still just a few bucks for the super premium ones. And there is way, way more inventory of them than interested buyers. The likelyhood of a coordinated break in of a large hosting service with the intention of stealing credit cards is pretty low, and the chance that they'd be exploited so quickly is even lower.

Unless the attacker dumped them all (semi) publicly, the more likely explanation is that the breakin caused people to check their accounts and a statistically normal percentage of them showed fraud from another origin. But anybody who sees it will be sure to get online and find others in a similar situation.

Everybody would be doing themselves a big favor if they stopped treating CC info as the #1 scary OMG data theft. The banks programmed you to care because congress made sure they're liable instead of you. Theoretically you might owe $50 due to fraud but practically you never pay a dime. Sure it's a bit of a pain in the ass to get resolved, but it's not worth stressing about until it happens.

I'd be way more concerned if my hoster lost my contact info, ip logs and identity challenge questions & answers.

You bring up very good points, thanks.

I contacted Linode support and they've said in clear terms that they have no evidence that payment information of customers was accessed. I initially signed up for Linode because my friends spoke highly of the tech people working at Linode. Right now amidst all the commotions it's ryan's words (some anonymous dude who joined #linode/irc.oftc.net) vs. an established company's. I'm just going to now stop worrying and get back to my work.

On an interesting note, the big target who actually incurred identifiable damage was seclists.org: http://seclists.org/nmap-dev/2013/q2/3

>I contacted Linode support and they've said in clear terms that they have no evidence that payment information of customers was accessed.

Well that's a first. They were very evasive about it earlier.

Apparently they weren't entirely honest with you, then: https://news.ycombinator.com/item?id=5556846

If it is indeed true that credit card numbers were compromised, it would behove Linode to tell their customers quickly so they can take the proper action.

With this lack of transparency, I feel like I had no choice but to block my card.

There's no lack of transparency here. Linode expressly said in their blog post that no CC details were leaked.

> In addition, we have found no evidence that payment information of any customer was accessed.

The question isn't transparency, but trustworthiness. Either Linode is telling the truth, and this anonymous IRC person with a pastebin is trolling everyone, or Linode is lying (or alternatively, Linode is incompetent and simply didn't detect the CC access). At the moment I'm going with Linode is telling the truth, because honestly, am I going to believe an anonymous person on IRC over a company I do business with?

I'm thinking here why Linode holds CC data on its servers in the first place. Anyone care to weight in here?

Secondly, if they hold that data, it's possible one day someone will find a security breach and will access that data. The best solution is to never hold that data.

Since I don't trust most of the systems I use AND Linode has not denied it holds that data... I'm more inclined to believe in this anonymous IRC guy and err on the side of caution.

If Linode had come out and said "Look, we don't hold your CC number in our database" then I think there would be very little reason to be concerned. However...

No weird activities on mine either but I will give a call to my CC company anyway. I have had to cancel the card I use on linode twice in the past few months because of suspicious activities. I just didn't think it would be coming from Linode

No weird activity here (no declined transactions or anything)

Mine is not a CC or a debit card, it's a 'prepayed card' so liability is limited.

Still, I'm considering calling the lost/stolen line of the card.

So here's what Linode support is actually saying when asked about the breach:

   Thank you for contacting us. We have no evidence at this time that any payment information was compromised.

I asked about the security measures and they answered with:

"We appreciate the response, and we can assure you that we have implemented all appropriate measures to provide the maximum amount of protection to our customers."

That severely underestimates the intelligence and wrath of their highly technical customer base.

Yeah I did the same thing, but new card takes 5-7 days in my case. Very disappointed that I had to find out about this incident here, imagine all the customers who dont happen to look on slashdot or hacker news.

Do this immediately if it is a debit card!

If it's a debit card that can be used as a credit card (and it must be, otherwise it couldn't have been used to pay for Linode), then it enjoys the same protection as regular credit cards when it's used as one.

Yeah, but it's one thing to deal with a line of credit that is maxed out by fraud and another to deal with an empty checking account.

Very true. If your bank gives you a separate savings account you can transfer the bulk of your money there and just use the card from the current account to limit your exposure.

What about for people who did not use a credit card to pay for linode but instead relied on PayPal. Should they follow the same steps? What about other cautious steps?

I don't think they allow paypal as a payment method

I've got a PayPal business mastercard which is connected to PayPal Smart Connect. They may not allow PayPal proper but you could pay using PayPal by way of their card. I use my PayPal card hooked into Smart Connect for a good number of recurring payments like this.

So I guess the answer would be, if you ended up hooking your PayPal account up to Linode in the way I described, yeah follow the same steps as other cards, otherwise it's not even possible to have a problem.

Quite. I like the company and their servers are good - but we need a detailed response, and we need one now.

I'd say support tickets or posting on their forum[1] may help try to get a response. But based one one of the support ticket responses posted in the comments in this HN story already, it sounds like Linode isn't allowed to release that kind of information yet. They may be waiting on the police and/or their lawyers to allow them to talk publicly about it. And if that isn't the gating factor, they are probably trying to determine what they are required and should share about the incident.

[1] http://forum.linode.com/viewtopic.php?f=20&t=9978

They may be waiting on the police and/or their lawyers to allow them to talk publicly about it.

Waiting for their own lawyers would be a particularly weak excuse. This is a priority, and they are responsible from conveying that urgency to their lawyers.

its a recursion! the forum points back to here

That's an endless loop. Recursion would be the forum pointing to itself.

/end nitpick

Anyone know of any good way to export linode images to other VPS providers? Seems like I'll have to be doing it manually.

If your using lvm you can create a snapshot to do this while online, if not just read from your disk (assuming sda here):

   1. (offline) Boot new and old VM servers from live CD
   2. old server: dd if=/dev/sda bs=8M | pbzip2 -c | netcat <newhost> <random high port>
   3. new server: netcat -l <same port> | pbzip2 -cd | dd of=/dev/sda bs=8M
Compression: You can use something besides pbzip2, maybe pigz of if you only have a single core use bzip2 or gzip. Security: You probably want to add encryption to this pipeline.

Woah woah woah isn't this just transferring the contents of /dev/sda in the clear over the wire? Shouldn't you at least do this over SSH?

>>Shouldn't you at least do this over SSH?

Yeah, I mentioned that at the bottom of my post. Using ssh or some other inline encryption would be a good idea if it is a system you care about. If you have a site to site VPN tunnel between your systems, you can skip adding the encryption.

Yeah, I mentioned that at the bottom of my post

It wasn't there when windsurfer replied to you (I was reading the thread earlier), hence his question.

I thought my only edit had been to replace might with probably. If this was not the case I'm sorry. Maybe I edited it to add that right after posting and forgot-

Thanks. I thought I was just dumb.

dd if=/dev/sda bs=1M | ssh root@<newhost> dd of=/dev/sda bs=1M

let SSH handle compression for you instead.

That is one (good) way to handle the encryption.

You don't want to leave out the compression or block size however, so add those back in. Most WAN links are low bandwidth enough that compression will not slow things down (this is usually true even on 1 GBE LAN links for pigz) and in my experience the speed-up is substantial.

pigz is much faster than using ssh compression as it is multicore. apt-get it or http://zlib.net/pigz/

the compression is unnecessary here, you're moving a max of less than a couple of hundred GB of data.

It's pretty necessary if you aren't moving between colocated boxes. "less than a couple hundred GB of data" is still a lot of data to be moving around, even on a 100Mb/s link, which almost no one in America has residentially, and isn't even a guarantee for colo'd machines.

In most cases, compression at the raw block level will result in HUGE size savings, especially since a lot of that may be free space. It might even make this transfer tenable.

A few hundred GB here, a few hundred GB there, pretty soon you're talking real bandwidth.

Just add -C to the ssh command to enable compression.

That's a great way to screw up the filesystem on the destination. You'll get mounted fs write combined with low-level block writes, filesystem in a blender.

LVM snapshot, that's the way I'd go about it. Is how we do our backups, to maintain a good state. Mount the snapshot, and do the same with it.

There's very nice utility called Ghost 4 Unix that makes the whole thing even easier: http://www.feyrer.de/g4u/

Looks like overkill for a simple one time task that can easily be handled the UNIX way. Worth looking into if you need to do this alot-

Your procedure already requires booting out of live cd, and then uses "unix way" that might be easy to screw up for less advanced people, so using G4U simplifies it substantially, does the same good job and by no means is a 'overkill' ;)

DigitalOcean should probably put this in their marketing materials.

Generally speaking, this is one example where having a good deployment system starts to look extremely valuable (along with tested backups and restores for non-deployed data).

I'd have to agree with you, but I didn't figure creating such a system for my hobby projects was worth it.

Check out http://www.cloudconverter.com - move from Linode to pretty much any other provider out there.

You mean the comprised image?

Please do realize: lack of suspicious charges on your credit card is not evidence that it was not stolen.

Aah!, the self-fulfilling nature of paranoia.

How about you guys cool it and stop organizing a lynching mob devoid of any real data? It's embarrassing. HN is supposed to be populated with lots of very smart, data-driven analytical folks. Yet, every time something like this happens out of the woodwork come people who would ran you and your children down in the event of an emergency rather than turn around, carefully evaluate the situation, and help you. Don't be a moron. Stop it. For all you know there's a serious law enforcement effort under way that prevents Linode from talking.

For the record, I am a Linode customer and just got a new server to migrate a couple of sites into. My plans have not been altered at all by this. I have no data to suggest I should.

I am a customer too, and I wield no torch or pitchfork, but I grow increasingly frustrated at the lack of response from Linode.

I understand your point about the idea that they may be unable to speak to the issue due to law enforcement efforts, but for the moment, acknowledgement would be satisfactory. I would be happy with, "We're aware of the rumors regarding the intrusion at Linode this past week. We are working with law enforcement and cannot comment on details at this time. However, we will provide a full postmortem once we are able to do so."

The problem is when the explanation never comes. It's OK if it's not this second, but tell us it's coming, and then follow through. Complete silence is frustrating.

Here's the response I got from my support ticket:

>Thank you for your inquiry, and I certainly understand your concern. We are still conducting an active investigation and unable to disclose most information at this time. This being said, we do not yet have any evidence that any payment information of any customers have been compromised. We will be releasing further information regarding the incident soon, so please keep watch of our website and blog for said information. If you have any further questions, please feel free to ask.

That actually sounds pretty close to what you're asking for, although I have to say it didn't make me feel much better. It would be nice if they would make a public statement too.

I get it. It's just that these things keep getting exaggerated, twisted and blown out of proportion in the best Fox News, CNN, MSNBC, et al. style. It's sad to see it happen on HN, where things ought to be far more analytical and data (aka: facts) driven.

Remember this is the SECOND time it has happened.

Before it was the widespread hacking that resulted in Bitcoins being stolen. And users were never told exactly what happened and what was done about it.

People have every right to expect the worse with a company with a track record as poor as Linodes.

Mob-mentality. Unfortunately naturally occurs whenever a group gets to be about the size of a mob.

I don't think that quite explains it. Someone who calls himself "ryan" on IRC makes unsubstantiated claims, and in response many dozens of people say loathsome, sneering things about the security practices of this company. The appearance of a mob emerges after a thread exists, it doesn't create the thread.

The IQ of a Mob = (The lowest IQ in the Mob) / (The size of the Mob)

This from a Terry Pratchett book. Can't remember which one, but it's one of the earlier Discworld books.

Reading through the IRC chat log it didn't look like he'd proved he had the CC numbers.

I mean, if I was a linode customer I'd definitely be on the phone to the bank, but this guy presented no evidence I'm aware of that he had the kind of access he claims.

From a purported abridged chatlog with the alleged hacker:

> 05:42 < ryan||> credit cards were encrypted, sadly both the private and public keys were stored on the webserver so that provides 0 additional security

> 06:00 < ryann> They did try to encrypt them, but using public key encryption doesn't work if you have the public and private key in the same directory


Here is what Linode replied to me when I asked them about that chat log in a support ticket:

  Thank you for reaching out. We appreciate and understand your concerns. At this time the evidence suggest that this activity was targeting a specific customer. We are unable to release any additional details regarding this incident at this time, as there is an ongoing investigation.
  We have no comment regarding ryan*'s comments in #linode. You are of course free to take any steps you deem prudent or necessary to ensure the integrity of your online presence.
  I am sorry that we cannot provide more information at this time. As always feel free to contact us at any time with any future concerns.


These guys are looking totally incompetent at this point.

If you believe this Ryan guy, credit cards stored on the same server as the key to decrypt them, Lish passwords stored in plain text, they've known for some time and lied about what actually happened and now they're saying "we won't do anything about it" via email?

"You are of course free to take any steps you deem prudent or necessary to ensure the integrity of your online presence."


Edit: not to mention they "made a deal" with the hacker not to tell anyone? What the hell?

> If you believe this Ryan guy

That's a rather key assumption. If you don't believe him, then all you have is a trolling (or at least self-aggrandizing) hacker whose credentials consist solely of logging into an IRC channel, refusing to identify who he was working with, and offering no tangible proof of having compromised any CC info.

On the other hand, it's conceivable that if ryan managed to get into the files a customer was hosting on Linode, and that customer was improperly storing CC info, then their customers' info would have been vulnerable, and ryan's claims would be sort of half-true. Even so, that wouldn't directly affect other Linode customers or put liability in Linode's lap.

Regarding the "made a deal" assertion, I wouldn't take the IRC log hook-line-and-sinker. There's probably a mixture of truth and lies.

for the record, i am the person who started the WHT thread.

there is a mixture of truth and lies on both sides, to be honest.

i am annoyed with it, because i reached out to several linode employees privately to given them an opportunity to explain what was going on -- they either said 'no comment' or said my linode was fine.

based on the irc log, that is clearly not the case. which is why i decided to raise my concerns publically.

luckily for me, my linode was not doing anything mission-critical, just some secondary monitoring and running an ircd for a network i like using, but there are others who are using linode for mission-critical work, and they deserve more transparency than this.

what makes you think you are more special then anyone else? Why would they tell you more details, than the rest of the people?

Sorry if you get offended that they didn't tell you much more. But seriously? you are not special. This whole thread is a lynch mob.

Is he more special than other people, or should we ALL have been informed properly?

To be fair the hacker didn't say the keys were stored on the same server as the credit card numbers, he said they were stored on the web server. It's most likely the database containing the CC numbers resides on a separate set of boxes than the web servers.

The Cigital-recommended way to hash your passwords is to use an HMAC/scrypt combo, with the HMAC key stored on the app server (not the database).

What Linode did may, or may not, be dumb. They are being tight-lipped so we can only guess.

Why would you use an HMAC for password storage? It's not like length extension attacks are relevant in that application.

the database server is local, read the entire log...

Despite what the other replies here are saying, this seems like a perfectly acceptable response to me. This comes off to me not as they're refusing to talk about it, but they _can't_ talk about it, presumably because of an ongoing investigation. I'm not sure what else people here are expecting them to say.

Then why not say that?

They did.

> We are unable to release any additional details regarding this incident at this time, as there is an ongoing investigation.

The pasted text from Linode appears to say exactly that:

> We are unable to release any additional details regarding this incident at this time, as there is an ongoing investigation.

Edit: Oops, didn't refresh before replying. jmilloy got it first! Sorry about that.

That is an absurd response. I don't care if they believe a specific customer was targeted, I want to know what happened and what information may have been compromised.

They probably can't say anything just yet. It's fair to be mad about them not realizing (or worse, covering up) the breadth of this breach, but do keep in mind that if they're working with the FBI, they've probably been asked to keep a lid on their official response for a few hours.

That is not the way to handle this issue. I've found my one problem with Linode is they are arrogant. It comes off pretty strong if you ever ask them questions in chat.

One thing to note is that the irc channel if that is what you mean by chat, is mostly populated by non linode staff. And many of us there tend to be sarcastic as we idle there and chat about all sorts of stuff while we are bored at work or whatever. Unless it's someone with ops, you aren't getting an official reply and even then for something official they usually refer to ticket system.

This is ridiculously unprofessional.

Well the steps I would like to take regarding linode aren't exactly legal. Satisfying yes, but not legal.

But I just checked and my credit card haven't been used anywhere I didn't use it.

Wouldn't doing that be a massive PCI violation? Aren't there extensive audits for this sort of thing?

Extensive PCI audits. Heh.

Like the "scan" that said our linux boxes were running an out-of-date version of IIS.

That compliance web form I absentmindedly clicked through sure had a lot of buttons.

Can't think of the exact word to describe that practice (what you did in response to the long list of questions which I've seen) but on the part of the company requesting you to answer the questions it's more or less a "absence of malice" type of thing that allows them to appear that they are doing the right thing while fully knowing that people are doing what you are doing. It's a "we will look the other way until we need to show that it's not our fault because we have passed the liability to you - look you acknowledge doing all the right things".

It does not mean anything until they decide you are not compliant and you need to prove you are compliant. (I've never had to but I'd appreciate insight from people who have)

I've done PCI "audits" for several companies I've worked for; it's a checklist you go down yourself. That's why its called a "pci self assessment".

Actually, if you're processing cards directly, you do in fact need to have an PCI-qualified outside firm† (a QSA) audit you for PCI compliance. But those audits are notoriously superficial; PCI audits are a race-to-the-bottom affair.

We are not one of those.

Note that you hire those firms yourself, and they work for you. They want you to pass the audit and will work to make that happen.

Every one of these reviews that I've been involved in has been conducted by a couple of guys with laughable abilities.

The quality of PCI security audits is a continual aggravation to everyone I routinely talk to in my industry. I've told more than one client: if you need a QSA audit, get the cheapest one you can. If you need a software security assessment, don't use a QSA firm.

"But those audits are notoriously superficial"

Will add that just having gone through an ICANN registrar audit (which by the way were specified and supposed to be done literally 10 or 12 years ago but never requested by ICANN) with a third party company hired (accounting firm) it's total compliance theater.

Add: "hired by ICANN after a bidding process". Same happened with data escrow which was just implemented a few years ago and is operated by Iron Mountain.

Companies are split into PCI Levels based on how much money/customers they handle. Level 1 are big companies like amazon, level 2 are medium sized online retailers generally, and level 3 are smaller retailers.

The 'lower' your level, the easier the PCI audits are. If you are level 1 you have mandatory external audits. If you are level 2 you have a 'self assessment' which is basically a checklist which says "Yes, I promise I'm in compliance".

If you have a confirmed breach, you are upgraded to Level 1 merchant audit requirements. This is generally quite costly as the external audit is extensive and must be paid for.

Most PCI audits are not mandatory unless you have had a breech that compromises data. Before that "self-certification" reigns.

The audits are toothless and ultimately the audit only happens once (if ever) and people keeping pub/secret key in the same place unprotected... well.... They're just unlikely to get security at all...

"Don't worry, all the doors are locked."

"Where are the keys?"

"In the locks."

The paste-bin link he provided seems to time out for me.

But if the things he claim in there is even half way true, nobody involved with linode should ever be allowed to be in business every again.

pastebin is a directory listing of linode.com - trying out a few of the files checks out, including very difficult to guess file names such as:


Am I the only one who is more confused about why there are compiled java classes and AMI BIOS updates in the www directory than about the hacking itself?

> Am I the only one who is more confused about why there are compiled java classes and AMI BIOS updates in the www directory than about the hacking itself?

Sysadmins being lazy.

Somebody needs to get a file from a workstation to a remote machine. There's a firewall in the way somewhere that prevents SSH directly between them or one of them is a Windows box that isn't running an SSH server.

The "correct" solution is complicated and takes 5 minutes to setup. So the sysadmin just copies the file to the web server and downloads it with a browser on the other end. Because port 80 is always open.

Something that took me a long time to notice was that you can actually copy files directly over a RDP session from/to the local machine or other RDP sessions using the clipboard.. Even nested RDPs. Has been an absolute timesaver to know about when doing Windows admin work.

Doesn't work for me when using an RDP client on Linux. Is this an RDP spec thing or a microsoft only feature?

I think it depends on how you mount your clipboard/drives on rdp connect. To get it right with windows you just check the share clipboard/share drive checkboxes and off to the races. With rdesktop you have to throw the -r flag and mount a clip board and then the -r flag and mount a drive. Not sure about other Linux clients but I'm sure there's a similar option in all of them.

Excellent, thank you very much. I use KRDC and a bit of googling shows that that uses rdesktop in the background. I shall investigate.

There is literally zero evidence to suggest they compromised anything more than a webserver.

That's totally shit.

Its also why we invoice and take wire payments rather than storing CC details. There's just so much to go wrong.

Also PKI is shit for this sort of thing. As demonstrated, the moment that public key is gone, then the whole system falls like a house of cards. For the non believers of this fact, why else would there be a certificate revocation list and root CA updates for windows periodically...

Using a processor who stores the card number outside of your infrastructure (ie. Stripe) can also be helpful.

Even better to use someone that isn't your payment processor, so that should you need to change payment processors you don't also have to re-acquire all the billing info from your customers. You can use Stripe today, PayPal tomorrow, and Braintree the next if that's what works best for your business.

Card vaulting as a service: https://spreedly.com/ ($10/mo for up to 5000 cards)

Isn't that just adding one more point of failure? I don't trust myself, I barely trust Stripe or Paypal, and I've not even heard of spreedly.

There's always going to be single points of failure, but which is more likely: you want or have to change payment processors (you've been terminated, your fees have gone up, you want to switch to a lower cost provider) or you want to change flat-rate vaulting services? Plus, Spreedly will give you your data if you leave, whereas there is no way to get stored billing info out of most payment processors.

Until they have a security breach.

Better rely on someone who's sole job is securing that info than doing it yourself.

Storing credit card info just helps make you a bigger target. If your a small company, better let someone else store card info, let them be the target.

Also you're fined by the credit card companies if you lose card information. I believe it's a per card fine, so it get expensive really quickly.

Actually I don't get why any company would choose to store credit card information, when most payment providers will do it for you.

Stripe is amazing... I trust them, someone hacks me, awesome, you got password hashes and stripe customer keys, all worthless.

Not exactly worthless, depending on the hack someone could still charge an awful lot to your customers and make you have a bad day.

But yes, significantly better than other situations.

True, but I'd rather fix the problem that got them in, force reset of passwords, and delete all customer keys and require them to create new ones than be like "uhhhh, our data was hacked and your credit card is safely encrypted... But we had the encryption key on the server too, oops"

But you still might need to consider data residency issues and making your customers aware of where their data is being stored.


But if that happens, it's not your responsibility (at least not 100%), it's theirs

Following the traditional responsibility/accountability dichotomy: They are responsible for storing the cc number securely but you are accountable when something goes wrong (because you gave them that task)

Much like Linode are responsible for hosting my clients site but I sigh am accountable when something goes wrong.

In what ways are wire payments better than using credit cards? In wire payments aren't you using the actual bank account numbers along with routing numbers which is also very sensitive information ?

Also I do not think, but I am not sure, that fraudulent wire payments/transfers are reversible.

They're not. You're right.

Wire transfers often are not able to be undone once they happen (and are accepted by the other bank). This is the reason why there's so much verification that happens in wire transfers. (I helped develop 2nd factor authentication used for authenticating wire transfers for a financial company)

Credit card charges can be reversed.


My Visa card that I used with Linode was stolen and used on an Amazon order I didn't authorise last week, my bank successfully blocked the charge. Someone else reported their Visa had also been compromised in the thread 2 days ago, looks like that confirms the suspicions: https://news.ycombinator.com/item?id=5542015

Poor show Linode. (edit: worth noting I use the card with other things too, I have no confirmation it was leaked through Linode other than the compromise happening at the same time these supposed leaks happened).

I have two accounts with them, one for my day job and one through my LLC. Right now neither card is showing unusual charges.

My day job had some Google Apps account compromises last month, and this is making me paranoid that the database that contained hashed/salted passwords for our Intranet hosted on Linode was the culprit. The time frame doesn't seem to line up, but we didn't see any evidence of phishing or compromised desktops being involved.

(Don't want to spread fear - I haven't been able to find any evidence our Linodes were compromised either.)

I'm normally huge a Linode evangelist, but I'm severely disappointed with the lack of transparency on this. I'm debating right now whether to rebuild all our nodes from scratch as I'm not sure they can be trusted.

I had a CC problem, and I use it for a very restricted number of online services. Luckily the fraudulent attempt was blocked by the CC company. Linode was not at the top of my list of suspects (there was another company that seemed to be storing passwords in cleartext), but now I'm wondering. This was a couple of months ago though... I wonder if that's the right time frame?

Switched to https://www.digitalocean.com/ last week. Excellent service and pricing.

Yeah, a week sounds like plenty of time to assess a hosting provider.

Oh, and the years of Linode assessment were adequate?

Every site can be hacked. It's just a matter of time. You just have to properly react: call your CC provider, check for any charges on your bill, and move on.

I do not think anyone here is actually worried about their funds; as mentioned below, any reputable provider will have such charges promptly reversed. The problem is instead with their response to the situation.

Linode has addressed the breach, but assured customers nothing of value had been compromised. This infers two thoughts. One: they knew of the breach and lied, thereby unveiling a unforthcoming and dishonest nature. Or two: they did not properly investigative the severity of the issue, thereby suggesting incompetence. Both equally reprehensible.

Mm, not equally. I'd rather have incompetence over malice.

Until they're hacked, too...

Or until they let other customers see all the data on VMs that you've shut down. Oh wait, that already happened: http://www.wired.com/wiredenterprise/2013/04/digitalocean/

DigitalOcean is new and they fixed the problem the same day the article was written:


If I had a choice between a VPS provider who either:

- Only has large issues (eg. leaks credit card data) and goes weeks without reporting them to customers, or

- Has lots of small issues (eg. forgetting to clean the free space of LVM volumes) but fixes them the same day,

I'd much prefer the latter.

EDIT: My bad, apparently the problem was reported on March 27 and wasn't fixed until April 2.

Wait a second.. you consider a provider giving data from your VMs to another random customer a SMALL issue?

Maybe you don't have anything of importance on your VMs, but plenty of people do. That data could contain credit card data, passwords, etc, etc, etc. It is very much a large issue.

Sure. My thoughts don't apply to everyone here, and I certainly can't claim to be unbiased since I like DO so much.

According to DigitalOcean, they stated that this impacts 3% of all machines, only the largest and most expensive servers. None of the smaller plans were leaking data.

I don't know how many credit card numbers were leaked from linode, but I'd guess more than 3%.

Second, if security is important to you, you can use 'dd' to clear the machine yourself before shutting it off. (In fact, good data destruction policies mandate the use of 'shred' et al anyway). On Linode, affected users don't even have a workaround (like this) to avoid information compromise.

There is no evidence to suggest that Linode leaked credit card data.

Personally I took precautionary measures and just called my bank to replace my credit card, which I think is the sane approach, as when it comes to hacking you have to assume the worst.

However, your statement on "has lots of small issues but fixes them the same day" is just stupidly childish. Linode's issues are bigger just because they are a bigger target.

Fuck it, I'm gonna recruit 7 friends and we'll set up our own VM cluster.

Unlike Linode, they accept PayPal.

I'm far more concerned about the integrity of my servers than about my CC, which has good fraud protection.

was not convinced by the website, looks nice but prices are kind of very cheap ?

also a twitter feed for the customers page ?

Great, now I am feeling paranoid although I don't see any unauthorized charges on my card. Does anyone know if debit cards are legally protected the same way as credit cards with 0% liability.

I'm pretty sure that depends on who issued your card.

Visa has a zero liability program for debit card - http://usa.visa.com/personal/security/visa_security_program/...

These are the FTC's rules [1], I'm not sure if Visa or Mastercard can make them 'better' (give you a larger window). They have an interesting tidbit below their chart -

>If someone makes unauthorized transactions with your debit card number, but your card is not lost, you are not liable for those transactions if you report them within 60 days of your statement being sent to you.

Isn't it free to get a new card? That'd be the easier way than worrying.

[1] http://www.consumer.ftc.gov/articles/0213-lost-or-stolen-cre...

Generally these days they are, but the problem is the money is removed is from your account by the time the charge appears, (at least a period of time), whereas on a credit card you have 30 days to review charges. This could cause overdrafts, etc. depending on timing and amount. Those too can generally be reversed, but the whole thing becomes more of a headache. I never use debit cards for any kind of recurring charges, and I avoid using them online in general.

The same is possible with a credit card - you can have your card maxed, get nailed with overage charges, declined transactions, etc. It's still a headache, but when I've had it happen to me I think I had the money restored within a few minutes of making the call.

Debit cards have less protection.

Wouldn't hurt just to ask your bank to re-authorise it anyway? It will change the three digits on the back.

they typically have the same protections, the only difference is that while credit cards only check that the charge can be made during the authorization period, debit cards lock up the bank's funds immediately which may cause your checking account to be temporarily unusable even if all the charges are successfully disputed.

a good practice that bankers constantly tell me is to have a separate credit card for online purchases for the fact alone that it is one step removed from your checking account.

They don't, protections are exactly the same.

They really aren't.

I know someone who got their debit card cloned. While the bank eventually repaid him, that did nothing to repay him the additional fees he owned his normal debtors (e.g. rent, utilities, etc).

With a credit card you aren't losing "actual" money. You are losing the bank's borrowed money which the bank pays back. With a debit card you're losing cash which you won't be able to replace yourself and which the bank might take days to weeks to replace.

Even if you NEED to borrow while your credit card is out of commission you can either use the overdraft facility on your debit card or other quick sources of credit. Hard to get quick cash without going to a pawn shop.

I would cancel that card right now. IMHO You should never ever ever ever use a debit card anywhere else other than the ATM. Credit cards give you way more financial protection.

Unless said ATM has been compromised :s

You should talk to your bank as it depends on the network your bank uses. Most of the time it's something like a 48 hour window to challenge charges but it's far less at some banks.

No, it's covered by the FDIC. Banks can offer better protections but at least 48 hours within discovery is mandatory. http://www.fdic.gov/consumers/consumer/news/cnfall09/debit_v...

From your link:

> [With a debit card,] "Until the bank provides provisional credit, you could temporarily be out of pocket for the amount in dispute," said Richard Foley, an FDIC attorney who specializes in consumer issues. "This would not typically happen with a credit card because consumers can withhold payment of the amount in dispute."

> Also, as discussed on the next page, consumers have better federal protections when they purchase faulty goods with credit cards.

They are. Protections are exactly the same (in the US at least.) You have protection from the moment that you learn of the problem, not when it happens.

Not that having your account drained doesn't suck, but your worst case scenario there isn't terrible unless you fail to check stuff and be responsible.

Nothing on my UK Mastercard.

FWIW, no charges on my card used with Linode at this point.

If this is true then all the trust that Linode has built up over the years was just thrown out the window. According to the hacker they've known for 2 weeks and made a deal with the hackers. Ultimately, they were as far from transparent as it gets and on top of that they did a horrible job with their security.

Hopefully, they own up and start being transparent.

If this is true then what alternative hosts should I look at, besides AWS?

Two alternatives often mentioned on here are DigitalOcean and RamNode.

I've only used DigitalOcean. My anecdotal experience from running a Chef Server on a 1GB instance has been pretty mixed. The price is good, but network and CPU performance feels very variable to me. A month ago their Amsterdam servers were unable to be resized, and there was nothing about it on their status page. I tweeted and was told they'd be working "some time later today". Doesn't fill me with much confidence in general.

I'd still choose Linode for anything of importance - their long reputation is well earned in my opinion. But, if this breach is true, I hope they handle it well.

>But, if this breach is true, I hope they handle it well.

That is unfortunately the problem though. If this is true, they have already handled it terribly as it has already been 2 weeks since the attack.

Yeah, I just migrated from Linode to Ramnode (just in time!) and I'm quite happy with the performance. Great network connectivity.

Yeap same experience here with DigitalOcean CPU performance. If CPU power is important to you, Linode's CPUs are a LOT better.

Don't be so logical please. This can happend to anyone in the industry.

"credit cards were encrypted, sadly both the private and public keys were stored on the webserver so that provides 0 additional security"

That's just poor security and 100% they're own fault. I accept that there are security issues with every platform, but basic security measures and being transparent is still expected. My biggest issue with them in all of this is not being transparent.

What are they supppose to say?

Looks like someone who likes attention on some random IRC channel who is apparently a hacker may have hacked our system and we don't know who/when/where/why/how or what they may have got. Nor are we sure we were even hacked???

It takes time for people to investigate stuff. It's not just a couple hours. Also some random guys words on IRC (who could very well own INSERT RANDOM HOSTING COMPANY for all we know and looking to scare people off Linode) should be taken with a grain of salt.

They are supposed to say that they would never ever store the CC numbers this way. Otherwise their customers (like me) have really no better option than to block their cards, which is quite an inconvenience. This is exactly the trouble I was hoping to avoid by not using a cheap VPS hosting.

If the information he offered is accurate (e.g. the public and private keys were stored together on the webserver), that wouldn't take a long time to confirm.

They could say "We have hired matasano security to help us investigate the breach."

The key thing (that "ryan" mentions not) is whether the private key was password-protected.

"Logical" is not a fancy synonym for "severe" or "unforgiving", and is probably not the word you wanted here. There are sometimes good, logical arguments that you can make for cutting people some slack.

I've been using digitalocean for a hobby project, and am planning on launching a more serious project with them. For the past two months I've used them (so, not much experience, but some) I haven't had any issues, and they are very reasonably priced.

I am using digitalocean, check them out.

Just like I can have application-specific passwords for my Google account, I wish I could have application-specific credit card numbers from my CC issuer.

If I had these, I would immediately cancel my Linode-specific CC# and reissue a new one. I would not have to worry that my other recurring bills will go unpaid, or spend hours dealing with tracking them down and changing them.

Bank of America provides this [1], as does Citibank [2] and likely others.

Paypal at one time provided this service as well, but it doesn't seem to anymore [3]

1: https://www.bankofamerica.com/privacy/accounts-cards/shopsaf...

2: https://www.citibank.com/us/cards/gen-content/messages/van/i...

3: https://www.paypal.com/va/webapps/mpp/security/general-freet...

Crazy, thanks for sharing. I use BofA and am interested, but here is the issue:

> Set your Valid through date for up to 1 year in the future

I'd really like it to be up to the expiration date of my card. It's probably worth doing anyway, I suppose. Thanks for the heads up. Given the zero-liability status, I'm surprised that banks don't promote this feature more visibly.

I've used the BoA and it's fantastic for one-time purchases. Like another commenter mentioned, it's only good for 1 year and you give a global cap. It would be nice if it was more permanent and allowed for a weekly or monthly cap so you could use it with subscription services or something like Amazon.

Nice. I wonder if any UK providers do this

My bank in Sweden does (Swedbank), so it's not a US-only thing. They even have an iPhone app for it.

The portuguese ATM network operator provides this for free (its called mbnet btw). You can even set expiry times and value limit, it's the best thing to use when paying for stuff online. Want to buy a 9€ game? Just create a 10€ card and use it.

Yup, I use this all the time. They even, recently, added support to multi-use cards with bigger expire dates.

I wonder how hard would it be to make a startup like this.

>I wish I could have application-specific credit card numbers from my CC issuer

Do some research on "virtual account numbers". I haven't used them myself, so I can't verify how well they work.

Visa have this (probably others too) that you can generate and either set a time or value limit on. Don't think you can set a monthly limit though unfortunately. They call it e-cards (in sweden at least).

Either this, or a assignable CVV codes. Something like that would be awesome.

That wouldn't work since CVV codes aren't sent with recurring transactions (they can't, since they cannot be stored).

Why can't they be stored?

Is it a legal requirement to prevent merchants without the CCV from using the credit card?

...oh wait

Because the CVV is used to indicate the "presence" of the customer at a transaction. CVV1 (which is on the magstripe) is used to indicate "card-present" physical transactions, CVV2 (printed on the back) is used for "customer just typed this in" non-physical transactions.

That makes much more sense. Thank you.

__should__ not be stored

Good point. Regardless, there should be __something__.

Remember when the Linode customer service portal was compromised which exposed everyone's VPS?: http://julianyap.com/2012/03/01/compromised-linode-vps.html

In that case, specific Bitcoin users were targeted.

It's pretty much why I don't trust Linode.

You can't trust a company which puts random AMI BIOS files on the main index directory on the main web site. You can't trust a company that can't even lock down their own Linode customer service portal (which could lead to a breach of each and every customer's VPS).

Perhaps history is fuzzy for people when new announcements come out or low prices are around.

Off topic but still relevant, but doesn't it seem a bit primitive that companies have to store you CC# for recurring payments? The one number that uniquely identifies your account and everyone you want to re-use it has to keep a copy. Couldn't the credit card company issue some unique ID to each vendor for recurrent payments? Ex. the vendor issues your CC# to the CC Company for charge and recurring process. The CC Co. responds by replying back with authorization and an ID unique to that vendor that says "Use this number for charging this customer again, but it will only work coming from you, so if you lose it, it can't be used elsewhere". The vendor then discards your real CC number.

> Off topic but still relevant, but doesn't it seem a bit primitive that companies have to store you CC# for recurring payments?

You really really don't have to. Any payment processor that isn't horribly incompetent does the unique token authorization scheme.

Storing CC #s for recurring payments is solely the domain of incompetents who have no business accepting payments from anyone.

What if you want to change processors? If you weren't storing the CC details, wouldn't you have to have customers enter all their details again? I imagine this could cause a drop in revenues due to people either forgetting, procrastinating, or just not bothering.

It's never cool to be actually- or quasi-locked into a vendor.

Most decent processors have processes in place for the transfer of CC numbers. I was involved with this process at a decent-sized magazine, it involved armed security, an encrypted hard drive in a locked container and millions of dollars of insurance. It's not an easy process, but is possible.

Sounds ... expensive.

Some will let you export the data, but it's a hassle. They burn it to a DVD and ship it to you and you have to sign tons of stuff freeing them of any liability.

Otherwise you just ask customers to re-authenticate. Often you have a few months headway for a switch like that.

Said processors allow you to export credit card data to another processor without having to store it yourself.

It seems there are a LOT of incompetents then. Are the payment processors you're talking about something the vendor interfaces with in the background and not a third party that is directly utilized by the customer (eg. Stripe, Paypal)?

This is basically what Stripe does. The "CC Company" basically doesn't offer more than a simple yes/no API -- "approved w/ #" or "declined". It'd be great if they could do more, but that's where the opportunity for folks like Stripe lies.

I guess it just surprises me that the CC companies don't do it themselves. Is the additional overhead of such a feature so high such as to warrant the choice between no payment processor (x% fee per transaction) vs. third party middle guy (x% + a bit more)? Not to mention the money the CC companies would save by not having so many fraudulent transactions (assuming they come out of their pocket, I honestly don't know).

Well, you can get AVS checks back too. But in general the entire system is so far behind the times. It really needs to be overhauled with security as the focal point.

It's amazing that it doesn't work that way. You could argue that it's because of the amount of infrastructure already in place, but why couldn't a new system be gradually and optionally rolled out?

Stripe (and probably others) has functionality like this where the seller's server never sees the CC number, and developers can store a unique token to re-charge the customer at a later date.

Depends on what part of the chain you are. Most gateways/processors offer some sort of token that the end user uses.

You then put your trust in the gateway/processor to store the credit card. Which I assume is most likely behind the best possible security stuff money can afford. Since that's their entire business. One screw up and their gone.

The majority of CC names have "Virtual CC numbers" which is precisely this - You generate a new number, which links to your account, but can only be used for the merchant you specify.

That's what Stripe do.

But isn't that just between Stripe and the company requesting payment?

e.g: Acme, Inc. sends Stripe your CC#, Stripe sends them some unique token, and they store that; correct?

So Stripe still has your CC#, and is at risk.

So this is really just risk mitigation; what I think TP is suggesting we need is unique authorizations at the banking level.

Something on the order of virtual credit cards, or temporary tokens, which are ultimately verified by your bank [or in other words: the lender(s) making anti-fraud guarantees, etc.]

(e.g: this token is authorized for 24 hours up to this limit; this token is authorized indefinitely up to $xx/mo.; this token is authorized for 1 year; etc.)

No. Customer sends Stripe their CC number, via AJAX in the browser. Acme, Inc. never has it even transiently. Stripe return a token to the browser, which is sent in a POST to Acme, Inc., then they verify it server side with a private API key.

Edit: yes Stripe has your number, but since their sole business is about securing that information, they probably do a better job of it than your typical online merchant.

I had a VPS on linode.

I think that Linode did a big mistake here. Let's wait for a formal communication.

But this is the moment to support them. Yes, maybe sounds crazy.

When you host on any third party datacenter, you take risks that something like this could happen. So, deal with it. Check your credit card, if your receive something wrong, call to your card and that's all. But we need to support also the good work, and this guys do great work in the hosting business. Just my opinion.

Well said. It's a fact of life that companies get hacked. So it's no surprise that it eventually happened to Linode. If you flee somewhere else, all you're doing is hoping that the other company you run to won't get hacked rather than using any logical thought.

I can think of two good reasons why you should flee Linode. It remains to be seen if either are actually true, and until indications say yes, then panic is unwarranted:

1. If it becomes apparent that Linode is far more vulnerable to hacking than other hosting providers. But one hack alone does not prove this.

2. If Linode grossly mishandles the situation. There have been a couple of allegations to that effect so far, but nothing substantial. I don't see any reason to claim that they've done this yet.

Linode has already grossly mishandled the situation by not coming out with a complete statement about what exactly happened. I only read this news because it was posed here -- no email notification, no update on their homepage, no twitter, no nothing.

The alleged hacker has made serious and specific claims, and Linode has done jack shit; without more information, how should I proceed? I don't want to call my bank and waste time getting a new credit card (not to mention replacing a million and two services) without a confirmation and I can't get a confirmation because Linodes people are having a circle jerk (or whatever the hell they do).

> I only read this news because it was posed here -- no email notification

There was an email notification a few days ago.

Linode already said they can't comment because of an ongoing investigation. What exactly do you want them to say?

> 2. If Linode grossly mishandles the situation. There have been a couple of allegations to that effect so far, but nothing substantial. I don't see any reason to claim that they've done this yet.

Linode's handling of the Bitcoin incident last year was sub-optimal. This too has been sub-optimal, given that credit cards were exposed but all we heard on Friday was to change our passwords, and even that was claimed to just be a super-careful precaution.

Linode needs to start giving us some frank talk ASAP. They've already burned through a very generous helping of benefit-of-the-doubt.

> This too has been sub-optimal, given that credit cards were exposed

If that's even true.

If this is true, they've blatantly lied thus far in their communications with their customers. I don't think that deserves support.

The service is not the problem, hiding what happened and the continued silence on these accusations is.

This is not a mistake/technical issue, it's becoming an ethics/service one.

So what happens now to all the goodwill Linode has amassed through the years? Does it all turn to shite, almost overnight?

This sounds very very bad, and as a customer it's very off-putting.

[Warning: imperfect analogy follows.] It's one thing if Linode is like someone who gets drunk and crashes their vehicle. That's 100% their fault and they've burned any goodwill. In this case, however, Linode is like someone who was carjacked. Perhaps Linode shouldn't have been driving that type of vehicle in an area known to have people attempting to carjack every single vehicle that drives by. Perhaps they should have installed thicker bullet-proof glass. Or even have taken measures not to trust any locks that the manufacturer insists are secure but have zero-day exploits. Regardless, Linode is still the victim of unscrupulous criminals. Maybe they could and should have done more but the bigger question is now that they've been carjacked, what are they doing to ensure that the carjackers haven't installed anything malicious that still remains in the vehicle?

I think a more accurate analogy is to say that Linode is moving your important files from one location to another in their moving van. They park at a 7-11 to run inside and grab a snack, leaving the van unlocked. An intruder comes along, opens the unlocked doors, makes a few copies of your files and leaves. Linode gets back in the van, notices the intrusion, does nothing except tell you that "you have nothing to worry about, but you may as well change your locks" and then when the truth comes to light, they basically stop returning your phone calls.

"No comment."

This analogy is more confusing than illuminating.

If the allegations are true, then Linode was keeping encrypted CC numbers, with the decryption key in nearly the same place.

Trying to make the analogy more sufficient by incorporating this type of fact would only make the carjacking analogy more absurd. At the end of the day, an analogy is not needed.

I think we should really wait to hear from Linode until we completely dismiss them, at least.

Seems like it usually takes an event like this for a company to really crack down on their security practices. I would wager that Linode will be one of the most secure providers in the coming months. Whether or not people will trust them is another story.

That's pretty much how trust works.

I'd give another trust vote to Linode, anyway this could happen to anyone.

It seems to me that the fundamental problem is not that they got hacked (although it seems that storing a decryption key in the same directory as the encrypted data is over the top careless), but their response to the disclosure that they got hacked. I realize that there may be limitations on exactly what they can say, but they should be as open as possible on what may have happened, what they are doing to protect their customers, and what their customers should do to protect themselves. Customers taking action when there wasn't a breach is less of a problem than not taking action when there was, in fact, a breach.

That's true -- it could indeed happen to anyone who keeps encrypted data adjacent to the keys.

Linode really needs to make a statement about what happened with this hack, stating if credit card information was taken. A lack of communication does not help me trust them. I'd rather have them speak up as to what happened and know if I need to have my CC reissued.

They may not know if CC info was taken or not.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact