
Ask HN: My VPS got hacked and now I'm facing a massive bill. What can I do? - Koekoeksklok
I&#x27;ve got a VPS which I use for small programming projects and college assignments. Two weeks ago I received an e-mail from my provider, stating that &quot;your VPS has been transmitting a lot of outgoing traffic which results in a very large traffic usage bill&quot;. In September on my 500 GB data-limit VPS, it had been transmitting 27 TB of data traffic. This resulted in a € 3300 extra charge on my € 15 VPS. I&#x27;m expecting a similar bill for this month.<p>Of course I immediately shut down my VPS after the notice two weeks ago, but by then it had been using these amounts of traffic for a month and a half.<p>What are my options here? I can&#x27;t afford to pay &gt; € 5000 unfortunately. Does anyone have similar experiences?
======
patio11
Incidentally, since many HNers probably come at this from a mental model of
"Anything which appears on an invoice is non-negotiable and simply must be
paid": a B2B service provider which collects payment after services are
rendered is _knowingly taking on credit risk_ and has already priced non-
collectability of some accounts into their services. You may be overestimating
how much drama is required for someone at their company to say "Wow, really?
OK, sorry about that. I'll write it off."

This is one of many, many, many reasons why we don't generally do cost-based
pricing and, when we do do cost-based pricing, the markup is absolutely
phenomenal. It has to include risk premiums. As long as it do include risk
premiums, you don't have to sweat the small stuff like e.g. an uncollectable
$4k invoice. (n.b. Small stuff! $4k hiccups are utterly routine events and
largely dealt with by processes rather than by treating them as sudden
emergencies, even if they feel like that to natural humans.)

~~~
readme
Also OP, if they don't write it off make sure to tell us all who it was so we
can avoid them.

The same thing happened to me with amazon. Amazon pid for it. It's highly
unreasonable in my opinion to ask the customer of a VPS to pay for damages
caused by a malicious attacker. It's tantamount to a landlord expecting you to
pay after an arsonist comes along and burns down your apartment, just because
you happened to be renting it at the time.

~~~
johnchristopher
May I propose another analogy:

\- a landlord* expecting you to pay after a squatter came along and opened a
faucet in the basement (where OP rarely goes) to fill its own super tanker

* or the water company ?

------
Blahah
1\. Report the incident to the police. Right now.

2\. Report it to the VPS provider. Explain that you've reported it to the
police. Ask for their cooperation in investigating the problem.

You do not have to pay. If they try to force you to pay, depending on your
country, you'll probably end up in small claims court where you'll find judges
are very reasonable people who usually side with the little guy. (IANAL)

~~~
donniezazen
I hope cyber crime police has improved their working in recent years. 5-6
years ago a Ebay seller defrauded me. I filed a complaint with all the
information I had since everything is online and involved bank transactions
there had to be a money trail. I never heard back from cyber crime cell. I
don't know how many folks were defrauded by that person before and after the
incident with me.

~~~
FireBeyond
Doubtful. I got a call from someone about a month after my GF's iPhone was
stolen. He'd bought it on eBay and asked if I'd got a new one yet - her phone
gave a number to call and was activation locked.

He wanted to know so maybe I'd give the password so "he wouldn't be ripped
off, too". I'll give him credit for having the balls to ask. But (as I later
found out) he knew he was buying an "activation locked" iPhone.

I said a few things like how do I know you're not the thief, etc. He pointed
me to the eBay listing which, sure enough even had the IMEI with two digits
transposed (plausible deniability, I'm sure).

He contacted the seller and said "Tell me why I shouldn't give this phone to
its rightful owner and then file a fraud complaint with eBay and get a
refund?" Unsurprisingly the seller offered to take the phone back. So he sent
it (I didn't care, and while I knew the insurance company was about as
unlikely to care as the police were, I didn't want to do anything that might
trigger insurance fraud questions - "This phone was reported stolen, unlocked
using your credentials. Explain.") and got me the seller's home address.

I contacted the insurance company. They didn't care, just told me to file a
police report and send them the case number.

Looked at the seller's profile, quite possibly the sketchiest thing I've ever
seen.

Bunches of phones, all "activation locked, no charger". Tablets, no charger.
Laptops, no accessories or charger. At least 50 or so.

Gave that info to my local PD. Their response, "We won't investigate. He
probably bought it from someone and is selling them. Could have gone through a
few people first."

I didn't want the original thief caught but this guy was openly selling stolen
gear. Hell, the message on my GFs phone said "I don't care about stealing the
phone. Will trade cash for it.".

They weren't interested. Bear in mind, this isn't someone complaining about
their car at the impound lot in LA, a la Big Lebowski, this is town of about
40,000 with a well-funded PD (I work for Fire in the same town).

The urge to drive to this jokers house in the middle of the night and pour
sugar in his gas tank was one I avoided, but only just.

------
BukhariH
I've been in the exact same situation with AWS (
[http://cl.ly/SHOu](http://cl.ly/SHOu) ).

It was a nerve recking couple of days but I contacted AWS support and they
were extremely good. They helped me secure my machine and then cancelled the
1.4K payment they were going to take from my account.

In all the whole process took 2.5 weeks and I only had to pay $15 for the I/O
requests.

The best thing I can recommend is to talk to your host and tell them honestly
you can't pay that much and you weren't the cause of the charges either.

~~~
rmc
In aws, you can set up billing alerts, so they will email you if you go over X
per month. It's a good idea to set that up, so at least you'll be alerted as
soon as possible if you get hacked.

~~~
banku_brougham
Hacked account => billing alarms get turned off. So you won't find out.

~~~
rmc
Yes, it's not foolproof, but you might get hacked by someone who forgets. It's
another lawer of protection.

------
thewhk
I work for a VPS provider in the US. These situations are common and we
usually just issue a credit and give a reminder to the customer to please
secure their server

That brings me to my point. How did the hack occur? When you get a VPS you are
fully responsible for what goes on in there. It is your responsibility to
secure it and keep it updated. It's not the provider's fault you did not apply
the latest security updates. It's not the provider's fault your Java
application was using outdated and vulnerable libraries nor is it their fault
you didn't set a CAPTCHA in front of your submission forms. Either hire a
competent sysadmin if you can't take care of that yourself or find a provider
that offeres managed hosting instead of a VPS, as that's what you'd most
likely need.

There are some cases where it's the provider's fault such as the Linode
BitCoin hack a few years back but mostly it's just poor server maintenance

~~~
waxjar
People that rent a $15/year VPS use it to run an IRC bouncer or a small web
log, something you don't need to know a whole lot of sysadmin stuff for. They
just need a machine that's always on.

It's hardly worth hiring a sysadmin for (I find that suggestion laughable, to
be frank). Managed hosting doesn't allow you to do much else besides hosting a
website in PHP, which is not enough for plenty of use-cases, including OPs.

~~~
lucb1e
> $15/year VPS

Do tell, where do I get one of those? Cheapest I know of is $60 ($5 a month).

~~~
travoltaj
Here's one that I know of:
[http://ramnode.com/vps.php](http://ramnode.com/vps.php)

Other than that, I'm sure you can find something on webhostingtalk forums.

------
theonemind
I work for a company that provides VPSes. In a situation like this, they can
see the usage is aberrant and they can see it's not normal based on past
bills. They'd likely offer a large credit if you say you didn't intend to do
this, and it doesn't look like a fraudulent account. That being said, they
themselves probably have bandwidth costs, and are not at all likely to forget
all of the charge, perhaps half at best.

------
onestone
Stop using providers which charge a ridiculous price for bandwidth (like AWS).
There are many excellent alternatives where a TB costs only a few
dollars/euros.

~~~
njsubedi
like?

~~~
notok22
Hetzner has the first 20 TB free and then charges 2 euros per TB.

~~~
JosephRedfern
Beware:
[https://news.ycombinator.com/item?id=6577465](https://news.ycombinator.com/item?id=6577465)

~~~
Nyr
Any sane provider with no DDoS protection will nullroute you on incoming DDoS,
that's not Hetzner specific.

If you expect to get DDoSed, buy protection or go with OVH.

~~~
vacri
The problem wasn't the nullrouting, it was 'contact support to re-enable...
and support isn't open until Monday'.

------
patio11
I would begin by contacting your VPS provider, explaining the circumstances
which caused the bill, and asking "What are our options?"

------
ColinCera
Have you talked to your VPS provider? They should be able to cut you a break;
after all, that 40TB of traffic cost them only a small fraction of what
they're charging you, so if they're reasonable you should at least be able to
get them to reduce the charges to their actual cost.

You might also offer to suggest writing up a post mortem for them, that they
can provide to their customers as a lesson/tutorial on how to protect a VPS.

Finally, you can suggest that they might want to implement (and perhaps help
them implement it) some kind of warning system, i.e., if a VPS suddenly begins
using exorbitant amounts of bandwidth, and far more bandwidth than it ever has
before, they really should email/text the owner an alert within 24 hours — not
let it go on for 6 weeks. I'm surprised that they don't cap/throttle the
bandwidth once you go over your plan's limit, to go along with sending you
alerts. It borders on negligence on their part that they don't already have
such a system in place.

~~~
MangoDiesel
In my opinion, it is negligence to an extent that OP should not have to pay
for this, and he should find a new VPS provider.

~~~
Khaine
Why? He failed to secure his server. Why is that the fault of the VPS
Provider?

------
Jare
Depends on your provider. Amazon AWS is known to have waived such bills in the
past, see for example [http://readwrite.com/2014/04/15/amazon-web-services-
hack-bit...](http://readwrite.com/2014/04/15/amazon-web-services-hack-bitcoin-
miners-github)

------
zhovner
To prevent such incidents Linode have alerts of traffic/cpu/disk thresholds.
For example you can configure notification if your bandwidth utilization more
than N Mbit/s in duration more than N minutes. Very useful for DDoS
prevention.

~~~
bluedino
And look at your dashboards once in a while. I'd find it unusual if I saw my
toy VPS cranking out 100mbs for a week straight!

------
dangoldin
I had something similar happen with AWS but the bill wasn't as high since they
ended up flagging my box as spam-producing and shut off all outbound traffic.
I'd just ask them and see if they can remove the charges, it worked in my
case.

------
matthewarkin
I had the same thing happen to me. I wrote about it on my blog
[http://mattarkin.com/protect-your-azure-linux-vm-aka-how-
to-...](http://mattarkin.com/protect-your-azure-linux-vm-aka-how-to-
avoid-a-1500-charge/). Basically I complained to Microsoft, they said they'd
waive the charge but since it was for a linux vm they said they couldn't cover
it. Then I complained to American Express claiming it was an unauthorized and
fraudulent charge. Amex sent the dispute to Microsoft and they never responded
so I wont the chargeback.

------
minopret
I can understand how that could happen and what a problem it would be. I had
an experience with a telephone bill myself, but the story is not going to help
you.

I would suppose your first and best resort is to consult your lawyer,
advocate, solicitor, barrister, Anwalt. I wonder what your relevant legal
jurisdiction is.

I wonder whether it would help if you can account for your own whereabouts and
your own usage of endpoint data services. I wonder if your method of payment
to your VPS provider is mediated by a financial service that can help you
dispute the bill.

I am not a lawyer.

------
jnardiello
I assume you are in europe. I'd suggest simply talking with your provider,
explaining the issue and asking them to investigate. I honestly expect them to
cooperate and be understanding.

If they insist for you to pay: simply don't. State the truth: You can't afford
it. Tell them the only way they will see this money is by taking legal action
against you and even in that case you won't be able to comply - as you don't
have the money.

Hope it helps :(

------
freshflowers
Just in addition to some other helpful comments: based on posting I assume
that your are Dutch or Belgian, located in Europe and are buying this VPS as a
private consumer, not a company.

Which means your case is probably covered by consumer protection rules when it
comes to informing you about data usage, and I seriously doubt a VPS provider
has covered their ass as well as mobile providers tend to do.

------
tdicola
Anyone have tips on how to secure their Linux VPS? I just set one up and
disabled SSH password login, locked down all the ports with iptables (using
ufw), and enabled fail2ban. Anything else I should install or configure to
make myself a little more secure? Was considering tripwire but I dunno how
much a headache it would be with false positives as I change things on the
server.

~~~
MayIHaveAnother
A very common attack vector is through installed web applications. Especially
if you run wordpress with a lot of plugins installed, be sure to enable
correct read/write settings for /var/www, and update your application
frequently.

Malicious entities runs 24/7 scans towards indexed URL's attempting to exploit
various vulnerabilities, and many of the vulnerabilities allows remote code
execution, upload of php files etc. This can be used to upload malicious code,
simple php-webshells, and then your VPS is suddenly a part of a DDoS/Scanning
network.

Exploited Wordpress sites are a problem, Zeus/Zbot-Trojan is often seen
downloading updates/configs from these, and they are also often used to
redirect users to Exploit Kits.

~~~
porker
I came across these two tools recently that seem interesting:
[http://www.rfxn.com/projects/linux-malware-
detect/](http://www.rfxn.com/projects/linux-malware-detect/)
[https://github.com/emposha/PHP-Shell-
Detector](https://github.com/emposha/PHP-Shell-Detector)

Not installed either yet (LMD could really use some .deb packages) but could
be a useful alternative to Tripwire

------
applecore
PSA: Set up billing alerts! You should always have a notification sent to you
when your monthly bill exceeds one or more dollar amounts. For example, if
you're using AWS, Amazon CloudWatch lets you set an alarm on a billing metric
to notify you automatically.

------
joshmn
Post on WebHostingTalk.com - just do it. You'll get attention from the host,
other hosts who will sympathize, and you'll see that they'll just write it
off.

Post the link when you do and I'll be sure to comment on it (I'm somewhat
very-active at WHT)

------
decisiveness
I seem to be missing something. You knew it was happening when you got the
first bill, but let it continue for another half month before shutting it
down?

------
logn
In addition to the other comments, make it absolutely clear to them (with
proof if needed) that you're a student.

------
gregcmartin
Make sure Elastic Search is not accessible from a public IP address (this is
what likely got you in the mess to begin)

------
general_failure
Do you know how you got hacked?

------
zack19
cut your credit card report it as stolen tell the host that it wasn't u :p

------
ishener
I feel really sorry for you situation. I first suggest talking to the hosting
provider and explain what happened. Any decent service will give some discount
in this case.

Unfortunately, I can't think of anything else. I wish it was realistic to tell
you to go to the police.

Also, if you would give your email, I would definitely consider sending a
donation through paypal... Hopefully other readers here will do the same.

~~~
patio11
There is one _great_ reason to go to the police: it establishes a paper trail
documenting that a crime was committed. It isn't necessary that the police
catch the bad guys for that paper trail to be advantageous.

(Examples: police reports make CC disputes and legal declarations much easier
and more likely to be given weight as other than self-serving explanations of
a deadbeat. It may also trigger insurance policies either for you or for the
VPS company.)

