Hacker News new | comments | show | ask | jobs | submit login

My logs show increased break-in attempts after moving to Digital Ocean. Their IPs are hot targets for hackers, making it all the more important to properly secure my droplets, something I'm having a hard time doing in Linux. With Linux, it's lot of beating around the bush. Overcomplicated config files, a lot of disabling and removing of things one don't need, having to deal with a messy and outdated firewall. With OpenBSD, lo and behold I'd have one of the simplest, most secure and well-configured droplets the world has ever seen.



Yeah noticed that as well. First thing I do is (on debian), update packages and the kernel, install ufw (allow only 22), fail2ban, postfix and logwatch. Then setup a non privileged user, sudo, turn of root ssh and password auth and move to key based auth.

Then I can sleep/configure the rest of the system.

Been meaning to package this all up in an ansible playbook but I can't be bothered :(

I'd be using FreeBSD if they offered it. Much like OpenBSD it's a lot more pleasant for sysadmins!


It took me about an hour to put all of this into an ansible playbook. I think there are couple of examples out on blogs about this as well. If there is interest I'll share my playbook.

Writing this playbook paid off already for the second installation of the box.


The most I know about securing a Linux vps I picked up from this: http://feross.org/how-to-setup-your-linode/ . If there is an ultimate resource on how to start securing a *nix vps then please do let me know. If I knew what to search for I could but a lot of this is unknown unknowns to me.


After having fixed a sudo user (and confirm it works!), please also disable PermitRootLogin by setting it to 'no'.

Restart ssh after editing sshd_conf.

If you're squeemish, run dropbear on another port so you can log into your machine in case you made a boo-boo configuring openssh.

Also, more resources for security settings:

https://benchmarks.cisecurity.org/downloads/multiform/index....


I remembered correctly, someone had already done that: http://lattejed.com/first-five-and-a-half-minutes-on-a-serve...

Shameless plug: a small side project I did with my team at nodeSWAT [1]: CMify [2] - a web service to easily generate your ansible playbooks. Somewhat naive so far, if there is interest we'll extend it.

[1] http://nodeswat.com [2] https://cmify.com


Don't mind me, I'll just wait here till you're ready.


Please do.


I'd like a copy of that playbook too :-), publish it somewhere?


Please!

Give us your address so we can send you money.


ditto


yes please


> update packages and the kernel

PSA: If you just update the kernel using system tools within the instance, it'll still boot the old one. (At least it did few months ago.)

The grub (or other bootloader) and kernels that are installed inside the instance's disk image are silently ignored and the vm uses a kernel stored outside of your instance's image. You can select which kernel image you want to boot in the control panel. The problem is that the kernels available in there are updated very, very rarely. For popular distributions like Ubuntu they can lag several months, for others it can get up to, say, _two years_ [1].

Since digitalocean "doesn't support custom kernels" it appears that the only way to use something up to date (and stay secure) is to write a script that loads a custom kernel using kexec. :S

edit: The fact that they use kvm (and that kexec works) which means real visualization but can't boot user's kernels is just very, very weird. The only technical reason i can think of is that they can't manage to make it work with their control panel. If that's the reason, then it's very worrying. (The "right" way is easier to code and it's the solution that you'd think of first and it makes custom kernels a non-problem, but they picked one that both has obvious problems and is harder to setup, i can't think of a reason why.)

[1]: https://www.digitalocean.com/community/questions/how-can-i-b...


Second shameless plug on this page: if running your own linux kernel or BSD is really important to you, http://prgmr.com can do that and we have out of band console access so you can debug why it's not working (I was having problems with that when I was checking whether some linux+xen bugs were also present on ec2.)

At this time we only support xen paravirtualization(PV) though, which almost, but not quite, all major distributions have. Notably we can't run 64-bit freebsd right now because they only have HVM support. I'd like to change that but it needs more testing first. Here is a partial list of distributions supporting xen: http://wiki.xen.org/wiki/DomU_Support_for_Xen Really any linux distribution with a modern kernel can be made to support it though as support has been in mainline for a long time now.


Actually, I really would love to use prgmr. The thing is, the price point is super far from Digital Ocean. Compare https://www.digitalocean.com/pricing/ to http://prgmr.com/xen/plans.html . The difference is a bit of a joke. If you're able to provide competitive pricing, I'd be thrilled though.


Yeah, I'm working on it. I announced upgrades to some of my customers several years back, and failed to deliver many of said upgrades, so I'm not going to give hard dates... but I recognize that competitive pricing is a prerequisite to the survival of the company, and I'm working towards that end.

As is industry standard, I'm upgrading existing customers (e.g. they stay on the same price plan, but they get more resources) before I offer those new price plans to the general public. I have maybe... 1/3rd of my customers upgraded to the plan I promised several years back (that has more disk and less ram than the plan I am upgrading to now.)

A big part of the problem is that I made a huge business mistake a year ago that cost me most of the money I would have used for upgrades. But, I've got a few options on the table, one of which is just consulting for a while;

if I go with used hardware, I'm within $50K of having enough hardware to upgrade everyone to "competitive, but not great" pricing. - that's maybe 3 months of full-time contracting work. Completely reasonable. Of course, used hardware is a lot like 'technical debt' - pay me now or pay me later, but it might make sense, just to get the monkey off my back. (That, and dealing with hardware problems is part of my core skillset; I can probably eek more reliability out of used garbage than most companies can.)

I'm talking 2-3x that for new hardware, which is a lot less realistic without some sort of loan or lease, something I should look into, but eh. I am thinking that used hardware until I'm competitive, then start buying new hardware once I start getting new customers on board might be the best way to go. The company is vastly easier to run when it's slowly growing rather than slowly shrinking.


I dug into this a while ago; I recall (but can't source at the moment) an eventual statement by the DO guys to the effect of:

1. their original "inject kernel into OS and boot from it that way" architecture was designed that way on purpose, so that users could downgrade kernels out-of-band as a way to rescue a bad upgrade;

2. but now they've got thousands of droplets configured that way, so switching architectures to something sensible will require either A. forcibly installing grub on, and restarting, every one of their nodes (not something people expect out of a VPS provider), or B. coming up with some sort of glue that can manage both the nodes expecting an injected kernel, and nodes that want to boot on their own, and providing a way to transition your nodes from one to the other.

Basically, it's a mess. A bit like the problems Heroku had with its Alpine stack.


It does still boot the old one. You have to power it off, change the kernel and power it up again.


Moving your SSH server away from port 22 and disabling password logins is also something you should consider doing.

Small thing but at least then it doesn't shout "Hey I'm a *nix box with remote login enabled"


Is it actually likely that someone would brute force a password? I always install fail2ban, but it's more just to stop bots from filling up my logs with junk and eating my bandwidth/CPU. Unless you use dumb passwords, the odds are phenomenally low.


I'm not sure, but I've come around to realize that it's always worth installing and configuring fail2ban just for the reasons you mentioned, if nothing else. I have a demo server that, for a while, was being hammered so hard by ssh brute-force bots that it kept getting knocked offline. I installed fail2ban and no more problems since. I personally can't recommend fail2ban highly enough.


fail2ban is based on the horribly flawed premise of correctly parsing arbitrary text logs from tons of different programs that wrote their log format without any concerns about parsing that output later. Not surprisingly there have been numerous vulnerabilities in fail2ban that let an attacker ban arbitrary hosts such as your DNS server, database, etc.

https://www.google.com/search?q=fail2ban+dos+bug

A better approach is something like pam_abl which is a pam module that will accomplish mostly the same thing but only for login attempts and without the crappy plain text log parsing.


Have you tried denyhosts?

I've always been installing denyhosts but I have not compared the two.


I'm amazed noone ever mentions GRSecurity as must have. It will reduce the risks of an actual comprise significantly. Non-patched Linux is always full of rootkit potential.


Well they don't issue GRSecurity kernels.

To be fair I'd just use FreeBSD which has a little better kernel architecture with respect to security.



Awesome, please do!

Don't forget https://github.com/kickstarter/rack-attack as well though.


Just install ufw. It's very simple to configure the rules for a basic server. Here's how: https://www.digitalocean.com/community/articles/how-to-setup...


Why would you use that over iptables? It's simple and doesn't require installing a package.

  > There is a lot of functionality built into these
  > utilities, iptables being the most popular nowadays, but
  > they require a decent effort on behalf of the user to
  > learn and understand them.
Is that a corner worth cutting?


Iptables is a crime against humanity from a config and documentation perspective. ufw makes it bearable.

I'd rather use pf on BSD though.


I agree. iptables is horrid.


It's just a nicer interface to iptables. Instead of changing my iptables config file you can do things like 'sudo ufw allow 22' or 'sudo ufw allow http' for example.

You should still understand iptables but you do not need to config everything manually.


iptables is powerful but definitely not simple.

Simple is being able to simply say "allow all outgoing traffic and incoming traffic should only be allowed for HTTP(S) and SSH" and being able to figure out how to do it by just invoking "ufw --help".

Maybe someday I'll learn about iptables, I'm sure it's going to be worth it, but for now ufw does the job for me.


> iptables is powerful but definitely not simple.

> Maybe someday I'll learn about iptables

Hmm...

    iptables -A INPUT -p tcp --dport 22 -j ALLOW
    iptables -A INPUT -p tcp --dport 80 -j ALLOW
    iptables -A INPUT -p tcp --dport 443 -j ALLOW
    iptables -A INPUT -p icmp -j ALLOW
    iptables -A INPUT -i eth0 -j DROP
So complicated...


> So complicated...

Indeed, it is. Even if you want to cargo cult that without understanding it, you might get bitten because running those commands again will not do what you expect, since they're not idempotent.

You will now reply telling me how to deal with this situation, for example if I want to now listen on a different port, or how I get FTP (or some other protocol that needs "-m state" to work. The need to do this proves that using iptables is more complicated that your example.


I've not played with iptables much, but when I do, I'm always struck by how easy it is to read rules, yet hard to write from scratch. This being said, a simple config like this is trivially googleable.


ufw is just a simple wrapper for iptables, it still uses iptables.


"ferm" is also a nice readable wrapper for iptables.


I think that's a little unfair... Your server needs to be hardened against drive-by SSH password guessing and similar nonsense regardless of how or where it's connected to the internet.


That is true but as per the poster above, I've seen 10x the attack traffic on a DO IP address so any zero-days and you're already herded into a predictable net block ready to be poked.


This is true for pretty much any VPS-only hoster, they all have well defined ranges and tons of targets to attack. I see thousands of attempts a day at Linode and hundreds at Leaseweb (dedi and VPSes) and DO. It's just part of doing business.


It's certainly true for dedicated-only hosters too.


I guess I should have said 'personal server', not like AWS or Google's Cloud platforms.


Do:

    sudo ufw allow proto tcp from any to any port 22
    sudo ufw enable
Then you can just edit /etc/ssh/sshd_config the same as you would on OpenBSD.


Or:

  sudo ufw limit ssh
  sudo ufw enable
To automatically rate limit connections -- see https://wiki.archlinux.org/index.php/Uncomplicated_Firewall#...


This is why I love having a static IP.

    sudo ufw allow from your.ip.addr to tcp port 22
Rest of the world doesnt know I have ssh up and if I'm ever in a hotel or something I can tunnel through my home or login to DO's panel and allow another ip


Don't do that. I did that once and our static ip changed suddenly despite lots of money being spent on it. That was a painful day driving from London to Manchester for me to get to the console.


There is a web console, you never have to drive... or worry :)


I could imagine this being quite dangerous, like being on holiday, there's a power outage, fire, ISP error, etc at home and no way to control your server.


I have essentially a static IP (A 'dynamic' IP with a 6month lease that stays the same on renewal and follows my account regardless of where I live), but you're super fucked if for some reason your IP changes. I usually set it to the /24 I'm assigned from.


How are you screwed? You can always login to the console in the web admin and change it




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: