This is my first time putting a guide like this together. I see a lot of really great feedback and I will be incorporating them into future updates.
I don't have time today but I will reply individually to all the comments that I can.
I appreciate any/all feedback/advice. If possible, could future issues be submitted to GitHub because it is easier for me to manage if everything is in one place. https://github.com/imthenachoman/How-To-Secure-A-Linux-Serve...
But right now I need to find all the other places this guide is linked to with comments so I can address them too. :)
There are two fresh threads on Reddit:
And yes, I saw that pull request but I didn't know what HN was at that time. After I saw that comment, and before they said HN = Hacker News, I started Googling around to see if I could find where my repo was being mentioned. That is how I found this post in HN.
"One key, the public key, can only encrypt data, not decrypt it" - this is cryptographically inaccurate. One should use it that way, though.
"Identity is verified by encrypting and decrypting data that both the client and server know". That's not how signing works. The crucial part of the process is to first establish an encrypted channel, then choose something random (not used previously) and finally verify the signature is correct. What exactly would be the data that "both the client and server know"?
"If you're sure there is nobody listening between the client you're on and your server, you can use ssh-copy-id to transfer and append the public key". Almost never happens unless you're in the server room, connected point-to-point. But then the remark about someone listening doesn't make sense. And not a single word about verifying server's fingerprint in the whole guide.
"Keep in mind doing this means you can't use the key for automation because you'll have no way to send the passphrase in your scripts". Unless you use an ssh-agent.
"umask is a Bash built-in which means a user can change their own umask setting". Wrong implication here. umask is a syscall. It doesn't matter what you use to call it. And what if you're using a different shell?
Suggesting painful default umask instead of just doing `chmod go-rwx $HOME` and adjusting /etc/adduser.conf is debatable.
Making this guide "distribution agnostic" is IMHO futile and there already are some distribution-specific guides like https://debian-handbook.info/.
CentOS: https://wiki.centos.org/HowTos/OS_Protection (limited)
Debian: https://www.debian.org/doc/manuals/securing-debian-howto/ind... (old)
Fedora: https://docs.fedoraproject.org/en-US/Fedora/19/html/Security... (old)
Mageia: https://wiki.mageia.org/en/Msec (limited)
Oracle Linux: https://docs.oracle.com/cd/E52668_01/E54670/html/index.html
Red Hat: https://access.redhat.com/documentation/en-us/red_hat_enterp...
Slackware: https://docs.slackware.com/howtos:security:start (limited)
Ubuntu: https://help.ubuntu.com/lts/serverguide/security.html.en (limited)
Upgrade the version of the browser to an approved version by obtaining software from the vendor or other trusted source. Method 1: View the following registry key: HKLM\Software\Mozilla\Mozilla Firefox\CurrentVersion Method 2: Search for the firefox.exe file using the search feature of the operating system. Examine the files properties for the product version (not the file version. For Windows OS, determine the version of the file by examining navigating to Properties/Version/Product Version. Examine for all instances of firefox.exe that are present on the endpoint. Criteria: If the version number of the firefox.exe file is less than 50.1.x (or ESR 45.7.x), this is a finding.
How about clicking on "About Firefox" in the menu? "Other trusted source" - like softonic, yes?
STIG itself is just a fancy check-list. Ideally, real world implementation is automated via something like OpenSCAP.
Perhaps they don’t trust it enough to execute it until they know it’s the latest version and was obtained from a trustworthy source.
No, one shouldn’t. To be even more pedantic, verifying a signature under PKI is, in fact, “decrypting with the public key” something (a hash, usually) that was encrypted with the private key. When considering the complete set of PKI operations (encryption + signing), both the public and private parts of a key pair are used in both their enciphering and deciphering capacities.
> Now you need to append the contents of the public key ~/.ssh/id_ed25519.pub to the ~/.ssh/authorized_keys file on the target server. You'll want to do this in a secure way since the public key gives access to your server.
I simplified things that would be too heavy to get into. The guide is not intended to teach everything about security -- that would be too much. The goal is to cover enough to give the reader a high level understanding. Once they have a basic understanding they can research more if they desire.
Can you tell me some of the "exotic details where defaults would be sane enough"? I can amend the guide.
Are you saying a public key can be used to decrypt data it encrypted? Or are you saying a public key could also be used to decrypt data that the private key encrypted?
You mention "server room". This guide is intended for a person running a simple server in their home. Hopefully SAs securing a large scale environment are not using information from GitHub. :/
I made some other updates to the guides that I hope address your other concerns?
Regarding your distribution agnostic comment, I do not see value in distribution specific guides on hardening. Sans a few edge distributions, most distributions are similar enough that the hardening steps are the same. It is okay to look for distribution specific documentation on how to install the distribution but it hurts the cause having distribution specific hardening guides.
Also, I want more folks to use Linux. Most distributions are so similar there is no value in having so many distribution specific guides -- all it does is create unnecessary confusion and steer potential prospective users away from Linux.
On the other hand, the objectives list contains: "this guide will attempt to cover as many of them as possible". Despite that, it still misses basic security rules.
"A desktop class computer [...] That I want to be able to SSH to remotely from unknown computers and unknown locations (i.e. a friend's house)."
SSH from an unknown computer? Also see my original comment about what is missing in description of a situation when connecting to your server from a new system.
> Are you saying a public key can be used to decrypt data it encrypted? Or are you saying a public key could also be used to decrypt data that the private key encrypted?
See my discussion with derefr in this thread.
> You mention "server room". This guide is intended for a person running a simple server in their home.
The "server room" here means you are sitting next to your server and can connect point-to-point. It doesn't mean you need a backup diesel generator.
And if it is really about a home server, concerns like "make sure nobody is listening" don't make sense.
> I made some other updates to the guides that I hope address your other concerns?
I've seen you accepted pull requests of some contributors that fixed the basics for you. Because they were generated right after discussion in this thread they concern things discussed here.
> Also, I want more folks to use Linux. Most distributions are so similar there is no value in having so many distribution specific guides -- all it does is create unnecessary confusion and steer potential prospective users away from Linux.
How many (which) distros have you managed as a professional sysadmin to state that with high confidence?
I'm quite sure that a beginner will just choose Ubuntu and proceed to something like https://linuxjourney.com/. If someone becomes more than a hobbyist one will hopefully go and read something from a trusted source, written by sysadmins with a good grasp of contemporary cryptography involved.
Thanks for that link -- never seen it before.
I was an SA for a Fortune 10 company for 5+ years supporting 150k+ servers but we don't need to get into comparing resumes here. I appreciate your time and don't want to waste more of it. However, if would like to continue the discussion maybe we can do it on GitHub? I don't care for HNs commenting style. Thanks!
If you have homegrown security and can show your QSA your detailed policy document and that it's a superset of CIS, STIG, NIST, etc. with documented exceptions then it'll be no problem.
I avoid homegrown whenever possible because it's a rabbit hole that never ends. If you instead say CIS level 2 then you can clearly define when you've done enough.
Something a lot of hardening guides seem to skip!
Which is to say: So you've shipped logs off, so then what? How are you going to monitor those regularly, what are you looking for, how are you going to make sure important information stands out?
Many people set up remote logging and then never check the logs until after there is an issue. An unread log isn't useful. Logs that are too spammy aren't going to be read.
There have been TOTP PAM modules for ages (these work on a Linux client and Linux server via e.g. SSH). You can even add YubiKey to PAM. Same for BSD Auth and macOS.
It's abandonware these days and hasn't been updated for years. The iOS app doesn't work anymore and can't use the camera for reading codes.
It's not broken for the 15 or so accounts I use it for.
Reading / using existing codes might work properly, or manually inputting them.
I've sent a note to the maintainers, but there's an open git issue for it. Not sure when their commits will roll to the store.
I'm told by multiple colleagues at redhat that they've raised this with internal support, so it should be sorted eventually. Lots of iphones in redhat and this is the app they officially support for TFA internally.
I think for home use, ufw is probably good enough. I've been using it for 3+ years and it's worked out okay for me okay.
I have not heard of ferm but I will check it out. Thanks!
For a home server ufw is probably good enough, as I said I wouldn't recommend it for "serious" use in a highly automated environment.
Correct me if I'm wrong, but isn't this what password-protected private key encryption is?
2FA would be private key (password-protected) and a separate (most likely one time use) password. Something you have (key that you decrypted) and something you know (the one-time password).
Keys are easily (and persistently) added to ssh-agent, at which point it is easy to forward and will generally silently authorize without any further user interaction. That reduces it to a single factor.
Compare that to a totp challenge or a yubi key plus a password. Having one of them won’t get you the other.
Another flip in the wall of rowhammer defenses (IEEE S&P 2018)
And scan your own network, both from the outside and inside.
Can you recommend any good network scanning tools? I've been on the hunt for a good one.
nmap is a popular tool for network scanning. I've also found tcpdump to be useful for looking at network traffic.
setuid and chroot are useful for programmers, so once the app is up and running, it can chroot into a data-dir and drop root privileges using setuid to a unprivileged user. As a sysadmin you can also start the app from within a chroot and run it as a
unprivileged user which is preferably. Most (free)BSD tutorials go though setting up a chroot jail, it's not as common in Linux.
Containers, VM's, and chroot will not stop a very determined attacker, but the more restrictions the harder it will get.
Security is applied in layers: First you want to prevent people from the outside. Access is most often gained through exploiting some service/daemon/app running on the server. So the app should have as little privileges as possible.
System access is often then gained by exploiting another app (so you want all apps to be locked down, not just the network facing ones), like getting Apache to run curl, that sends a internal request to another app that has a known vulnerability and happens to run as root. (nobody thought that app not accessible from the outside needed to get patched).
Once the attacker is inside the server, you also want to prevent access to other computers in the network.
Yes but first and foremost you don't expose ssh to all internet, it should be allowed only from known IP addresses and you should VPN to have connect to that known addresses. Section about firewall config just tells to open ssh...
I've been running my home server with an completely open SSH port since 2012 and haven't been hacked yet. And I know since I check occasionally with Tripwire and/or RPM verification from a clean boot.
The only downside to an open SSH port is the thousands and thousands of log spams from connection attempts. The upside is that I can log in from anywhere using my phone directly or as a hotspot for the laptop.
Oh yeah, and VPN just pushes the problem back one level. For me SSH is the VPN. Requiring VPN access first to get to SSH is just pushing any security problems back into the VPN server. Because now IT is the one with the open port to the internet.
RPM verification checks that all of the files installed through RPM have checksums that match the original RPM. It also checks that the RPM cryptographic signatures match.
So that should guarantee that files like the kernel, systemd, /bin/sh, /lib/libc.so.6, etc are not compromised.
A system can still be vulnerable to persistent attacks installed in unwatched files such as /root/.bash_profile, /etc/profile.d, extra files in /usr/systemd/system, etc. So you also have to check for extra files that you didn't install.
I don't have anything except Secure Boot to protect against UEFI attacks.
For work related stuff if I have to deal with multiple people accessing multiple servers then it is different story. Strictly forbidding ssh to have people connect from one IP is a lot more control. Then also key auth but I don't have to setup some google auth.
Ultimately though, many would argue that’s just security through obscurity so you’d still want something like fail2ban or denyhosts running - namely a tool that monitors your log files for failed log in attempts (or other suspicious activity) and then auto blacklist that IP in your filewall. I’d personally recommend fail2ban over denyhosts for a variety of reasons but ultimately either is better than none.
What I also like to do is have the public SSH box a bastion server with its own unique credentials so you effectively have the same multi-tiered authentication as you would for VPN.
You can also add MFA for SSH too if you wanted. In fact there are a few PAM modules you can use; from captchya’s (to reduce bot effectiveness) to Google Authenticator.
Even enhanced SSH tools like the file server protocol (SFTP / scp / etc) and port forwarding can be enabled or disabled for specific logins / groups or everybody if you wanted.
So it’s entirely possible to harden SSH in the same way you would VPN.
If you're using public/private keys you can use the "from" option for the keys. But it's not fool proof.
If you want to get clever then you can enable port knocking but my personal preference is just good old fashioned whitelist of IPs with fail2ban running ready to auto-blacklist any of those IPs that have too many failed login attempts in a given period of time.
Although I wonder if its necessary if you use an app password. What is the worst a bad-actor can do with the app password?