
How to Secure a Linux Server - known
https://github.com/imthenachoman/How-To-Secure-A-Linux-Server
======
imthenachoman
Hey folks! I am the author of this guide. I did not know this was here or I
would have commented sooner.

This is my first time putting a guide like this together. I see a lot of
really great feedback and I will be incorporating them into future updates.

I don't have time today but I will reply individually to all the comments that
I can.

I appreciate any/all feedback/advice. If possible, could future issues be
submitted to GitHub because it is easier for me to manage if everything is in
one place. [https://github.com/imthenachoman/How-To-Secure-A-Linux-
Serve...](https://github.com/imthenachoman/How-To-Secure-A-Linux-
Server/issues/new)

But right now I need to find all the other places this guide is linked to with
comments so I can address them too. :)

~~~
wodny
It has been mentioned in one of the pull request that this guide was being
discussed on HN right after it got here:
[https://github.com/imthenachoman/How-To-Secure-A-Linux-
Serve...](https://github.com/imthenachoman/How-To-Secure-A-Linux-
Server/pull/7).

There are two fresh threads on Reddit:

[https://www.reddit.com/r/linuxadmin/comments/arx7xo/howtosec...](https://www.reddit.com/r/linuxadmin/comments/arx7xo/howtosecurealinuxserver_an_evolving_howto_guide/)

[https://www.reddit.com/r/linux/comments/arx7st/howtosecureal...](https://www.reddit.com/r/linux/comments/arx7st/howtosecurealinuxserver_an_evolving_howto_guide/)

~~~
imthenachoman
Holy crap! I did not see those two on Reddit. I had posted to Reddit and was
monitoring those comments but didn't know of these. Thanks! Although, I'm sad
both of those threads got more comments then the original one I posted. Not
that it really matters in life. Heh.

And yes, I saw that pull request but I didn't know what HN was at that time.
After I saw that comment, and before they said HN = Hacker News, I started
Googling around to see if I could find where my repo was being mentioned. That
is how I found this post in HN.

Thanks!

------
wodny
This guide contains (at least) inaccurate statements. It's oversimplifying and
omitting important things while putting emphasis on some exotic details where
defaults would be sane enough.

"One key, the public key, can only encrypt data, not decrypt it" \- this is
cryptographically inaccurate. One should use it that way, though.

"Identity is verified by encrypting and decrypting data that both the client
and server know". That's not how signing works. The crucial part of the
process is to first establish an encrypted channel, then choose something
random (not used previously) and finally verify the signature is correct. What
exactly would be the data that "both the client and server know"?

"If you're sure there is nobody listening between the client you're on and
your server, you can use ssh-copy-id to transfer and append the public key".
Almost never happens unless you're in the server room, connected point-to-
point. But then the remark about someone listening doesn't make sense. And not
a single word about verifying server's fingerprint in the whole guide.

"Keep in mind doing this means you can't use the key for automation because
you'll have no way to send the passphrase in your scripts". Unless you use an
ssh-agent.

"umask is a Bash built-in which means a user can change their own umask
setting". Wrong implication here. umask is a syscall. It doesn't matter what
you use to call it. And what if you're using a different shell?

Suggesting painful default umask instead of just doing `chmod go-rwx $HOME`
and adjusting /etc/adduser.conf is debatable.

Making this guide "distribution agnostic" is IMHO futile and there already are
some distribution-specific guides like [https://debian-
handbook.info/](https://debian-handbook.info/).

~~~
okl
Arch:
[https://wiki.archlinux.org/index.php/Security](https://wiki.archlinux.org/index.php/Security)

CentOS:
[https://wiki.centos.org/HowTos/OS_Protection](https://wiki.centos.org/HowTos/OS_Protection)
(limited)

Debian: [https://www.debian.org/doc/manuals/securing-debian-
howto/ind...](https://www.debian.org/doc/manuals/securing-debian-
howto/index.en.html) (old)

Fedora: [https://docs.fedoraproject.org/en-
US/Fedora/19/html/Security...](https://docs.fedoraproject.org/en-
US/Fedora/19/html/Security_Guide/index.html) (old)

Mageia: [https://wiki.mageia.org/en/Msec](https://wiki.mageia.org/en/Msec)
(limited)

Oracle Linux:
[https://docs.oracle.com/cd/E52668_01/E54670/html/index.html](https://docs.oracle.com/cd/E52668_01/E54670/html/index.html)

Red Hat: [https://access.redhat.com/documentation/en-
us/red_hat_enterp...](https://access.redhat.com/documentation/en-
us/red_hat_enterprise_linux/7/html/security_guide/)

Slackware:
[https://docs.slackware.com/howtos:security:start](https://docs.slackware.com/howtos:security:start)
(limited)

SuSE:
[https://www.suse.com/documentation/sles-15/singlehtml/book_h...](https://www.suse.com/documentation/sles-15/singlehtml/book_hardening/book_hardening.html)

Ubuntu:
[https://help.ubuntu.com/lts/serverguide/security.html.en](https://help.ubuntu.com/lts/serverguide/security.html.en)
(limited)

~~~
Pokepokalypse
And for the masochists:
[https://iase.disa.mil/stigs/Pages/a-z.aspx](https://iase.disa.mil/stigs/Pages/a-z.aspx)

~~~
okl
Crikey, look at that mess:

 _Upgrade the version of the browser to an approved version by obtaining
software from the vendor or other trusted source. Method 1: View the following
registry key: HKLM\Software\Mozilla\Mozilla Firefox\CurrentVersion Method 2:
Search for the firefox.exe file using the search feature of the operating
system. Examine the files properties for the product version (not the file
version. For Windows OS, determine the version of the file by examining
navigating to Properties /Version/Product Version. Examine for all instances
of firefox.exe that are present on the endpoint. Criteria: If the version
number of the firefox.exe file is less than 50.1.x (or ESR 45.7.x), this is a
finding._

How about clicking on "About Firefox" in the menu? "Other trusted source" \-
like softonic, yes?

~~~
TimTheTinker
> How about clicking on "About Firefox" in the menu?

Perhaps they don’t trust it enough to execute it until they know it’s the
latest version and was obtained from a trustworthy source.

~~~
okl
At least the version there is baked into the (signed) .exe while the registry
information can be edited independently.

------
jSherz
The CIS benchmarks are a great place to start for hardening a system
([https://www.cisecurity.org/cis-benchmarks/](https://www.cisecurity.org/cis-
benchmarks/)) and there's also OpenSCAP gives you a nice way to scan systems
for compliance against a set of hardening rules ([https://www.open-
scap.org/](https://www.open-scap.org/)).

~~~
Geee
Thanks. What about using the hardened images they provide?
[https://www.cisecurity.org/cis-hardened-image-
list/](https://www.cisecurity.org/cis-hardened-image-list/)

~~~
jarito
Their hardened images are fine but there are two issues. First, you'll need to
test / dev with them as full implementations of CIS can break things (though
not as bad as the STIGs) and, IIRC, they charge extra for you to use their
images on public clouds.

------
supakeen
No mention of shipping off logs to another place? It's probably good to assume
someone will gain access and make after-the-fact forensics a primary concern
as well.

Something a lot of hardening guides seem to skip!

~~~
Someone1234
A lot of hardening guides skip the long tail for security.

Which is to say: So you've shipped logs off, so then what? How are you going
to monitor those regularly, what are you looking for, how are you going to
make sure important information stands out?

Many people set up remote logging and then never check the logs until after
there is an issue. An unread log isn't useful. Logs that are too spammy aren't
going to be read.

------
koolba
Here’s a bonus one: If you install Docker don’t add your non-admin to the
docker group as it’s effectively passwordless sudo.

~~~
bpye
Does Docker allow you to enforce user namespaces for all comtainers? I think
that avoids the issue.

------
bufferoverflow
Related: My First 10 Minutes On a Server - Primer for Securing Ubuntu

[https://www.codelitt.com/blog/my-first-10-minutes-on-a-
serve...](https://www.codelitt.com/blog/my-first-10-minutes-on-a-server-
primer-for-securing-ubuntu/)

------
snuxoll
Nothing about SELinux or AppArmor? While these are enabled by default on some
of the big distros I think a basic guide on managing and troubleshooting would
be beneficial.

~~~
peterwwillis
Yeah, the best impact would come from using SELinux, AppArmor, and GRSec.
Everything else is just tweaking defaults, which for up-to-date software
probably isn't going to impact your security much at all.

~~~
imthenachoman
I don't disagree but you gotta start with the basics. SELinux can be rather
advanced/complex and probably warrants its own guide. I still have to learn it
better before I try to write a guide on it.

------
pettycashstash2
I didn’t know you could add google Authenticator to your server. Thank you for
this write up.

~~~
Fnoord
You don't have to use a closed source TOTP client such as Google's. There's
RedHat's FreeOTP and there's another open source one called antOTP. You can
find them on F-Droid as well.

There have been TOTP PAM modules for ages (these work on a Linux client and
Linux server via e.g. SSH). You can even add YubiKey to PAM. Same for BSD Auth
and macOS.

~~~
noinsight
> RedHat's FreeOTP

It's abandonware these days and hasn't been updated for years. The iOS app
doesn't work anymore and can't use the camera for reading codes.

~~~
dfed-mpls
I use it on ios, and so does most of Red Hat employees (they state it is a
preferred version of tfa over the google app.)

It's not broken for the 15 or so accounts I use it for.

~~~
noinsight
Try the "scan code" button which should activate the camera and the app will
crash. I just tested it with my iPhone and verified. The iOS App Store
application was last updated 4 years ago. There's also comments there saying
it doesn't work anymore.

Reading / using existing codes might work properly, or manually inputting
them.

~~~
dfed-mpls
Nope. I used that feature just today to add another account. Works on my
iPhone X.

~~~
dfed-mpls
Also, the last commit to the app was a year ago, if you head to their github
page.

[https://github.com/freeotp/freeotp-
ios/commits/master](https://github.com/freeotp/freeotp-ios/commits/master)

------
ThePhysicist
I would not recommend using “ufw” for configuring the firewall on a production
server (as it’s not easily composable and lacks configurability for more
complex rules). I can recommend “ferm” instead as it allows you to compose
multiple config files (important for automation e.g. when using Ansible) and
allows describing more complex rules that e.g. involve policies.

~~~
imthenachoman
How would you define a production server? Do you mean in the context of a home
server or a server used by a large company? The guide is intended for a server
for home use. I hope anyone securing a large corporate server is not using
information on GitHub. If they are then the company has far bigger problems
than security.

I think for home use, ufw is probably good enough. I've been using it for 3+
years and it's worked out okay for me okay.

I have not heard of ferm but I will check it out. Thanks!

~~~
ThePhysicist
Sorry, very late reply: We deploy all our infrastructure with Ansible, hence
we want to have a way to configure the firewall for each role individually
without overwriting previous configuration. For example, we have firewall
rules for IP-Sec connections, SSH connections from the bastion host and then
specific configurations for applications like databases or message queues.
With ferm we can just create individual configs for each of these and put them
in a directory where they are loaded sequentially and automatically. This
allows us to iteratively define firewall rules and deploy different Ansible
roles.

For a home server ufw is probably good enough, as I said I wouldn't recommend
it for "serious" use in a highly automated environment.

------
dev_dull
> _2FA for ssh_

Correct me if I'm wrong, but isn't this what password-protected private key
encryption is?

~~~
mgbmtl
The author probably meant "two-step" (out of band verification) rather than
two-factor.

~~~
imthenachoman
A pass phrase on a certificate is two step. Password + TOTP is two factor.

------
khamba
I was pleased to notice that the author used DuckDuckGo search link in their
article instead of Google.

~~~
imthenachoman
Always. Google gives me 5 thousand pages of ads before the results I want. I
hate to admit that sometimes I still use Google for searching. Like I love
their shopping search.

------
rasengan
It would also be proper to disable non root access to /proc among other
things. You can do that by simply mounting with hidepid=2 or adding it to
fstab.

~~~
shereadsthenews
Why? Disabling access to /proc will disable a huge number of useful features,
such as the ability of a process to monitor and manage its memory usage, to
debug itself, and so forth.

~~~
mbaeten
For example, /proc/self/pagemap can be used for rowhammer attacks.

Source: Another flip in the wall of rowhammer defenses (IEEE S&P 2018)

------
z3t4
Dont forget about namespaces which lets you isolate apps. and Apparmor to
restrict file and network access. And also chroot and setuid to drop
privileges and change the root path. And unix sockets / named pipes. Instead
of letting your daemons access the whole network have them listen on a unix
socket.

And scan your own network, both from the outside and inside.

~~~
imthenachoman
I am not familiar with namespaces in this context? Can you please point me in
the right direction? I have added AppArmor to my to-do list. I know what
chroot and setuid are, not sure what to mention about them in the guide? And I
will look into unix sockets.

Can you recommend any good network scanning tools? I've been on the hunt for a
good one.

~~~
z3t4
I think the most useful for sysadmins are the network namespace that lets you
put an app into it's own network, so it can not see the rest of the network.
Container technology makes use of namespaces to create lightweight
VM/containers. See man namespaces, lxc

nmap is a popular tool for network scanning. I've also found tcpdump to be
useful for looking at network traffic.

setuid and chroot are useful for programmers, so once the app is up and
running, it can chroot into a data-dir and drop root privileges using setuid
to a unprivileged user. As a sysadmin you can also start the app from within a
chroot and run it as a unprivileged user which is preferably. Most (free)BSD
tutorials go though setting up a chroot jail, it's not as common in Linux.

Containers, VM's, and chroot will not stop a very determined attacker, but the
more restrictions the harder it will get.

Security is applied in layers: First you want to prevent people from the
outside. Access is most often gained through exploiting some
service/daemon/app running on the server. So the app should have as little
privileges as possible. System access is often then gained by exploiting
_another_ app (so you want all apps to be locked down, not just the network
facing ones), like getting Apache to run curl, that sends a internal request
to another app that has a known vulnerability and happens to run as root.
(nobody thought that app not accessible from the outside needed to get
patched). Once the attacker is inside the server, you also want to prevent
access to _other computers_ in the network.

~~~
imthenachoman
Got it. Thank you!

------
ozim
Securing ssh, 2FA for ssh, using key authentication...

Yes but first and foremost you don't expose ssh to all internet, it should be
allowed only from known IP addresses and you should VPN to have connect to
that known addresses. Section about firewall config just tells to open ssh...

~~~
zlynx
It might help protect against an OpenSSH zero-day but those are pretty rare.

I've been running my home server with an completely open SSH port since 2012
and haven't been hacked yet. And I know since I check occasionally with
Tripwire and/or RPM verification from a clean boot.

The only downside to an open SSH port is the thousands and thousands of log
spams from connection attempts. The upside is that I can log in from anywhere
using my phone directly or as a hotspot for the laptop.

Oh yeah, and VPN just pushes the problem back one level. For me SSH _is_ the
VPN. Requiring VPN access first to get to SSH is just pushing any security
problems back into the VPN server. Because now IT is the one with the open
port to the internet.

~~~
imthenachoman
Can you use Tripwire for free for home/consumer stuff? And what does RPM
verification do? Just make sure all the packages you have installed are legit?

~~~
zlynx
Tripwire is packaged up in Fedora. It's listed as GPLv2 licensed. So, yes.

RPM verification checks that all of the files installed through RPM have
checksums that match the original RPM. It also checks that the RPM
cryptographic signatures match.

So that should guarantee that files like the kernel, systemd, /bin/sh,
/lib/libc.so.6, etc are not compromised.

A system can still be vulnerable to persistent attacks installed in unwatched
files such as /root/.bash_profile, /etc/profile.d, extra files in
/usr/systemd/system, etc. So you also have to check for extra files that you
didn't install.

I don't have anything except Secure Boot to protect against UEFI attacks.

~~~
imthenachoman
Got it. Thanks!

------
joobus
Does anyone have experience with deborphan? Not previously aware of it, I
installed it, and it reported 2 programs which I use very frequently (conky
and cryptsetup) as being orphans. I dare not run the uninstall script in the
linked guide.

~~~
imthenachoman
Would you mind pasting the output of `deborphan` showing those two or creating
a new issue on my GitHub page? I want to show it to the author of `deborphan`
to see if he knows why its happening.

[https://github.com/imthenachoman/How-To-Secure-A-Linux-
Serve...](https://github.com/imthenachoman/How-To-Secure-A-Linux-
Server/issues/new)

------
okl
Is using gmail to send mails from your server a good idea?

~~~
tlb
It's better than running your own smtp servers. But the article proposes using
your own gmail account, to which the server has an app password. So a server
compromise leads to your own email being compromised. It should be a separate
gmail account.

~~~
imthenachoman
So I use a separate gmail account for my server but forgot to mention in
guide. I will add it.

Although I wonder if its necessary if you use an app password. What is the
worst a bad-actor can do with the app password?

