
Security Checklist - nthompson
https://securitychecklist.org/
======
nmjohn
There's a problem with viewing security as a "checklist" problem, it treats
everything as binary, black or white problem/solutions. In reality, this is
_far_ from the case. For example:

> Is TLS1.2 the only supported protocol?

Do you know what the implications are from actually implementing that? One
probably should do additional research before making that decision.

> Have you ensured that your content cannot be embedded in a frame on another
> website?

X-Frame-Options isn't the only way to achieve that.

> Have you ensured that the Internet Explorer content sniffer is disabled?

This really is only relevant if your site is hosting untrusted content.

> Are you using fail2ban to throttle ssh login attempts?

> Have you disabled password-based login over ssh, and only allowed key-based
> login?

The first really is of debatable value if the second is also used.

> Do forms set a cross-site request forgery cookie?

Cookies are not the only way to do this. More common, in my experience, is
including the csrf token in the dom (either in a form field as a "hidden"
input or as a meta tag)

> Do you have an account recovery flow? Delete it immediately.

Lol.

It's a great checklist for someone who knows what every item on the list is
and more importantly, _why_ it is on the list - but it is not something that
should be blindly followed.

~~~
movedx
>> Are you using fail2ban to throttle ssh login attempts?

>> Have you disabled password-based login over ssh, and only allowed key-based
login?

> The first really is of debatable value if the second is also used.

Actually, no. It makes perfect to block repeat offenders because they might
get lucky; they might be exploiting an RCE you're not aware of that takes
multiple steps; and they could be trying to DoS that server by filling the
logs and thus your disk (and with most people using default 10GB disks on
their VMs these days, this isn't too hard to imagine.)

Besides, the right answer is: use bastion hosts to proxy SSH connections,
preventing them from the outside world, and also port knocking :-)

~~~
nmjohn
> Besides, the right answer is: use bastion hosts to proxy SSH connections,
> preventing them from the outside world

Agreed

> It makes perfect to block repeat offenders because they might get lucky

No, just no, please don't spread this nonsense. There is no such thing as
"lucky" when the probably being discussed is 1 in (2^4096 - 1).

> they might be exploiting an RCE you're not aware of that takes multiple
> steps

I'll take my chances. If unauthed RCE exists in SSH (regardless of number of
attempts required) there are far more serious implications than any server I
manage. Additionally, I'm curious if you have an existing CVE where this kind
of exploit has ever been discovered.

> they could be trying to DoS that server by filling the logs and thus your
> disk ... this isn't too hard to imagine

Hard to imagine? No. have I ever actually seen in the real world? Also no.
Even if one server happened to be dos'ed by this, not a big concern, that's
why you run multiple redundant servers behind a load balancer. The only viable
attacks would be random in nature since attackers have no idea what the IPs of
your actual app servers are. (And if someone can mount an attack that can
determine them, you probably have bigger problems then a dos attack from ssh
logs)

All in all, I feel like this just strengthens my original point, of viewing
security as a checklist is a dangerous approach, one needs to actually
understand what they are doing.

I'm all for layered security, but the problem with using it just because "why
not" is that this methodology leads to an environment where there is so much
"stuff" nobody knows what is secure, what is not secure, and what strange
dependencies were the only reason something was secure in the first place. As
far as security goes, everything should have a well-defined purpose.

~~~
movedx
Looks like we will have to agree to disagree then. I've obviously had
different experiences to you, and have seen disks being filled up, systems
getting exploited after repeated attacks, and so forth, but you seem too
stubborn to work with on this point.

Good luck!

------
msbarnett
> Are all form fields (with the exception of password fields) validated with a
> restrictive regex?

Please, whatever you do, do not try to regex validate people's names. A friend
couldn't complete an order on a site the other day because it was rejecting
his name as "invalid" per its shitty regex.

See also Falsehoods Programmers Believe About Names[1]

[1] [http://www.kalzumeus.com/2010/06/17/falsehoods-
programmers-b...](http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-
believe-about-names/)

~~~
DrScump
<do not try to regex validate people's names>

Or email addresses, without knowing _all_ the legal characters. (An annoying
percentage reject '+', which is a great Gmail tool for spotting how your email
address "leaks out".)

~~~
NickNameNick
And a huge number don't suport email addresses with 'bare' tld's.

see rfc 7085 "Top-Level Domains That Are Already Dotless"

[https://webcache.googleusercontent.com/search?q=cache:rySaMA...](https://webcache.googleusercontent.com/search?q=cache:rySaMANuZuIJ:https://tools.ietf.org/html/rfc7085+&cd=2&hl=en&ct=clnk&gl=us)

~~~
breakingcups
I thought those weren't allowed / encouraged by either ICANN nor IAB? They
have a real-world possibility of colliding with internal hostnames.

------
tptacek
Quibbles:

* You're right now fine with 2048 bit keys. The thing that jeopardizes 2048 bit keys might end up knocking RSA out entirely.

* There's really no point to fail2ban if you disabled passwords for SSH, and doing that is much more important than fail2ban.

* The checklist item for password security should make it clear that you need a _password hash_ (PBKDF2, bcrypt scrypt, Argon2), not just a "hash".

Given the context, the only big thing that's missing is HPKP.

~~~
beefhash
Wouldn't it be wise to wait for argon2 to get some more cryptanalysis and
testing in the field before jumping on it, sticking to scrypt or bcrypt in the
meantime?

~~~
tptacek
No. The reason not to use Argon2 is that it doesn't have great library support
yet. Password hashes don't really fail like ciphers do. Nothing that is
discovered in Argon2 is going to make it worse than PBKDF2, and using PBKDF2
is just fine if that's all you've got.

~~~
devit
Huh?

A password hash could happen to be reversible, or could map lots of passwords
to the same value.

~~~
tptacek
No, neither of those things are going to happen.

It might happen if _you_ designed a password hash _from first principles_ ,
but that is not how Argon2 was designed, or, really, _any_ of the Password
Hashing Contest finalists. They're built on top of existing crypto primitives.

------
elchief
4096 bit RSA key is overkill and slow. 2048 is fine for years

You probably need to support TLS 1.1 for some older clients

Password entropy checks should only be used in a corporate environment. You'll
lose customers in a retail environment. You should certainly encourage your
customers to use high entropy passwords, but forcing them to will just lose
you money.

Login throttling and IP banning is a waste of time. Login throttling will just
DoS yourself. You need CAPTCHA or Proof of Work.

Synchronizing Token (CSRF token in hidden form fields) is better than CSRF
cookies

Use a real HTML parser (*Soup) to invalidate XSS form input not regex

~~~
theandrewbailey
I might be missing something, but I've never come across software having TLS
1.1 support without TLS 1.2.

~~~
elchief
[https://en.wikipedia.org/wiki/Template:TLS/SSL_support_histo...](https://en.wikipedia.org/wiki/Template:TLS/SSL_support_history_of_web_browsers)

Plus user agents that aren't browsers

------
koolba
> Do you have an account recovery flow? Delete it immediately.

Lol. Have fun doing this at scale for any consumer site.

Like everything else in security, having a password reset/account recovery
flow is a trade-off between security and convenience. For the vast majority of
websites, convenience wins.

------
lgarron
I don't see e.g. a GitHub repo to comment on, but the HSTS header example is
`Strict-Transport-Security: max-age=63072000; includeSubdomains; always`

`always` has no meaning, and probably indicates a typo in an nginx
configuration.

As others have pointed out, there is other TLS advice here that you shouldn't
follow blindly, like a 4096-bit cert (especially if your intermediate is
smaller) or TLS 1.2 only. Rationales and trade-off explanations would help
avoid misuse of this checklist.

The SSL Labs checker is probably the best tool to use for all the automatable
TLS checks.

~~~
tombrossman
I think the "always" is from Apache, and if I'm reading the site correctly it
is an example of a header value being returned. An example Apache directive
from [https://cipherli.st/](https://cipherli.st/) would be:

Header always set Strict-Transport-Security "max-age=63072000;
includeSubdomains; preload"

That said, when I test an Apache site I've set up using that same directive I
don't get the "always" so I'm not sure where it's coming from (in the
example).

------
garrettr_
> Do you have an account recovery flow? Delete it immediately.

What about users who need to, you know, recover their account?

~~~
nthompson
This checklist was designed for corporate-to-corporate and corporate-to-
military sites which host data deemed valuable. If you are designing consumer
websites where the user data is not particularly sensitive, then this advice
is not appropriate.

In my applications, it was the customers who demanded that account recovery
flows be removed, which I imagine is far from your own experience.

I will make a note about the intended customer group in the page . . .

------
foobarbecue
It'd be nice if this was an actual checklist. i.e. checkboxes rather than
images of a checkmark so that you could run through it.

~~~
gerry_shaw
I thought so too. I also thought it so too so I created this:
[https://github.com/gshaw/security-
checklist](https://github.com/gshaw/security-checklist)

~~~
nthompson
Damn dude. This looks way nicer than my page (and has links to hipper tools) .
. . I need to take a course in web design . . .

------
ejcx
By the way. If you aren't well versed in web application security, don't just
blindly follow guides likes this, especially when dealing with headers. You
really should learn about the technologies

> Strict-Transport-Security: max-age=63072000; includeSubdomains; always

I agree. All sites should be using HTTPS only and HSTS is a great thing, but
if you blindly follow this guide you might break your subdomains that don't
support HTTPS (third party or whatever crazy situation you have). The same
thing goes for HPKP which tptacek suggested below.

There's a right way and a very wrong way to use HPKP and if you go mucking
with these headers without fully understanding them yourself you might break
your site for a ton of people.

~~~
ams6110
Can you recommend a good reference for properly setting up HPKP?

~~~
tombrossman
This is a good guide: [https://scotthelme.co.uk/hpkp-http-public-key-
pinning/](https://scotthelme.co.uk/hpkp-http-public-key-pinning/)

I had a difficult time getting it to work without throwing errors for either
Nginx or SSL Labs tests and it was down to good old single vs. double-quotes.
Copy-pasting the output from a policy generator is not at all guaranteed to
work well.

Here's a template you can use, substituting EXAMPLE for your certificate
hashes (and optionally the report URI). Tested and working on Nginx and
multiple third-party validators:

add_header Public-Key-Pins 'pin-sha256="EXAMPLE"; pin-sha256="EXAMPLE"; max-
age=86400; report-uri=" [https://report-
uri.io/report/EXAMPLE"';](https://report-uri.io/report/EXAMPLE"';)

------
Taig
What is the meaning of the last point?

> Do you have an account recovery flow? Delete it immediately.

~~~
tyingq
They seem to be saying you should have no online process to deal with lost
passwords, forgotten userids, etc.

Which is, of course, not practical in most situations.

~~~
andreyf
I thought this at first, too, but giving the author the benefit of the doubt,
perhaps they meant that if someone deletes their account, you should actually
delete their account, not just disable it.

I haven't seen anyone do this, but as a user, I would prefer an option for my
account information to be emailed to me in a relatively readable format, so
that I can keep my data / re-upload it if I ever want to undelete my account.

~~~
tyingq
Sounds like a great idea, but I wouldn't word that as _" Do you have an
account recovery flow? Delete it immediately."_

Account recovery is shorthand for forgotton passwords and userids everywhere I
look.

~~~
andreyf
might be a stretch but
[https://en.wikipedia.org/wiki/Principle_of_charity](https://en.wikipedia.org/wiki/Principle_of_charity)

------
tptacek
The "account recovery flow" thing is, for everyone asking, presumably an in-
joke about how every application pentested for the first time ends up having
some gameover flaw in its password reset.

------
cdevs
I know the majority decision is keys over passwords but every time I hear a
security issue its over keys being stolen as typical malicious programs knows
the standard go to place to grab them after getting you to sudo something
evil. and the next response is you have time to change them before blah blah
blah but attackers don't send you a email that says hey I have your keys!
Wouldn't fail2ban with passwords be better for those who don't know how to
manage every step of this ?

------
r3bl
This seems great! Now, all that is needed is someone who will automate this
list across X most popular websites.

------
Laaw
Is the problem with account recovery the fact that it means "deleted" isn't
actually deleted?

------
mrmondo
Good start indeed, it'd be nice to have a small open source tool to do these
checks, maybe even with a simple web interface - anyone seen a simple security
checklist / analyser that's well maintained with the latest standards?

~~~
nthompson
SSL labs is the gold standard:

[https://www.ssllabs.com/ssltest/](https://www.ssllabs.com/ssltest/)

But I don't see a way to automate a test for the integrity of (say) a password
recovery flow or that one user cannot access another user's data.

~~~
mrmondo
Yeah that's what I use at present, if it's not an A+ it's not OK. Would be
nice to have an open source version of this that could perform additional
checks though.

------
tacone
It would be nice for this to become a github repo with all the bash scripts in
the page.

------
lolidaisuki
Web server security is pretty narrow portion of all security fields.

------
xpinguin
I could call that list deliberately malicious from user experience standpoint:

>Are password entropy checks done during user sign-up, using, say
AUTH_PASSWORD_VALIDATORS?

No. It's my information to be stolen, not yours. So then it is my choice,
whether to use 123 as password or not. Why should I care to manage the complex
password, when I use your service eg. twice an year and have no important
information there? (if you really believe that people are eager to fill
website with their authentic personal info unless they do not have other
options, you are probably fooling yourself).

The better alternative is just no registration at all :)

>Are failed login attempts throttled and IP addresses banned after a number of
unsuccessful attempts

So, you hadn't listened the previous piece of advice and forced me to create
password that would've passed through the password checker. Six months passed
and now I have to remember (I really don't want to bother with managing and
storing password to your service anyway) it. As you could imagine, it takes
several tries, dozen or two, maybe even three - depending on that cool
password validator of yours. Do you say, that I need to use tor or have some
pool of spare IP adresses just to login to your service?

>Are all form fields (with the exception of password fields) validated with a
restrictive regex?

Aha, start with an email and surname, polish with an address;) Then your
service will make it straight to the oblivion even faster!

>Do you have an account recovery flow? Delete it immediately.

Quite appropriate actually: when all tor exit nodes are banned by your login
attempt throttler, that retards with severe memory impairment (whom you
sometimes by mistake call "clients" in your marketing bullshit) still must not
have a glimpse of a chance to use their account!

~~~
Karunamon
> _No. It 's my information to be stolen, not yours._

And when the site gets broken into years later, irresponsible users (or
"morons" as you put it) will pillory them publicly even though it's their own
damn fault for using a crappy password or using the same password for their
email.

The number of people who will stop using a site because they can't meet basic
password strength requirements is minimal and not worth caring about.

~~~
xpinguin
>irresponsible users (or "morons" as you put it)

Me?! God is my witness, I do not. Rather, it is creators of that "checklist",
who suggest that your users are idiots, who should be punished even for being
unable to remember their password. IMHO, the really bad thing about the list
is the fact that technically legit (I am not a webdev though) points are
casually mixed with statements implying derogatory stance towards target
users. I do not want you to control the ways I manage my information, that's
the point of my rant.

After all, it is not a big deal if some crappy SV startups whose ultimate fate
is to die silently after an year or two anyways, will adopt the practice.
Problem arises when thing would go in the wild, bringing only headache and
distraction to those who is able to control their information flow.

