
How to SSH Properly - old-gregg
https://gravitational.com/blog/how-to-ssh-properly/
======
Ayesh
\- Use Ed25519 keys. They are now supported in pretty much every server out
there (all recent OpenSSH versions, GitHub, GitLab, etc). RSA 2048 keys are
unbreakable for the foreseeable future, and using 4096 bit keys are just being
paranoid with no gain. You can fit 4x Ed25519 keys in a tweet.

\- Setup and use ssh-agent. They make the life so easy.

\- You can use Yubikey and pretty much every other solution out there with
with OpenSSH now.

~~~
ddevault
>Setup and use ssh-agent. They make the life so easy.

My #1 ssh usability tip: put this into ~/.ssh/config:

    
    
        AddKeysToAgent yes
    

It'll automatically add keys to your agent the first time you use them during
a session, so you don't need a separate step for adding keys every time you
log in.

~~~
inetknght
> _It 'll automatically add keys to your agent the first time you use them
> during a session_

If you have more than one private key in your agent, then SSH will try each
one sequentially. That can lead to mistakenly getting banned by monitoring
software for what can appear indistinguishable to be an authentication attack.
Not to mention the _wrongness_ of presenting "any and all keys" to any host
you connect to.

If you have multiple keys in your agent, you _really_ should manually
configure what key to present using `-i` or `IdentityFile`. But you should
also use `IdentitiesOnly` too.

~~~
shawnz
> If you have multiple keys in your agent, you _really_ should manually
> configure what key to present using `-i` or `IdentityFile`.

If you specify -i or IdentityFile, the agent isn't used at all. You will have
to type the keyfile password even if the key is in the agent, therefore making
the agent useless.

This "feature" has been a big annoyance for me, since I like to use different
keys per machine.

~~~
kadoban
There's no way that's true. I just checked and I have that option set on many
of my ssh aliases, and ssh-agent functions exactly as expected.

Maybe it depends on version or something? I've had the same config for years
though over several very different OSes and presumably versions of everything.

~~~
shawnz
Wow, you're right. I tried again just now and it worked fine. In the past I
tried for some time to get this setup working and was never able to do it. I
don't know what I was doing wrong.

~~~
manuel_w
> If you specify -i or IdentityFile, the agent isn't used at all. You will
> have to type the keyfile password even if the key is in the agent, therefore
> making the agent useless.

I thought so as well, so the reply to this message surprised. I looked into
it, and the following is the case on my Debian 18.04 machine running KDE:

With specifying the key through `IdentityFile` I can happily connect without a
running ssh-agent. So it's true, that it can do without using the ssh-agent.
But if the key has a password, it will prompt for it everytime it's used. I
wouldn't phrase it "the agent isn't used at all", though, because for when
ssh-agent is running, and it contains the key+password, it seems to happily
use that agent, as it doesn't prompt for the password anymore.

Side note: If my understanding is correct, `IdentifyFile` lessens the need for
consulting the ssh-agent as stated above, but, except from consulting it for
the key password, the agent might be consulted for one more reason as well:
Iterating over keys if using the specified one proved unfruitful. For this to
stop, you'd have to specify `IdentitiesOnly yes` as well. But this I didn't
test, so it's based on theoretical understanding only.

Edit: Oh, the last part was already explained in some other thread which
branched of from this. So this post didn't actually provide some new insights,
it seems.

------
jonluca
The title of this doesn't properly match the contents. This is basically a how
to set up an enterprise SSH environment using certificates. This doesn't tell
you anything about "How to SSH Properly".

~~~
Aachen
I almost thought so too and was itching to hit back and post an annoyed
comment, but there are actually two more sections: using 2FA (straightforward
TOTP branded by our favorite not-evil corp though I've not no idea what
they've got to do with it) and setting up a jump host / bastion host.

So maybe the title shouldn't be "I like ssh certs" but "ssh security for
organisations", but your point still stands.

~~~
mynameisvlad
> though I've not no idea what they've got to do with it)

It's the name of the TOTP PAM module they use, which was created by Google:
[https://github.com/google/google-authenticator-
libpam](https://github.com/google/google-authenticator-libpam)

------
m463
I like that with macos, you can create keys with a passphrase, and then that
passphrase can be provided by the keychain which is unlocked at login

    
    
      ssh-keygen -t ed25519 -b 521 -f .ssh/id_foo_ed25519
      cat id_foo_ed25519.pub | ssh foo tee -a .ssh/authorized_keys   
      ssh-add -K ~/.ssh/id_foo_ed25519
    

along with:

    
    
      Host foo
            HostName foo.bar
            User me
            StrictHostKeyChecking yes
            IdentitiesOnly yes
            IdentityFile ~/.ssh/id_foo_ed25519

~~~
teekert
This sounds really nice, this would be great to have for Linux or wsl...

~~~
messo
Isn't this default behavior in Gnome? At least when using Fedora, Ubuntu or
the more integrated distros. I'm pretty sure I used the fingerprint reader to
access SSH with one of my keys on Fedora a few years back.

------
exabrial
As an alternative to ssh certs, we lookup keys in an LDAP Cluster. Each
approach has pros/cons. SSH certificates are decentralized and resilient
against failure, but revocation of a breached certificate isn't instantaneous.
Keys-in-LDAP offers less resiliency, but offers instant key revocation. It's
also simpler and offers a familiar experience to most devs.

The most important thing is to look at your attack surface realistically and
plan accordingly. Just blinding putting in an SSH cert authority of an
unsecured-non-redundant ldap cluster is worse than no key management at all.

~~~
closeparen
1) Programmatically issue certificates with short lifetimes after
authenticating against your IdP. This way if a user is deactivated in LDAP
their access will expire on its own after a few hours.

2) Firewall off all but a few bastion hosts and ProxyCommand through those.
Immediate revocation can be assured by updating the CRLs on those specific
hosts.

~~~
webvictim
This is definitely the premise of what I was going for with the post. I'm a
firm believer in the idea that short-lived certificates which expire by
default are one of the best ways to provide access to infrastructure, and
enforcing that access comes from a limited list of bastions gives you an easy
choke point to withdraw access as desired when you need to.

~~~
sytringy05
Isn't there a netflix ssh CA that does this?

~~~
V-eHGsd_
yes, it's called bless.

[https://github.com/Netflix/bless](https://github.com/Netflix/bless)

there's also cashier

[https://github.com/nsheridan/cashier](https://github.com/nsheridan/cashier)

------
dastx
All of this is done for you if you set up Hashicorp Vault. In case someone
doesn't know, it provides secrets as a service that allows you to store and
retrieve static secrets as well as dynamic secrets. The great thing about it
is that you can set up authentication from multiple sources including EC2/GCE
instances, LDAP and much more. But it also allows you to generate dynamic
secrets including, but not limited to, SSH certs, IAM users, AD users, sql
users and much more.

It's Hashicorps most mature tool and should be in everyone's toolchain.

~~~
outworlder
Vault requires a non-trivial amount of setup and configuration, and then you
need to configure SSHD to use it. You also have to maintain the vault server,
HA, yada yada. And educate people on how to use it.

Vault truly shines in its ability to provide ephemeral credentials. It can
even "package" arbitrary secrets for you, and ensure that they are only ever
retrieved once.

It's a fantastic piece of software, one that more companies should be using,
but it is not a push button deployment. It is borderline impossible to
implement in many 'traditional' organizations.

~~~
eeZah7Ux
> truly shines in its ability to provide ephemeral credentials. It can even
> "package" arbitrary secrets for you

Those credentials should not be generated, stored or provided centrally in the
first place.

------
anilakar
I've found it a good practice to add a YYYYMMDD serial number:

    
    
        -z 20200402
    

That way, keys generated within a certain time interval can be revoked with a
simple serial number check.

------
jontro
How is ssh certificates the next level up? Feels like the added complexity of
using this is might be worthwhile for larger orgs but maybe not smaller shops

~~~
old-gregg
Disclaimer: I work at Gravitational, one of our engineers is the author of the
article. The added complexity is indeed significant, that's why we built
Teleport [1] which is much, much simpler open source alternative to OpenSSH
for such use case (the blog post does not mention Teleport because it was
written for OpenSSH users).

But generally, certificates have a number of advantages: they expire quickly
(you can have one-shot certs), they contain your full name, email address, or
whatever you wish via metadata, this allows to implement RBAC on top of SSH.
They address "trust on first use" issue, and they're easy to synchronize with
other protocols, i.e. you can have the same CA issue certificates for both
Kubernetes and SSH (with identical RBAC rules).

When implemented properly, they're actually easier to use. The whole world
uses certificates to do online banking and shopping, there's no reason not to
have the same seamless experience for SSH.

[1] [https://gravitational.com/teleport](https://gravitational.com/teleport)

~~~
Gaelan
btw, huge props for not mentioning teleport in that blog post. So many
companies have blog posts that masquerade as tutorials for "how to do X" but
then change halfway through to "OOOORRRR you can not do all that complicated
stuff and just buy our product instead," which feels pretty scummy. Thanks for
taking the high road.

~~~
old-gregg
Thank you, Gaelan! I am not the author, but will pass this along to Gus.

------
egwor
One thing I find useful is to config default usernames.

You can do this by adding a line like below into ~/.ssh/config

Host alias domain.com HostName domain.com User aUser

This allows you to just write `ssh alias` and it will convert it to ssh
aUser@domain.com.

~~~
megavolcano
you can also specify which private key to present using

IdentifyFile /path/to/key/file

when I have different hosts that require different keys to access, this is
very useful so I don't have to specify them every time

and agent forwarding is useful as well, for example, I can push to github from
a vm using my laptop ssh key and I don't have to generate an additional key
for the vm

ForwardAgent true

with the new Windows terminal, I setup different tabs to be SSH to different
machines based on my .ssh/config file

{ "guid": "{1c9b268e-7606-4a07-b097-d8bc62fb5207}", "hidden": false, "name":
"ubuntu.localdomain", "commandline": "ssh.exe me@ubuntu", "colorScheme":
"Elementary" }, { "guid": "{b7161acc-8235-4283-9a85-e93df3d09125}", "hidden":
false, "name": "db.localdomain", "commandline": "ssh.exe me@db",
"colorScheme": "Ubuntu" },

Windows Terminal with a very nicely define ssh config is a killer combo.

~~~
inetknght
If you have more than one private key in your agent, then SSH will try each
one sequentially. That can lead to mistakenly getting banned by monitoring
software for what can appear indistinguishable to be an authentication attack.
Not to mention the _wrongness_ of presenting "any and all keys" to any host
you connect to.

If you have multiple keys in your agent, you _really_ should manually
configure what key to present using `-i` or `IdentityFile`. But you should
also use `IdentitiesOnly` too.

~~~
megavolcano
thanks for the tips!

------
sneak
Is active MITM that much of a threat model item that running a whole SSH host
key CA is the answer? I can't see that it is.

I thought this was going to be about Yubikeys or other HSMs.

~~~
munchbunny
It's not really MITM that's the issue. It's about maintaining access control
for SSH access at scale. Once you have a big enough team, and especially if
you store sensitive customer data, you generally want to start thinking about
a system that can temporarily provision SSH access on demand so that nobody
has standing access, at which point going through a CA and bastion servers is
a canonical approach.

Exactly how big your team needs to be is a judgement call, but I think a
reasonable threshold where you really should start thinking about CAs for SSH
access is around where it's no longer possible for the dev ops people to be
aware of what everyone who needs production access is working on.

~~~
sneak
I hadn't realized that this blog post is from the same people who make
Teleport, which is a bastion/logging/access ssh proxy for solving exactly that
problem. It seems to me that the linked article is probably just content
marketing spam (much like Digital Ocean's extensive "how to install $x on a
vps" posts) to boost the domain's pagerank/authoritativeness.

I still don't think people should be using SSH CAs for anything that isn't a
short-lived, single-use bastion/proxy key. They're on the right track with
Teleport; not so much with this standalone article.

~~~
dielectricboog
Those how to install X on a VPS articles are requested by active customers,
occasionally written by customers, and consumed far more often than you think
by customers. DigitalOcean responded to Linode Library (as it was then called)
when building that out, and having been somewhat involved in the development
of Linode Library, I can assure you that neither SEO nor PageRank was near the
top of the rationale list. Maybe tenth on the list as a nice bonus.

The people who worked on Linode Library in the beginning were engineers, not
social media people nor content marketers. The person who wrote its first CMS
is a former nuclear submariner and Perl guy with like nine followers on
Twitter. I’m sure you imagine a huge meat grinder spam operation paying by the
word, but basically one _engineer_ thanklessly wrote more than half of it in
the beginning and went on to a distinguished career at 10gen/Mongo, not
sitting on Mechanical Turk. I’d bet DO’s operation is identical and exists for
the same reasons, and they’re just as proud of it as I am my extremely limited
contributions to an offering that _objectively improves the operations
discipline_.

It’s annoying to watch valuable work that expands the profession get dunked on
and called spam by a snide comment for absolutely no reason.

There’s no reason to be so cynical about things in a hot take, particularly
when it was an irrelevant shot in context. Not everyone is blessed with innate
knowledge nor a gift for studying man pages, and those docs are useful
checklists to make sure I didn’t forget anything with common software like
ntpd _to this day_ in my own, principal level, career. People learn Unix
operations by buying a Linode. For a $10/month VPS you also get a bunch of
tested, free guides that work on them and teach you how to do things. They’re
not paywalled to customers and show up in your Google results. I know. The
abject horror is almost unbearable.

~~~
chrisweekly
Well said. (Despite the final sentence's understandable but unnecessary
parting shot.)

------
ansible
With regards to using ssh-agent, this is what I have in my .bash_profile:

    
    
      export SSH_ENV=$HOME/.ssh/environment_${HOSTNAME}
    
      function start_agent {
           echo -n "Initialising new SSH agent... "
           touch ${SSH_ENV}
           chmod 600 ${SSH_ENV}
           /usr/bin/ssh-agent -s | sed 's/^echo/#echo/' > ${SSH_ENV}
           echo "succeeded."
           . ${SSH_ENV} > /dev/null
      }
    
      # Don't use Gnome keyring for SSH
      if [[ $SSH_AUTH_SOCK =~ /run/user ]]; then
        unset SSH_AUTH_SOCK
        unset SSH_AGENT_PID
      fi
    
      # Source SSH settings, if applicable
      ssh-add -l > /dev/null 2>&1
      if [ $? -eq 0 -o $? -eq 1 ]; then
          echo "SSH Agent found."
      else
          echo -n "Looking for SSH Agent... "
          if [ -f "${SSH_ENV}" ]; then
              . ${SSH_ENV} > /dev/null
              ssh-add -l > /dev/null 2>&1
              if [ $? -eq 0 -o $? -eq 1 ]; then
                  echo "found pid: ${SSH_AGENT_PID}."
              else
                  echo "not found."
                  start_agent;
              fi
          else
              start_agent;
          fi
          ssh-add -l
      fi
    

This starts up a new ssh-agent if it can't find one, but uses the same agent
if it is already running on this machine (I remote into a bunch of systems).
So even if I use a bunch of screen sessions, or remote in to a machine
multiple times, it will always try to use the same agent.

~~~
jpeeler
Have you considered using
[https://github.com/funtoo/keychain](https://github.com/funtoo/keychain)?

Edit: I see it's been mentioned elsewhere here.

~~~
ansible
Well, that's interesting. And a _whole_ bunch of code. From a quick skim, it
is basically solving the same problem in a different style.

I didn't want or need to wrap ssh-agent. I'm fine with using it and ssh-add by
themselves. I just wanted to automatically find and re-use an existing agent
rather than spawning a new one.

------
kitotik
It seems weird there wasn’t more emphasis on the implications of running your
own CA. I can imagine lots of people coming across this article and doing a
quick copy pasta not realizing what just happened.

------
et1337
There are a number of turnkey solutions for this. One is ScaleFT[1]. My
company is another: StrongDM[2]. Our product also keeps audit logs for
compliance.

[1] [https://www.scaleft.com/](https://www.scaleft.com/)

[2] [https://www.strongdm.com/](https://www.strongdm.com/)

~~~
old-gregg
[https://gravitational.com/teleport](https://gravitational.com/teleport)

Free and open source, made by the authors of the article.

~~~
jlgaddis
> _... made by the authors of the article._

And, just so everyone is clear, your employer.

------
ZoomZoomZoom
More fitting to the title would be this page:
[https://infosec.mozilla.org/guidelines/openssh](https://infosec.mozilla.org/guidelines/openssh)

It's a nice practical guideline with all the basics covered.

------
rauhl
On a related topic, what is the current best practice for avoiding TOFU for
cloud providers? Sure, SSH certificates provide that for normal users, but how
does the SSH CA know — ideally in an automated fashion — what that public key
is? An admin can SSH into the machine to check it, but then he has to trust
the key he sees. He could access it via the console, but that is a manual
process. All the other approaches I can think of involve generating at least
one key outside the machine which uses it, which seems like a poor solution.

~~~
wraithm112
At least on AWS, you can parse the System Console Output. On the first boot,
the box dumps its pub keys there. We have a tool that parses the output:

[https://github.com/bitnomial/aws-
ec2-knownhosts](https://github.com/bitnomial/aws-ec2-knownhosts)

Right now, this is pretty specific to our use of it, specifically our use with
Terraform EC2 instances. We'd happily suggest changes to make it more generic.
But you can see the parsing logic there.

------
otterley
On AWS you can use SSM Session Manager and gain secure shell access to
instances without using SSH at all. As long as the instance has the SSM Agent
on it and network connectivity to the SSM API, it just works:
[https://docs.aws.amazon.com/systems-
manager/latest/userguide...](https://docs.aws.amazon.com/systems-
manager/latest/userguide/session-manager.html)

(Disclaimer: I work for AWS, but opinions are my own and not necessarily those
of my employer.)

~~~
pritambaral
Does it work with the `ssh(1)` client? I'd be okay with using this as long as
it replaced _only_ the network-connectivity and authentication parts. I do use
many of the other features made possible by SSH:

1\. Network forwarding (local-to-remote, remote-to-local, dynamic, and unix
socket support). The link you supplied mentions only local-to-remote.

2\. Agent-forwarding (don't worry, I have confirmation-on-every-use, see my
other comment:
[https://news.ycombinator.com/item?id=22753590](https://news.ycombinator.com/item?id=22753590)).
This I use all the time to be able to authenticate to other SSH servers and
git hosts. This is a must-have for me now for remote development and pair-
programming.

3\. sshfs, sftp, and rsync. Again, absolute beauties at the job of managing
files remotely.

This is just off the top of my head, because I use these features day-to-day,
but there's many other nice things ssh does that have nothing to do with the
underlying connection or authentication method.

~~~
tialaramex
Agent forwarding is powerful but dangerous. Prefer ProxyJump where applicable.

With Agent forwarding any authorised machine you connect to (and thus anyone
who controls that machine) gets to use your credentials for the duration, to
my knowledge no general purpose clients give you feedback on this usage, so
you won't know if this happened - if Agent forwarding is enabled it can be
used even if you never use it. With ProxyJump the intermediate doesn't get any
view of your credentials, not even whether those are the same credentials you
used for that intermediate host, or different. It is only enabling you to
connect to a host it can reach that you can't reach directly and nothing more.

~~~
pritambaral
> ... to my knowledge no general purpose clients give you feedback on this
> usage, so you won't know if this happened

I'm afraid you're mistaken here.

ssh-add(1) and ssh-askpass(1) support confirmation per-use:
[https://man.openbsd.org/ssh-add.1#c](https://man.openbsd.org/ssh-add.1#c)

The agent I personally use — gpg-agent — also respects this protocol and asks
me to confirm each use:
[https://www.gnupg.org/documentation/manuals/gnupg/Agent-
Conf...](https://www.gnupg.org/documentation/manuals/gnupg/Agent-
Configuration.html#index-sshcontrol)

Even if it did not, I can always configure gpg-agent to never cache the
passphrase, thus making it ask me for the passphrase every time. This would
also adequately serve as a means of confirmation, but thankfully this tedious
option is not necessary, because of the above.

> Prefer ProxyJump where applicable.

I already do, but it's pretty orthogonal to agent-forwarding. The uses of
agent-forwarding I mention in my original comment simply cannot be served by
ProxyJump.

~~~
tialaramex
Today I Learned. Seems like a key component (AddKeysToAgent Confirm) was in
OpenSSH 7.2 in 2016, but I felt like I checked this wasn't an option more
recently. Happy to be proven wrong, thank you.

------
davidcollantes
Assuming RSA keys are being used for compatibility?

~~~
Kovah
Wanted to ask the same. A lot of tutorials still use RSA but I see no reason
to use it in 2020. ECDSA keys are supported since an OpenSSL version from
2013.

~~~
GekkePrutser
I still use RSA mainly because Yubikey doesn't support them yet with OpenPGP
keys.. I felt the increase in security usage was a worthy compromise to using
RSA with a key length that's not yet broken.

I think Fido2 mode does support it but I don't know whether I trust Fido2
fully for SSH.. Especially because it seems to still store a private key
locally, which defeats the whole purpose of using a hardware key (I saw this
here: [https://developers.yubico.com/SSH/](https://developers.yubico.com/SSH/)
). The whole idea is to not have the private key anywhere but inside the token
itself. It's also a bit early in development where RSA has proven itself. It
has some known weaknesses (including high quantum computing vulnerability) but
they're well understood.

~~~
tialaramex
No, the FIDO OpenSSH support doesn't really store a private key "locally" but
I can see why it's confusing. I'll explain.

FIDO was conceived for the Web, and so the SSH authentication model wasn't a
core consideration in its design.

In SSH authentication always goes like this:

Client: "I can prove I'm Alice because I can do X" Server: "OK, do X" Client:
(Proves they are Alice by doing X)

OR

Client: "I can prove I'm Alice because I can do X" Server: "Not good enough.
What else?" (Repeat with a different method or give up)

Whereas WebAuthn (and U2F and other models FIDO was proposed for) have
authentication go like this:

Server: Prove you are Alice, if you are you can do Y or Z Client: (Does Z)

In FIDO there's a magic "cookie" used, in theory this could be just a "serial
number" for a credential, but in reality for a typical FIDO dongle (such as
Yubico's Yubikey) it's actually your private key for that site, encrypted
using a symmetric key known only to that Yubikey. This way your Yubikey can
unlock a genuinely unlimited number of sites, each with completely fresh
credentials that can't be correlated, without infinite flash storage. A site
says "Here's that cookie you gave me, now prove you're GekkePrutser" and your
Yubikey consumes the cookie, decrypts it to get a private key, then uses that
private key to prove you're GekkePrutser.

But as we saw in SSH the authentication doesn't happen in that order. The
Security Key needs the cookie first, but the remote server is waiting to hear
how you'll prove who you are first, it's a stalemate.

To break the stalemate the cookie is stored on your local system, so that can
be fed to the Security Key, and then things can proceed.

Down the road apparently the OpenSSH team will add support for FIDO2's
"usernameless" behaviour which doesn't need a cookie but does consume limited
resources on the Yubikey. If you just can't accept the idea of storing the
opaque cookie value because you know it has your private key encrypted inside
it, then that mode will suit. It's also useful for "road warriors" who are
willing to trust other people's computers briefly but don't carry their own.
But that isn't finished today and it seems they don't expect it to be the
usual way this is used, since it requires a FIDO2 (not just FIDO) Security
Key.

------
perlgeek
Somewhat off-topic, but I've noticed that ssh-keygen is _much_ faster at
generating key pairs that GnuPG, due to the latter wanting more "proper"
entropy.

Does that mean that GnuPG is overly paranoid? Or ssh-keygen's keys potentially
insecure? (I really hope not). There must be some good explanation for this
huge speed difference.

~~~
Aachen
The difference must be urandom vs random. You'll find a lot of words spent
arguing the misinformed vs the enlightened if you look for those terms, on HN
or elsewhere. I'm in the urandom camp and think that's the only thing that
makes sense on the condition that it blocks/errors until seeded properly once,
which means so much as that gpg is misinformedly paranoid in my opinion and
that if ssh keygen had been insecure it would have been fixed long ago, but do
read up and form your own opinion.

~~~
emmelaich
I don't think it's urandom v random.

It really just waits longer.

------
prussian
Host certificates are fine, but user certs are pointless landmines when
features like AuthorizedKeysCommand exist. why worry about revocations,
expiries and all that noise when you can simply have some script just pull the
allowed keys? sure maybe login is a bit slower, but does that matter much?

~~~
webvictim
Don't get me wrong, using AuthorizedKeysCommand is a lot better than having a
static ~/.ssh/authorized_keys file on a server, but it isn't anything like as
powerful as using user certificates.

Certificates can do a lot more than authorized keys can, like enforcing the
use of specific principals, commands and options and embedding that
information into the file itself without needing to modify each server's SSH
configuration. They're also self-contained and will still work in situations
where some external service providing a list of keys goes down. I've been on
the rough side of a huge LDAP outage which prevented necessary access to the
infrastructure to fix it, and it was a horrible experience. There's none of
that problem with certificates as long as you make sure you have one which is
currently valid.

I'm also generally of the opinion that it's safer to enforce the use of
authentication which expires by default rather than relying on some external
process to do that for you.

~~~
Spivak
But AuthorizedKeysCommand and certs are at least equally powerful because
they're both ways of specifying the content of the same authorized_keys file.

~~~
webvictim
It's something of an implementation detail - you don't generally specify the
usage of certs on a user-by-user level, you do it by trusting the entire CA in
/etc/ssh/sshd_config and then using the signed content of the individual cert
(expiry date, principals etc) to dictate whether someone should be allowed to
get access or not.

Look at it in terms of building in a decision at compile-time rather than at
runtime. With AuthorizedKeysCommand, you're running something just-in-time on
an SSH login to determine whether something should be allowed to proceed. With
a CA and a process for issuing certificates, that decision is made at the time
the cert is issued and then the cert is good for the duration it's issued for.
It's entirely self-contained as sshd itself is making the decision about
whether the cert is within its validity period or not.

It's obviously a decision that people can make based on their own
infrastructure, but my opinion is that the compile-time model is more reliable
as it's a fully self-contained system and doesn't rely on an entire fleet of
servers being able to connect back to an external service at runtime to
determine whether you should be allowed to log in. That sort of thing
invariably comes back to bite you when you really _need_ to be able to log in
and you can't because the external service is down.

------
293984j29384
Slightly off topic but the author recommends expiry periods for certificates.
Other than rotating a key because of compromise, this seems unnecessary. We've
finally decided that passwords should no longer be mandatory rotated, why
should certificates?

~~~
stephenr
How do you revoke access if the certificate has no expiry?

~~~
293984j29384
Forgive my ignorance but what's the correlation between expiry and revoking?
I'd assume there are several reasons you'd want to revoke a certificate such
as it's been compromised. How would it's expiry date help that?

~~~
webvictim
Author here. My take on this is that fail-closed is a vastly better security
model than fail-open. I am genuinely surprised that OpenSSH actually issues
certificates with no expiry date as a default.

If you have a certificate which expires within a day by default then an
unsuccessful revocation is no longer a huge cause of stress. In the worst
case, you lock down access to your bastions and disallow the issue of any
future certificates for that user. Within a day, any potential threat from
that certificate has vanished. This seems preferable to having a mandatory
requirement of an up-to-date revocation database which is synced everywhere.

~~~
293984j29384
Just to make sure I understand your example, for users who require regular
access you re-issue certificates daily? I could see that being useful for a
"one off" type thing (i.e. you want to temporarily grant access for one day)
but how does that help regular users?

I'm also not sure it's easier to "lock down access to your bastions" and wait
out the certificate expiration instead of having a certificate revocation
database. Although OpenSSH does not provide a mechanism to distribute the
revocation list it seems trivial to add a certificate to the revocation list
and distribute it in an automated fashion.

Lastly, since you have to both lock down hosts and wait out the expiration,
does that not constitute a fail-open system? I really don't think an
expiration date mechanism makes this a fail closed system. Either method
requires manual intervention upon compromise.

~~~
webvictim
Yes, even for very regular users I would recommend setting up a process
requiring users to get a new certificate on a daily basis with a short
validity period. You can automate a lot of this and make it a simple one-
command process to get a new certificate - even something like a simple shell
script called by ProxyCommand is a good habit to get into. In bigger
organisations you'd likely want to centralise this process somehow or
institute other tooling.

The overarching reason isn't really a question of "helping users" as such,
although I would strongly encourage making the certificate issuing process as
quick and easy as possible to encourage adoption and reduce pushback. The
people it really helps are security teams and organisations as a whole who can
now have more confidence that they haven't left holes in their infrastructure
which can be exploited by bad actors. It also checks a lot of boxes for
auditing, compliance and reporting purposes which are huge positives in a
corporate environment. If you're able to say "yes, disgruntled former employee
X had a certificate that would have given them access to all these servers,
but it expired three days ago" then that's a lot better than saying "X has a
certificate that gives them access to all our servers, but we _think_ we've
blocked it from being used everywhere".

Overall, I agree that the model does lend itself better to things like access
to critical production infrastructure (where access should be the exception
rather than the rule), but in my opinion it's a good practice to get into for
access to everything. The ability to log that a certain user requested a
certificate at a certain time and then link that to exactly where the
certificate was used (via centralised logging, for example) is incredibly
powerful.

You're perhaps correct that both do constitute fail-open systems at first. The
difference is in the vulnerability period - with an expiring certificate, that
ends at a fixed point in the future. With a certificate that has no expiry,
that period never ends until such time as you rotate your CA and force
everyone to get a new certificate - something which is also far less of a
burden when your certificates expire every day by default and you have a
process for getting a new one, incidentally.

~~~
293984j29384
I appreciate your detailed response but I think we'll just have to agree to
disagree here. My personal opinion is that there isn't any value in this
arbitrary temporal benchmark for certificates expiring. When a certificate is
compromised, or needs to be revoked, it needs to be revoked immediately. At
that point, your trusting the same mechanisms to remove access in either
system. An auditor is going to be interested in the period between the user
having access and that access being revoked. The fact that the key expires
later on (even within just hours) is irreverent, as it's after revocation and
it's already invalid. Anything less provides the bad actor with plenty of time
to do something malicious. The example you give in quotes would be immediately
followed with "Okay, but how did you disable that access immediately?"

You could make keys valid for only a minute and it wouldn't add any security,
as only seconds are needed for a malicious action to take place.

------
GRMrGecko
This is perfect timing as I just wrote this tool to help sign ssh host keys
and user keys. [https://github.com/GRMrGecko/ssh-
cert](https://github.com/GRMrGecko/ssh-cert)

------
fractalf
Is there a way to set up 2 ways auth with andOTP instead of the google auth?

------
sandGorgon
what's your opinion of using epass2003/pkcs11 for ssh?

[https://stuff.purdon.ca/?page_id=70](https://stuff.purdon.ca/?page_id=70)

[https://sigg-iten.ch/learningbits/2014/11/13/first-steps-
wit...](https://sigg-iten.ch/learningbits/2014/11/13/first-steps-with-the-
feitian-epass2003-smart-token-in-os-x/)

~~~
jlgaddis
Oh wow, I had forgotten all about the ePass2003.

I've still got one here, but it's probably been 10 years since I've used it. I
haven't forgotten how much of a PITA it was to get it working, though!

------
i9wsu
Just update and use Use Ed25519 keys

resource of how to: [http://wolf-tm.com](http://wolf-tm.com)

------
nelsonmarcos
It's a long article written on April 1st. Please, don't say it's an april's
fool joke.

