We register the procedure for emergency access to SSH hosts with hardware keys

In this post, we will develop a procedure for emergency access to SSH hosts using offline hardware security keys. This is just one of the approaches, and you can adapt it for yourself. We will store the SSH CA for our hosts on a hardware security key. This scheme will work on almost any OpenSSH, including SSH with SSO.

What is this all for? Well, this is a last resort. This is a backdoor that will allow you to access your server when for some reason nothing else helps.

Why use certificates instead of public/private keys for emergency access?

• Unlike public keys, certificates can have a very short validity period. You can generate a certificate valid for 1 minute or even 5 seconds. After this period, the certificate will become unusable for new connections. This is ideal for emergency access.
• You can create a certificate for any account on your hosts and, if necessary, send such "one-time" certificates to colleagues.

What you need

• Hardware security keys that support resident keys.
  Resident keys are cryptographic keys that are stored entirely within a security key. Sometimes they are protected by an alphanumeric PIN. The public part of the resident key can be exported from the security key, optionally along with the private key descriptor. Resident keys are supported, for example, by USB keys of the Yubikey 5 series. It is desirable that they are intended only for emergency access to the host. For this post, I will only use one key, but you should have an additional one for backup.
• A safe place to store these keys.
• OpenSSH version 8.2 or higher on your local machine and on the servers you want to access on an emergency basis. Ubuntu 20.04 ships with OpenSSH 8.2.
• (optional but recommended) A CLI tool for verifying certificates.

Prepare

First you need to create a certification authority, which will be located on the hardware security key. Paste the key and run:

$ ssh-keygen -t ecdsa-sk -f sk-user-ca -O resident -C [security key ID]

As a comment (-C) I indicated [email protected]so you don't forget which security key this CA belongs to.

In addition to adding the key to the Yubikey, two files will be generated locally:

1. sk-user-ca, a key descriptor that refers to the private key stored in the security key,
2. sk-user-ca.pub which will be the public key for your CA.

But don't worry, there's another private key stored on the Yubikey that can't be retrieved. So everything is safe here.

On hosts as root, add (if not already added) to your SSHD configuration (/etc/ssh/sshd_config) the following:

TrustedUserCAKeys /etc/ssh/ca.pub

Then on the host, add the public key (sk-user-ca.pub) to /etc/ssh/ca.pub

Reload the daemon:

# /etc/init.d/ssh restart

Now we can try to access the host. But first we need a certificate. Create a key pair to be associated with the certificate:

$ ssh-keygen -t ecdsa -f emergency

Certificates and SSH Pairs
Sometimes it's tempting to use a certificate as a substitute for a public/private key pair. But a single certificate is not enough to authenticate a user. Each certificate also has a private key associated with it. That's why we need to generate this "emergency" key pair before we issue ourselves a certificate. The important thing is that we show the signed certificate to the server, indicating the key pair for which we have the private key.

So public key exchange is still alive and well. It rolls even with certificates. Certificates simply save the server from having to store public keys.

Next, create the certificate itself. I need ubuntu user authorization in 10 minute interval. You can do it your way.

$ ssh-keygen -s sk-user-ca -I test-key -n ubuntu -V -5m:+5m emergency

You will be prompted to sign the certificate with your fingerprint. You can add additional usernames separated by commas like -n ubuntu,carl,ec2-user

That's it, now you have a certificate! Next, you need to specify the correct permissions:

$ chmod 600 emergency-cert.pub

After that, you can view the contents of your certificate:

$ step ssh inspect emergency-cert.pub

Here's what mine looks like:

emergency-cert.pub
        Type: [email protected] user certificate
        Public key: ECDSA-CERT SHA256:EJSfzfQv1UK44/LOKhBbuh5oRMqxXGBSr+UAzA7cork
        Signing CA: SK-ECDSA SHA256:kLJ7xfTTPQN0G/IF2cq5TB3EitaV4k3XczcBZcLPQ0E
        Key ID: "test-key"
        Serial: 0
        Valid: from 2020-06-24T16:53:03 to 2020-06-24T17:03:03
        Principals:
                ubuntu
        Critical Options: (none)
        Extensions:
                permit-X11-forwarding
                permit-agent-forwarding
                permit-port-forwarding
                permit-pty
                permit-user-rc

Here, the public key is the emergency key we created, and sk-user-ca is associated with the certificate authority.

We are finally ready to run the SSH command:

$ ssh -i emergency ubuntu@my-hostname
ubuntu@my-hostname:~$

1. You can now create certificates for any user on the host that your CA trusts.
2. You can remove emergency. You can save sk-user-ca, but you don't need to because it's also on the security key. You might also want to remove the original PEM public key from your hosts (eg in ~/.ssh/authorized_keys for the ubuntu user) if you used it for emergency access.

Emergency Access Action Plan

Paste the security key and run the command:

$ ssh-add -K

This will add the CA's public key and key handle to the SSH agent.

Now export the public key to make a certificate:

$ ssh-add -L | tail -1 > sk-user-ca.pub

Create a certificate with an expiration date, for example, no more than an hour:

$ ssh-keygen -t ecdsa -f emergency
$ ssh-keygen -Us sk-user-ca.pub -I test-key -n [username] -V -5m:+60m emergency
$ chmod 600 emergency-cert.pub

And now SSH again:

$ ssh -i emergency username@host

If your .ssh/config file is causing some connection problems, you can run ssh with the -F none option to bypass it. If you need to send a certificate to a colleague, the easiest and most secure option is Magic Wormhole. To do this, you need only two files - in our case, these are emergency and emergency-cert.pub.

What I like about this approach is the hardware support. You can put the security keys in the safe and they won't go anywhere.

As advertising

Epic servers - Is cheap VPS with powerful processors from AMD, CPU core frequency up to 3.4 GHz. The maximum configuration allows you to solve almost any task - 128 CPU cores, 512 GB RAM, 4000 GB NVMe. Join now!

Source: habr.com