Little tricks with SSH

This article gathers our best techniques for more effective use of SSH. From it, you will learn how to:

• Add a second factor to your SSH login
• Use agent forwarding securely
• Exit a stuck SSH session
• Keep a permanent terminal open
• Share a remote terminal session with a friend (without Zoom!)

Adding a second factor to your SSH

You can add a second authentication factor to your SSH connections in five different ways:

1. Update your OpenSSH and use a security key. In February 2020, OpenSSH added support for FIDO U2F (Universal Second Factor) security keys. This is a great new feature, but there is a catch: only those clients and servers that have been updated to OpenSSH version 8.2 and above will be able to use security keys, as the February update introduces new key types for them. You can check the client version of SSH with the command ssh -V the server version can be checked with the command nc [servername] 22
   
   The February version added two new key types—ecdsa-sk and ed25519-sk (along with their corresponding certificates). To generate the key file, simply insert your security key and run the command:

   $ ssh-keygen -t ecdsa-sk -f ~/.ssh/id_ecdsa_sk

   This will create a public and private key and bind them to your U2F device. The role of the private key on the U2F device is to decrypt the secret key descriptor on disk when the security key is activated.

   Additionally, as a secondary factor, you can provide a passphrase for your keys.

   The resident key is another supported OpenSSH type of generating -sk keys. With this approach, the descriptor is stored on the U2F device and allows you to have it with the security key whenever needed. You can create a resident key with the command:

   $ ssh-keygen -t ecdsa-sk -O resident -f ~/.ssh/id_ecdsa_sk

   Then, to return the descriptor back to memory on a new device, insert the security key and run the command:

   $ ssh-add -K

   When connecting to the host, you will still need to activate the security key.

2. Use PIV+PKCS11 and Yubikey. To connect to devices with earlier versions of SSHD using a security key, a different approach is required. Yubico has a guide on using U2F+SSH with PIV/PKCS11 (English). This is not the same as FIDO U2F, and although the method works, it takes considerable effort to figure out what magic drives it.
3. Apply a custom yubikey-agent ssh agent. Filippo Valsorda wrote an SSH agent for Yubikeys. It is completely new and contains minimal features.
4. Use Touch ID and sekey. Sekey is an open-source SSH agent that stores secret keys in a secure enclave on your Mac and allows the use of Touch ID for access.
5. Use Single Sign On SSH. I wrote a tutorial to help set up this method (in English). One advantage of single sign on SSH is the ability to leverage your identity provider's security policies — including support for multi-factor authentication (MFA).

Secure use of agent forwarding

In SSH, agent forwarding allows a remote host to access the SSH agent of your local device. When you use SSH with agent forwarding enabled (typically through ssh -A), the connection has two channels: your interactive session and a channel for agent forwarding. Through this channel, the Unix socket created by your local SSH agent connects to the remote host. This is a risky method because a user with root access on the remote device can access your local SSH agent and potentially impersonate you on the network. Using the standard SSH agent from the Open SSH suite, you may not even realize it has happened. Having a U2F key (or Sekey) will help effectively block any attempts to exploit your SSH agent from the outside.

Even with such precautions, it is a good idea to use agent forwarding as infrequently as possible. Do not use it for every session — use agent forwarding only when you are certain it is necessary for the current session.

Exit a hung session

Network interruptions, uncontrolled program behavior, or a control sequence that blocks keyboard input — all of these are possible causes of an SSH session disconnecting.

You can end a hung session in several ways:

1. Automatically exit on network interruption. You need to add the following to your .ssh/config:

   ServerAliveInterval 5
   ServerAliveCountMax 1

   ssh will send echo to the remote host every ServerAliveInterval seconds to check the connection. If more than ServerAliveCountMax echoes do not receive a response, ssh will terminate the connection on timeout and exit the session.

2. Exit the session. SSH by default uses the symbol ~ (tilde) as a control character. The command ~. closes the open connection and returns you back to the terminal. (Control sequences can only be entered from a new line.) The command ~? will display a complete list of available commands in this session. Keep in mind that to type the symbol ~ on international keyboards, you may need to press the ~ key twice.

Why do frozen sessions happen at all? In the early days of the internet, computers rarely moved from one place to another. When using laptops and switching between multiple IPv4 WiFi networks, your IP address changes. Since SSH relies on TCP connections, which in turn rely on an endpoint with a stable IP address, each time you switch networks, your SSH connections lose the socket descriptor and effectively become lost. When your IP address changes, your network stack takes time to detect the loss of the descriptor. When network issues arise, we don't want one of the nodes in the TCP connection to terminate it too soon. Therefore, the protocol will attempt to resend data several times before finally giving up. Meanwhile, your session in the terminal will appear frozen. IPv6 adds several mobility-related features that allow a device to retain its home address during network changes. Perhaps one day this will no longer be such a problem.

How to keep a persistent terminal open on a remote node

There are two different approaches to keeping a connection alive when switching between different networks or wanting to disconnect for a while.

1. Use Mosh or Eternal Terminal

If you really need a connection that doesn't drop when you switch between networks, use the Mosh mobile shell. This is a secure shell that first uses an SSH handshake and then switches to its own encrypted channel for the duration of the session. Mosh thus creates a separate, highly resilient, and secure channel capable of withstanding internet interruptions, IP address changes of your laptop, significant network breaks, and more, all thanks to the magic of UDP connections and the Mosh synchronization protocol.

To use Mosh, you will need to install it on both the client and the server and open ports 60000-61000 for stateless UDP traffic to your remote host. After that, connecting is as simple as using mosh user@server.

Mosh operates at the level of screens and keystrokes, giving it a number of advantages over passing a binary stream of standard input and output between the SSH client and server. If we only need to synchronize screens and keystrokes, it becomes much easier to recover a dropped connection later. While SSH buffers and sends everything that happens, Mosh only needs to buffer keystrokes and synchronize the last frame of the terminal window with the client.

2. Use tmux

If you want to 'come and go as you please' and keep a terminal session on a remote node, use the terminal multiplexer tmux. I love tmux and use it constantly. If your SSH connection drops, you just need to reconnect and enter tmux attach. Additionally, it has wonderful features like in-terminal tabs and panes similar to tabs in the iOS terminal, and the ability to share terminals with others.

Some like to enhance their tmux with Byobu, a package that significantly improves the usability of tmux and adds many keyboard shortcuts. Byobu comes with Ubuntu and is easy to install on a Mac via Homebrew.

Sharing a remote terminal session with a friend

Sometimes, when debugging complex issues on your servers, you might feel the need to share an SSH session with someone who is not in the same room as you. tmux is perfect for such a task! Just a few simple steps are needed:

1. Make sure that tmux is installed on your bastion node or on any server you plan to work with.
2. Both of you will need to connect via SSH to the device using a single account.
3. One of you should start tmux to initiate the tmux session.
4. The other should run tmux attach.
5. Voilà! You have a shared terminal.

If you want more advanced multi-user tmux sessions, try tmate, which is a fork of tmux that significantly simplifies shared terminal sessions.

Source: habr.com