What is SSH encryption and how does it work

Secure Shell (SSH) is a commonly-implemented security protocol with a range of different uses. Its most renowned application allows users to securely access remote computers and servers, but it can also be used for tunneling, port forwarding, secure file transfers and more.

In this guide, we’ll cover what SSH is, what it is used for, the history of the protocol, its technical details, as well as the security issues that need to be taken into consideration.

SSH is made up of three separate protocols: the transport layer, the authentication layer and the connection layer. Together, these serve to authenticate the other party in the connection, provide confidentiality through encryption, and check the integrity of the data. SSH is now most commonly implemented as either the proprietary SSH-2, or as the open-source iteration, OpenSSH.

The uses of SSH

SSH is a versatile protocol. Its structure and security features allow it to be used in a number of ways, such as for remote access, port forwarding, tunneling and secure file transfers.

Remote access

Remote access gives users a way to log in to another computer or server from their own machine. It is used to access the target machine’s local files or perform services on it, all without having to physically be there.

Programs like Telnet and rlogin also have this functionality, but they lack the security features of SSH. The encryption and authentication measures involved in SSH allow users to connect to another server or computer in a protected manner, even over a potentially dangerous intermediate network.

Remote access with SSH is commonly implemented so that employees can work remotely, or to allow the IT department to accomplish tasks without having to physically go to the machine. It can be used for remote administration, network infrastructure management, to set up automation, create backups and more.

Port forwarding

Port forwarding is used to transfer requests from one address and port number to another set. It applies network address translation (NAT) to redirect ports between a local network and a remote computer, allowing you to access a device from outside of the network.

Port forwarding can be done in three different ways:

  • Local port forwarding – Local port forwarding allows you to connect your local client and an external network. It can be effective for doing things like accessing websites that are blocked locally, or for connecting to a database that is behind a firewall.
  • Remote port forwarding – This type of forwarding lets server-side applications access services on the client side. SSH’s remote port forwarding allows users to securely connect to remote servers through their local PC by redirecting a local port to a remote SSH server.
  • Dynamic port forwarding – This allows users to send their data through a particular port to a remote computer or server by using a number of SSH servers that are acting as proxies.

Tunneling

ssh-2

Tunneling protocols use encapsulation to move data between networks. Tunnels can be deployed to allow non-native protocols to run through networks that normally wouldn’t support them. Another common use is for providing security over an unsafe network.

Tunneling protocols wrap critical packets inside the payload of another packet. SSH tunneling allows users to get around network security, to link devices using a non-native network protocol, and to secure the data that is being transmitted. They are frequently used to connect remote users to their organization’s online resources in a secure manner.

SFTP

The SSH File Transfer Protocol (FTP), sometimes known as the Secure File Transfer Protocol provides a safe way to access, transfer and manage files. It is a secure alternative to FTP, and leverages the SSH protocol to securely send, receive and administer files.

SCP

The Secure Copy Protocol (SCP) is similar to SFTP, but more limited in its scope. It only allows secure file transfers, rather than the full set of features that allow SFTP to act as a remote file system protocol.

Platforms & applications that use SSH

Proprietary SSH or OpenSSH can be used on all major operating systems. It is available on Unix-based platforms like OpenBSD, macOS, Linux and Solaris, while Windows users can use SSH through PowerShell.

The history of SSH

SSH was developed at the Helsinki University of Technology in 1995 by Tatu Ylönen in response to a password-sniffing attack on the university’s network. It aimed to provide an alternative to protocols like FTP, TELNET, rsh and rlogin, which did not ensure confidentiality or authenticate users in a secure manner.

SSH was released for free to the public in 1995 and was well-received. Amid its rapid adoption, Ylönen founded SSH Communications Security by the end of the same year in order to continue the development and commercialize SSH.

In 1995, Ylönen also published an Internet Engineering Task Force (IETF) Internet Draft that documented the SSH-1 protocol. Limitations were soon found in the protocol, and these couldn’t be addressed without affecting backwards compatibility. The solution was a new version of the protocol, and SSH-2 was launched by Ylönen’s company in 1996.

SSH-2 featured new algorithms, which prompted the IETF to found a working group that aimed to standardize the protocol. The group was nicknamed SECSH, for Secure Shell, and it published its first Internet Draft for SSH-2 in 1997.

The software for SSH-2 was released in 1998, but it wasn’t immediately adopted in a widespread manner because of its more restrictive licensing. In 2006, an altered version of the protocol was made a standard by the IETF. This was more secure, using message authentication codes to check integrity and the Diffie-Hellman key exchange for authentication.

In 1999 the OpenBSD project released OpenSSH. OpenSSH is a free version of the protocol that is based on modifications that Björn Grönvall made to SSH 1.1.12. The developers went back to this older version and heavily altered it, because it was the last version of SSH that was completely open source. OpenSSH is now the most widely used option and it has since been implemented in a range of operating systems, such as Windows, macOS, Linux, Solaris and others.

SSH-1 vs SSH-2 vs OpenSSH

As noted above, SSH-1 is the first version of the protocol, which was originally released under an open-source license. It is considered insecure, and shouldn’t be implemented. This leaves the proprietary version, SSH-2, and the freely available version, OpenSSH, as viable alternatives.

SSH-2 and OpenSSH are essentially the same when it comes to their architecture and how they work. The main difference is that the proprietary version comes with a range of support options, while those that use OpenSSH need to rely on the resources created freely by the community.

SSH: The technical details

SSH-1 functioned as a single protocol, but we won’t go into it here since it is obsolete. Instead, we will focus on SSH-2 and OpenSSH, which are both made up of three separate protocols:

  • The transport protocol – This establishes the connection and provides the underlying security.
  • The authentication protocol – This layer is used to authenticate the client.
  • The connection protocol – This protocol handles the channels that data is transmitted over.

Each of these protocols serves a unique role that works towards establishing and securing a connection, authenticating the other party, and transferring data. The default TCP connection port is 22, and connections are set up between an SSH client and an SSH server along the client-server model.

The remote login process of SSH proceeds according to the following basic structure (with variations depending on the configuration), which we will cover in more detail later on:

  • The client contacts the SSH server to begin the connection.
  • The server then sends its public key to the client to authenticate its identity.
  • The two parties negotiate the parameters for the connection, then establish a secure channel along those lines.
  • The user then logs into the operating system of the server host and can now administer their tasks remotely.

Transport protocol

The transport layer is a low-level protocol that takes care of the following tasks.

  • Server host authentication
  • Key exchange
  • Encryption for data confidentiality
  • Integrity checks to verify that the data has not been altered
  • Establishing a session ID which can be used in the other protocols

The transport protocol only authenticates the server and not the client (client authentication is done in the authentication protocol if it is required).

In the transport layer, the connection is initiated by the client and the two parties then negotiate how the keys will be exchanged, which public key algorithm will be used, which symmetric-key cipher will encrypt the data, which message authentication algorithm will be used to check the data, and which compression method (if any) will be implemented.

Once the connection begins, both the server and the client need to send through an identification string, which includes the protocol version (2.0).

Algorithm negotiation

To set up the parameters of the connection, both sides send through a packet containing a list with the following options:

byte              SSH_MSG_KEXINIT

byte[16]      cookie (random bytes)
name-list    kex_algorithms
name-list    server_host_key_algorithms
name-list    encryption_algorithms_client_to_server
name-list    encryption_algorithms_server_to_client
name-list    mac_algorithms_client_to_server
name-list    mac_algorithms_server_to_client
name-list    compression_algorithms_client_to_server
name-list    compression_algorithms_server_to_client
name-list    languages_client_to_server
name-list    languages_server_to_client
boolean      first_kex_packet_follows
uint32         0 (reserved for future extension)

Each side lists the parameters that they are willing to accept in the connection, separated by commas. The preferred algorithm should be listed first.

For key exchange (kex_algorithms), the first algorithm that both parties support will be chosen for the connection (there may also be other factors that need to be met, depending on which algorithm has been chosen). If the two parties cannot find a mutually supported algorithm that satisfies these requirements, then the connection fails.

Server host key algorithms are the supported algorithms for the server host key. The server lays out the algorithms that it has host keys for, while the client specifies the algorithms that it is prepared to accept. The selection will depend on whether the key exchange method that was settled upon requires an encryption-capable host key or a digital signature

Both sides list the symmetric-key algorithms that they are willing to accept, with the preferred methods at the top. The first option to appear on the client’s list that also happens to be on the server’s list must be used. If no agreement can be made, the connection fails.

Both the MAC algorithm and the compression algorithm are negotiated in the same manner.

Key exchange

The key exchange is responsible for server authentication, and it sets up the keys that will be used to secure the connection in the following steps. It generally starts with the parties sending their lists of supported algorithms to one another. Alternatively, each side can guess the other side’s preferred algorithm and send a packet that fits that algorithm’s parameters at the start.

If one party’s guess is correct, that packet is used as the first key exchange packet. If neither guess is right, then each side must take a step back and send their lists of preferred algorithms. If the key exchange message includes the server’s digital signature as proof of the server’s legitimacy, it is considered explicit server authentication. If it uses the shared secret instead, it is referred to as implicit server authentication.

The key exchange is also responsible for establishing a shared secret and a hash. The hash value from the initial key exchange becomes the unique identifier for the session, and is also used as part of the digital signatures that prove that the party is the true owner of its private key.

The hash function that is used will depend on the key exchange method decided on in the negotiation. When the key exchange is completed, all future communications will use the new set of keys and algorithms.

According to an Internet Engineering Task Force (IETF) Internet Draft, the following key exchange methods are considered secure:

  • curve25519-sha256
  • curve448-sha512
  • diffie-hellman-group-exchange-sha256
  • diffie-hellman-group14-sha256
  • diffie-hellman-group15-sha512
  • diffie-hellman-group16-sha512
  • diffie-hellman-group17-sha512
  • diffie-hellman-group18-sha512
  • ecdh-sha2-nistp256
  • ecdh-sha2-nistp384
  • gss-group14-sha256
  • gss-group15-sha512
  • gss-group16-sha512
  • gss-group17-sha512
  • gss-group18-sha512
  • gss-nistp256-sha256
  • gss-nistp384-sha384
  • gss-nistp521-sha512
  • gss-curve25519-sha256
  • gss-curve448-sha512
  • rsa2048-sha256

Server host key algorithm

These public-key algorithms are used for server authentication as well as to securely establish the shared session ID. They can also be optionally used to authenticate the host. SSH is designed to work with a range of public key algorithms, encoding types and formats:

  • It uses public key algorithms for encryption and/or digital signatures.
  • A range of encoding methods can be implemented, allowing configuration with different data formats, padding and byte order.
  • Various key formats allow keys to be encoded in different ways, as well as a     range of certificate representations.

The default algorithms include the following, however there are some other variations that can also be implemented:

  • ssh-rsa
  • ssh-rsa-sha256
  • ssh-dss
  • ssh-dss-sha256
  • x509v3-sign-dss
  • x509v3-sign-dss-sha256
  • x509v3-sign-rsa
  • x509v3-sign-rsa-sha256

Encryption algorithms

Symmetric-key algorithms are used to encrypt the data and provide confidentiality. The parameters and shared key that are used in the encryption process are established in the earlier phases of the connection. The chosen algorithm encrypts the payload, the packet length, the padding length, and the padding fields.

A range of different encryption algorithms are accepted in SSH, but for security purposes, it is best to stick with AES. Keys should be a minimum of 128-bit, but larger keys are preferred.

MAC algorithms

The transport protocol verifies the integrity of the data by adding a message authentication code (MAC) to the packet. This MAC is based on the shared secret (which is established in the key exchange), the packet sequence number and the packet contents. It is calculated before encryption takes place.

Implementations need to offer an independent algorithm to run in each direction, although it is ideal if the same one is used on both sides. A wide variety of message authentication algorithms can be implemented, however SHA-256 and above should be used in most situations to ensure a high level of security.

Compression

Compression is not mandatory in the SSH protocol, and its implementations must allow connections to proceed without compression. Compression can only be implemented as an option, using schemes like zlib. If compression is used, it only affects the payload. The MAC and the packet length field are then calculated based on the compressed payload.

Transport protocol packet

The transport protocol packet is formatted to include the following information (as well as some less pertinent details which have been left out):

  • The packet length
  • The padding length
  • The payload
  • Padding
  • A message authentication code (MAC)

ssh-3

Authentication protocol

This protocol is used by the server to authenticate the client. It can do this with a variety of different mechanisms, many of which rely on the session ID established in the transport protocol. Some use the encryption and integrity checks from the transport protocol in conjunction with the session ID, while others use these algorithms by themselves.

The server uses its local policy to decide which methods of authentication it accepts from an individual user. Since the server has already been authenticated in the transport protocol, there is no need to authenticate the server once more.

The security of the authentication protocol is dependent on the transport protocol that it runs over the top of. If the transport protocol cannot guarantee confidentiality or check the integrity of the data, then this limits how the authentication protocol can be safely used.

As an example, if integrity protection isn’t being applied by the transport protocol, then requests such as password changes should not be allowed, because this would leave opportunities for attackers to tamper with the data without being noticed.

The authentication protocol uses public-key authentication under the assumption that neither the private key of the server host, nor the key of the client host have been compromised. If the server has been compromised, this can lead to the username and password of the client being released to the attacker.

For host-based authentication to be secure, the client must not be compromised. If this is a possibility, then other authentication methods should be added. It’s important to note that the authentication process is only as strong as the weakest exchange method that a server accepts.

The authentication protocol process

The authentication protocol begins when the server sends the client a list of its accepted authentication methods. The client can then choose from these methods in any order. This process gives control to the server, but also allows enough flexibility so that the client can arrange to use the most convenient method.

The most common client authentication methods include:

  • Public key – This method uses algorithms such as RSA, DSA and ECDSA to authenticate the client through public-key cryptography. Some implementations use x.509 certificates as well. The server checks the client’s digital signature against their public key to verify the client’s identity.
  • Password – Passwords can also be used to authenticate the client. The client sends through its password (which should be encrypted by the transport protocol). If the password matches the server’s stored value, it is accepted and the authentication moves forward.
  • GSSAPI – Under this method, external schemes such as Kerberos can be used for single sign-on.
  • Keyboard interactive – This technique provides one-time password     authentication by having the server prompt the client for information.

Connection protocol

The connection protocol sets out how multiple channels of data will be combined over the secure transport layer. It also deals with the parameters that are used to access secure subsystems on the server host, as well as proxy-forwarding and accessing shells.

The connection protocol sits on top of the transport layer and authentication protocols. It allows remote command execution, as well as forwarded X11 and TCP/IP connections. If either the server or the client has been compromised, then the connection protocol is not secure.

Channels

Channels are the basic communication paths, and they can be opened by either party. Channels can include terminal sessions, forwarded connections and other forms of communication. When there are multiple channels, they are multiplexed into one connection.

Opening channels

Each channel is numbered on both of its ends, although the numbers can potentially be different on either side. When one side requests to open a channel, it sends its channel number as part of the message, as well as information on the initial window size and the maximum packet size.

The initial window size indicates how much data the party that opens the channel can receive. If more data needs to be sent, the window has to be adjusted first. Likewise, the maximum packet size specifies just how large of a packet can be received.

When one side requests a channel to be opened, the other side will open the channel if it can accommodate it. If not, it will send through a failure message. Channels can fail to be opened for the following reasons:

  • Prohibited by the administration
  • Failed connection
  • Unknown channel type
  • Resource shortages

If either side of the connection wants to send more data than the window currently allows, they can send a message requesting to add more bytes.

Closing channels

Once one side of a connection has completed its data transmission, it should send a message indicating that it has finished using the channel. Despite this, the channel is kept open, and data can still be sent by the other party. If a party wants to completely terminate the channel, it does so with a separate termination message.

SSH security

The various versions of SSH have each had their own security issues, although current configurations of SSH-2 and OpenSSH are considered far safer than SSH-1.

SSH-1

SSH-1 is generally considered to be flawed, with a range of different vulnerabilities. These include a bug in SSH 1.5 that allowed unauthorized users to insert content into the SSH data stream. This attack took advantage of the CRC-32 algorithm’s minimal data integrity protection.

This attack was mitigated with the SSH Compensation Attack Detector, which was integrated into most newer implementations. This fix came with a new vulnerability, which had the power to execute arbitrary code with root privileges.

There is also a vulnerability that lets adversaries change the last block in a session that uses IDEA-encryption, as well as one that allows a compromised server to forward the client authentication process to a different server.

Because of these security issues, SSH-2 should be used instead. To keep your implementation secure, you should also disable renegotiation to SSH-1, because attacks can take advantage of this to access your data via SSH-1’s weaker level of protection.

SSH-2

SSH-2 is vulnerable to a theoretical attack against its default mode of encryption, CBC. It allows the attacker to recover up to 32 bits of the plaintext from an encrypted block. This can be mitigated by using Counter mode (CTR), and turning the block cipher into a stream cipher instead.

At the end of 2014, Der Spiegel released NSA documents that implied that the NSA could sometimes break SSH. This leaked NSA PowerPoint states that the NSA can “Potentially recover usernames and passwords”, although no further details are given. It is not known what methods the agency used to do this, but it seems unlikely that it would lie about its capabilities in its own internal documentation.

In 2017’s Vault 7 leak, it was revealed that the CIA had two tools that could be used to intercept and steal SSH logins and passwords. BothanSpy targeted Windows Xshell clients, while Gyrfalcon was used against against the OpenSSH client on a number of different Linux distributions.

These tools are capable of stealing credentials and then transmitting them back to a CIA server. Neither of these attacks can break the protocol itself; they just use other side channel attacks that can get around it in certain implementations.

Despite these attacks, SSH-2 is considered secure in most situations, as long as it is implemented appropriately. High-value targets or those who use outdated or poor implementations should consider other options.

OpenSSH

In OpenSSH version 2, an attack was discovered that took advantage of a weakness in the SSH binary packet. The attack allowed researchers from the University of London to recover 14 bits of the plaintext from an encrypted block. This was mitigated in release 5.2 by making the protocol read the entirety of an invalid packet length or message authentication code, rather than ending the connection.

In versions 6.8 and 6.9, Linux could be used to execute arbitrary commands on the terminals of other users. This was accomplished through a privilege escalation vulnerability that allowed attackers to inject characters with the TIOCSTI input/output control.

Is SSH safe?

While it may seem like SSH has a lot of security issues, it’s relatively normal for a number of vulnerabilities to be found in a protocol’s various implementations. This doesn’t mean that the SSH protocol is unsafe. Instead, it just means that it needs to be implemented correctly.

As long as you are using either SSH-2 or OpenSSH and it is configured in a way that is appropriate for your usage, you should feel confident in the protection that SSH provides your connection. That’s why it is still such a frequently used protocol, especially for remote access and tunneling purposes.

See also: Common encryption types explained

Cyber network technology background by TheDigitalArtist licensed under CC0