foreword

Just a week ago, I wrote an essay on the topic indicated in the title and was faced with the fact that, let's say, there is not so much cognitive information on the network. Mostly dry facts and setup instructions. Therefore, I decided to slightly correct the text and post it as an article.

What is FTP

FTP (File Transfer Protocol) is a protocol for transferring files over a network. It is one of the basic Ethernet protocols. Appeared in 1971 and initially worked in DARPA networks. At the moment, like HTTP, file transfer is based on a model consisting of a set of protocols TCP/IP (Transmission Control Protocol/Internet Protocol). Defined in RFC 959.

The protocol defines the following:

How to check for errors
Data packaging method (if packaging is used)
How does the sending device indicate that it has finished the message
How does the receiving device indicate that it has received the message

Interaction between client and server

Let's take a closer look at the processes that occur during FTP operation. The connection is initialized by the user's protocol interpreter. The exchange is controlled via the control channel in the TELNET standard. FTP commands are generated by the user's protocol interpreter and passed to the server. Server responses are also sent to the user via the control channel. In general, the user has the ability to contact the server's protocol interpreter and means other than the user's interpreter.

The main feature of FTP is that it uses a dual connection. One of them is used to send commands to the server and happens by default on TCP port 21, which can be changed. The control connection exists as long as the client communicates with the server. The control channel must be open when transferring data between machines. If it is closed, the data transfer stops. Through the second there is a direct data transfer. It is opened every time a file is transferred between the client and the server. If several files are transferred at the same time, a separate transfer channel is opened for each of them.

FTP can operate in active or passive mode, which determines how the connection is established. In active mode, the client creates a control TCP connection with the server and sends its IP address and an arbitrary client port number to the server, after which it waits for the server to start a TCP connection with this address and port number. If the client is behind a firewall and cannot accept an incoming TCP connection, passive mode can be used. In this mode, the client uses a flow of control to send a PASV command to the server, and then receives its IP address and port number from the server, which are then used by the client to open a data stream from its arbitrary port.

It is possible that data can be transferred to a third machine. In this case, the user organizes a control channel with two servers and organizes a direct data channel between them. Control commands go through the user, and the data goes directly between the servers.

When transmitting data over a network, four representations of data can be used:

ASCII - used for text. The data is converted, if necessary, from the character representation on the sending host to "XNUMX-bit ASCII", and (again, if necessary) to the character representation of the receiving host, before transmission. In particular, line feed characters are changed. As a result, this mode is not suitable for files containing more than plain text.
Binary mode - the sending device sends each file byte by byte, and the receiver stores the byte stream upon receipt. Support for this mode has been recommended for all FTP implementations.
EBCDIC - used to transfer plain text between hosts in EBCDIC encoding. Otherwise, this mode is similar to the ASCII mode.
Local Mode - Allows two computers with identical settings to send data in native format without conversion to ASCII.

Data transfer can be carried out in any of three modes:

Streaming - data is sent as a continuous stream, freeing FTP from doing any processing. Instead, all processing is done by TCP. An end-of-file indicator is not needed, except for splitting the data into records.
Block mode - FTP splits data into several blocks (header block, number of bytes, data field) and then transfers them to TCP.
Compression mode - data is compressed by a single algorithm (usually run-length coding).

An FTP server is a server that provides the ability to use the file transfer protocol. It has certain features that distinguish it from ordinary web servers:

User authentication required
All operations are performed within the current session
Ability to perform various actions with the file system
A separate channel is used for each connection

FTP client is a program that allows you to connect to a remote server via FTP and also perform the necessary actions on it with file system elements. The client could very well be a browser, in the address bar of which one should enter an address that is the path to a specific directory or file on the remote server, according to the general URL flowchart:

ftp://user:pass@address:port/directory/file

However, using a web browser in this context will only allow you to view or download the files of interest. In order to take full advantage of all the advantages of FTP, specialized software should be used as a client.

FTP authentication uses a username/password scheme to grant access. The username is sent to the server with the USER command, and the password with the PASS command. If the information provided by the client is accepted by the server, the server will send an invitation to the client and the session will begin. Users can, if the server supports this feature, log in without providing credentials, but the server can only grant limited access for such sessions.

A host providing an FTP service can provide anonymous FTP access. Users typically log in with "anonymous" (may be case sensitive on some FTP servers) as their username. Although users are usually asked to provide their email address instead of a password, no verification is actually done. Many FTP hosts that provide software updates support anonymous access.

protocol scheme

The client-server interaction in an FTP connection can be visualized as follows:

Secure FTP

FTP was not originally conceived as secure, as it was intended for communication between several military installations and institutions. But with the development and spread of the Internet, the risk of unauthorized access has increased many times over. There was a need to protect servers from various kinds of attacks. In May 1999, the authors of RFC 2577 reduced the vulnerabilities to the following list of issues:

Bounce attacks
Spoof attacks
Brute force attacks
Packet capture, sniffing
Port stealing

Normal FTP does not have the ability to transfer data in encrypted form, as a result of which usernames, passwords, commands, and other information can, if desired, be easily intercepted by intruders. The usual solution to this problem is to use "secure", TLS-secure versions of the vulnerable protocol (FTPS), or another more secure protocol, like SFTP/SCP, provided with most implementations of the Secure Shell protocol.

FTPS

FTPS (FTP + SSL) is an extension of the standard file transfer protocol that adds to its basic functionality the creation of encrypted sessions using the SSL (Secure Sockets Layer) protocol. Today, protection is provided by its more advanced counterpart TLS (Transport Layer Security - transport layer protection).

SSL

The SSL protocol was proposed by Netscape Communications Corporation in 1996 to ensure the security and privacy of Internet connections. The protocol supports client and server authentication (authentication), is independent of applications, and is transparent to the HTTP, FTP, and Telnet protocols.

The SSL Handshake protocol consists of two steps: server authentication and optional client authentication. At the first stage, the server sends its certificate and encryption parameters in response to the client's request. The client then generates a master key, encrypts it with the server's public key, and sends it to the server. The server decrypts the master key with its private key and authenticates itself to the client by returning a message authenticated by the client's master key.

Subsequent data is encrypted and authenticated with keys derived from this master key. In a second, optional step, the server sends a request to the client, and the client authenticates to the server by returning the request with its own digital signature and a public key certificate.

SSL supports a variety of cryptographic algorithms. The RSA public key cryptosystem is used during the connection establishment. After the key exchange, many different ciphers are used: RC2, RC4, IDEA, DES, and TripleDES. It also uses MD5, an algorithm for creating a message digest. The syntax for public key certificates is described in X.509.

One of the important advantages of SSL is its complete software and platform independence. The protocol is developed on the principles of portability, and the ideology of its construction does not depend on the applications in which it is used. In addition, it is also important that other protocols can be transparently overlaid on top of the SSL protocol; either to further increase the degree of protection of target information flows, or to adapt the cryptographic capabilities of SSL to some other, quite specific task.

SSL connection

The secure channel provided by SSL has three main properties:

The channel is private. Encryption is used for all messages after a simple dialogue that serves to determine the secret key.
The channel is authenticated. The server side of the dialog is always authenticated, while the client side is optionally authenticated.
The channel is reliable. Message transport includes integrity checking (involving MAC).

Features of FTPS

There are two FTPS implementations that use different security methods:

The implicit method involves using the standard SSL protocol with session establishment before sending data, which, in turn, breaks compatibility with regular FTP clients and servers. For backward compatibility with clients that do not support FTPS, TCP port 990 is used for the control connection and 989 for data transfer. This preserves the standard port 21 for the FTP protocol. This method has been deprecated.
Explicit - Much more convenient, as it uses standard FTP commands, but encrypts the data on response, which allows you to use the same control connection for both FTP and FTPS. The client must explicitly request a secure data transfer from the server, and then approve the encryption method. If the client does not request a secure transfer, the FTPS server is free to either keep or close the unsecured connection. An identity negotiation and data security mechanism was added under RFC 2228 which includes a new FTP AUTH command. While this standard does not explicitly define security mechanisms, it does specify that a secure connection must be initiated by the client using the algorithm described above. If secure connections are not supported by the server, a 504 error code should be returned. FTPS clients can obtain information about the security protocols supported by the server using the FEAT command, however, the server is not required to disclose what levels of security it supports. The most common FTPS commands are AUTH TLS and AUTH SSL, which provide TLS and SSL security, respectively.

SFTP

SFTP (Secure File Transfer Protocol) is an application layer file transfer protocol that runs on top of a secure channel. Not to be confused with (Simple File Transfer Protocol), which has the same abbreviation. If FTPS is just an extension of FTP, then SFTP is a separate and unrelated protocol that uses SSH (Secure Shell) as its basis.

Secure Shell

The protocol was developed by one of the IETF groups called Secsh. The working documentation for the new SFTP protocol did not become an official standard, but began to be actively used for application development. Subsequently, six versions of the protocol were released. However, the gradual increase in functionality in it led to the fact that on August 14, 2006, it was decided to stop working on the development of the protocol in connection with the fulfillment of the main task of the project (development of SSH) and the lack of a sufficient expert level to move on to the development of a full-fledged remote file system protocol .

SSH is a network protocol that allows you to remotely control the operating system and tunnel TCP connections (for example, to transfer files). Similar in functionality to the Telnet and rlogin protocols, but unlike them, it encrypts all traffic, including transmitted passwords. SSH allows a choice of different encryption algorithms. SSH clients and SSH servers are available for most network operating systems.

SSH allows virtually any other network protocol to be transmitted securely in an insecure environment. Thus, you can not only work remotely on a computer through a command shell, but also transmit an audio stream or video over an encrypted channel (for example, from a webcam). SSH can also use compression of transmitted data for subsequent encryption, which is convenient, for example, for remote launch of X WindowSystem clients.

The first version of the protocol, SSH-1, was developed in 1995 by researcher Tatu Ulyonen from the Helsinki University of Technology (Finland). SSH-1 was written to be more private than the rlogin, telnet, and rsh protocols. In 1996, a more secure version of the protocol, SSH-2, was developed that is incompatible with SSH-1. The protocol gained even more popularity, and by 2000 it had about two million users. Currently, the term "SSH" usually refers to SSH-2, because. the first version of the protocol, due to significant shortcomings, is now practically not used. In 2006, the protocol was approved by the IETF working group as an Internet standard.

Two implementations of SSH are common: a private commercial one and a free one. The free implementation is called OpenSSH. By 2006, 80% of computers on the Internet were using OpenSSH. The private implementation is being developed by SSH Communications Security, a wholly owned subsidiary of Tectia Corporation, and is free for non-commercial use. These implementations contain almost the same set of commands.

The SSH-2 protocol, unlike the telnet protocol, is resistant to traffic sniffing attacks, but is not resistant to man-in-the-middle attacks. The SSH-2 protocol is also resistant to attacks by joining in the middle (English sessionhijacking), since it is impossible to join an already established session or intercept it.

To prevent man-in-the-middle attacks, when connecting to a host whose key is not yet known to the client, the client software shows the user a "key fingerprint" (English keyfingerprint). It is recommended to carefully check the “key mold” shown by the client software with the server key snapshot, preferably obtained via reliable communication channels or personally.

SSH support is implemented in all UNIX-like systems, and most of them have an ssh client and server as standard utilities. There are many implementations of SSH clients for non-UNIX operating systems as well. The protocol gained great popularity after the widespread development of traffic analyzers and methods for disrupting the operation of local networks, as an alternative to the insecure Telnet protocol for managing important nodes.

Communication using SSH

SSH requires an SSH server and an SSH client. The server listens for connections from client machines and, when a connection is established, performs authentication, after which it starts servicing the client. The client is used to log into a remote machine and execute commands.

Comparison with FTPS

The main thing that distinguishes SFTP from standard FTP and FTPS is that SFTP encrypts absolutely all commands, usernames, passwords and other confidential information.

Both FTPS and SFTP protocols use a combination of asymmetric algorithms (RSA, DSA), symmetric algorithms (DES/3DES, AES, Twhofish etc.), and a key exchange algorithm. For authentication, FTPS (or to be more precise, SSL/TLS over FTP protocol) uses X.509 certificates, while SFTP (SSH protocol) uses SSH keys.

X.509 certificates include a public key and some information about the owner's certificate. This information allows, on the other hand, to check the integrity of the certificate itself, the authenticity and the owner of the certificate. An X.509 certificate has a corresponding private key, which is usually stored separately from the certificate for security reasons.

The SSH key contains only the public key (the corresponding private key is stored separately). It does not contain any information about the owner of the key. Some SSH implementations use X.509 certificates for authentication, but they don't actually verify the entire certificate chain - only the public key is used (which makes such authentication incomplete).

Conclusion

The FTP protocol, of course, still plays an important role in the storage and distribution of information on the network, despite its venerable age. It is a convenient, multifunctional and standardized protocol. Many file archives have been built on its basis, without which technical work would not be so effective. In addition, it is easy to configure, and server programs and clients exist for almost all relevant and not so platforms.

In turn, its secure versions solve the problem of confidentiality of stored and transmitted data in the modern world. Both new protocols have their pros and cons and serve slightly different roles. In areas where a file archive is needed, it is preferable to use FTPS, especially if classic FTP has already been used there before. SFTP is less common due to its incompatibility with the old protocol, but it is more secure and has more functionality, as it is part of the remote control system.

List of sources

Source: habr.com

SFTP and FTPS protocols