🥇Best in class: the story behind the AES encryption standard

Since May 2020, official sales of WD My Book external hard drives supporting AES hardware encryption with a 256-bit key have started in Russia. Due to legal restrictions, previously such devices could be purchased only in foreign online electronics stores or on the "gray" market, but now anyone can get a secure drive with a proprietary 3-year warranty from Western Digital. In honor of this significant event, we decided to take a short digression into history and find out how Advanced Encryption Standard appeared and why it is so good compared to competing solutions.

For a long time, the official symmetric encryption standard in the United States was DES (Data Encryption Standard), developed by IBM and listed as Federal Information Processing Standards in 1977 (FIPS 46-3). The algorithm is based on developments obtained during a research project codenamed Lucifer. When the US National Bureau of Standards announced a competition on May 15, 1973 to create an encryption standard for government agencies, the American corporation entered the cryptographic race with the third version of Lucifer, which used the updated Feistel network. And along with other contestants, it failed: none of the algorithms submitted for the first competition met the strict requirements formulated by the NBS experts.

Of course, IBM could not just accept defeat: when the competition was restarted on August 27, 1974, the American corporation again submitted an application, presenting an improved version of Lucifer. This time, the jury didn't have a single complaint: having done a competent job on the bugs, IBM successfully eliminated all the shortcomings, so there was nothing to complain about. Having won a landslide victory, Lucifer changed his name to DES and was published in the Federal Register on March 17, 1975.

However, during open symposiums organized in 1976 to discuss a new cryptographic standard, DES was heavily criticized by the expert community. The reason for this was the changes made to the algorithm by NSA specialists: in particular, the key length was reduced to 56 bits (initially, Lucifer supported working with 64- and 128-bit keys), and the logic of the permutation blocks was changed. According to cryptographers, the "improvements" did not make sense and the only thing the National Security Agency was striving for by introducing modifications was to be able to freely view encrypted documents.

In connection with the above allegations, a special commission was created under the US Senate, the purpose of which was to verify the validity of the actions of the NSA. In 1978, a report was published following the results of the investigation, which stated the following:

representatives of the NSA participated in the finalization of DES only indirectly, while their contribution concerned only the change in the operation of the permutation blocks;
the final version of DES proved to be more resistant to cracking and cryptographic analysis than the original, so the changes made were justified;
a key length of 56 bits is more than enough for the vast majority of applications, because cracking such a cipher will require a supercomputer worth at least several tens of millions of dollars, and since ordinary attackers and even professional hackers do not have such resources, there is nothing to worry about.

The conclusions of the commission were partially confirmed in 1990, when the Israeli cryptographers Eli Biham and Adi Shamir, working on the concept of differential cryptanalysis, conducted a large study of block algorithms, among which was DES. The scientists concluded that the new permutation model was much more resistant to attacks than the original one, which means that the NSA really helped eliminate several holes in the algorithm.

Adi Shamir

At the same time, the limitation on the key length turned out to be a problem, and a very serious one at that, which was convincingly proved in 1998 by the public organization Electronic Frontier Foundation (EFF) as part of the DES Challenge II experiment conducted under the auspices of the RSA Laboratory. A supercomputer was built specifically for cracking DES, codenamed EFF DES Cracker, which was developed by John Gilmour, co-founder of EFF and head of the DES Challenge project, and Paul Kocher, founder of Cryptography Research.

Processor EFF DES Cracker

The system they developed was able to successfully pick up the key to the encrypted sample using a simple enumeration method in just 56 hours, that is, in less than three days. To do this, DES Cracker needed to check about a quarter of all possible combinations, which means that even under the most unfavorable set of circumstances, it will take about 224 hours to crack, that is, no more than 10 days. At the same time, the cost of the supercomputer, taking into account the funds spent on its design, amounted to only 250 thousand dollars. It is easy to guess that today it is even easier and cheaper to break such a cipher: not only has the hardware become much more powerful, but also thanks to the development of Internet technologies, a hacker does not have to buy or rent the necessary equipment at all - it is quite enough to create a botnet from PCs infected with a virus.

This experiment clearly demonstrated how obsolete DES is. And since at that time the algorithm was used as part of almost 50% of data encryption solutions (according to the same EFF), the question of finding an alternative was more acute than ever.

New challenges - new competition

In fairness, it should be said that the search for a replacement for the Data Encryption Standard began almost simultaneously with the preparation of the EFF DES Cracker: in 1997, the National Institute of Standards and Technology (NIST) of the United States announced the launch of an encryption algorithm competition designed to identify a new "gold standard" of crypto security. And if in the old days a similar event was held exclusively “for its own”, then, mindful of the unsuccessful experience of 30 years ago, NIST decided to make the competition completely open: any company and any individual could take part in it, regardless of location or location. citizenship.

This approach paid off even at the stage of selection of applicants: among the authors who applied for participation in the Advanced Encryption Standard competition were world-famous cryptologists (Ross Anderson, Eli Biham, Lars Knudsen), and small IT companies specializing in cybersecurity (Counterpane ), and large corporations (German Deutsche Telekom), and educational institutions (Ku Leuven, Belgium), as well as start-ups and small firms that few have heard of outside their countries (for example, Tecnologia Apropriada Internacional from Costa Rica).

Interestingly, this time NIST approved only two main requirements for participating algorithms:

the data block must have a fixed size of 128 bits;
the algorithm must support at least three key sizes: 128, 192, and 256 bits.

Achieving such a result was relatively simple, but, as they say, the devil is in the details: there were much more secondary requirements, and it was much more difficult to meet them. Meanwhile, it was on their basis that the NIST reviewers selected the contestants. Here are the criteria that the contenders had to meet:

the ability to withstand any cryptanalytic attacks known at the time of the competition, including attacks through third-party channels;
the absence of weak and equivalent encryption keys (equivalent means such keys that, although they have significant differences from each other, lead to identical ciphers);
encryption speed is stable and approximately the same on all current platforms (from 8 to 64-bit);
optimization for multiprocessor systems, support for parallelization of operations;
minimum requirements for the amount of RAM;
no restrictions for use in standard scenarios (as a basis for building hash functions, PRNG, etc.);
the structure of the algorithm should be reasonable and easy to understand.

The last point may seem strange, but if you think about it, it is not without meaning, because a well-structured algorithm is much easier to analyze, and it is also much more difficult to hide a “bookmark” in it, with the help of which a developer could gain unlimited access to encrypted data.

The call for applications for the Advanced Encryption Standard contest lasted a year and a half. In total, 15 algorithms took part in it:

CAST-256, developed by the Canadian company Entrust Technologies based on CAST-128, created by Carlisle Adams and Stafford Tavares;
Crypton, created by cryptologist Che Hoon Lim from the South Korean cybersecurity company Future Systems;
DEAL, the concept of which was originally proposed by the Danish mathematician Lars Knudsen, and later developed by Richard Outerbridge, who applied for participation in the competition;
DFC, a joint project of the Paris School of Education, the National Center for Scientific Research of France (CNRS) and the telecommunications corporation France Telecom;
E2, developed under the auspices of Japan's largest telecommunications company Nippon Telegraph and Telephone;
FROG, the brainchild of the Costa Rican company Tecnologia Apropriada Internacional;
HPC, invented by American cryptologist and mathematician Richard Schreppel of the University of Arizona;
LOKI97, created by Australian cryptographers Lawrence Brown and Jennifer Seberry;
Magenta, developed by Michael Jacobson and Klaus Huber for the German telecommunications company Deutsche Telekom AG;
MARS from IBM, in the creation of which Don Coppersmith, one of the authors of Lucifer, took part;
RC6 written by Ron Rivest, Matt Robshaw and Ray Sidney specifically for the AES competition;
Rijndael, created by Vincent Reimen and Johan Dahmen of KU Leuven;
SAFER+, developed by Cylink Corporation of California in cooperation with the National Academy of Sciences of the Republic of Armenia;
Serpent by Ross Anderson, Eli Biham and Lars Knudsen;
Twofish, developed by Bruce Schneier's research group based on the Blowfish cryptographic algorithm proposed by Bruce back in 1993.

According to the results of the first round, 5 finalists were determined, among which were Serpent, Twofish, MARS, RC6 and Rijndael. The jury members found flaws in almost each of the listed algorithms, except for one. Who turned out to be the winner? Let's extend the intrigue a bit and first consider the main advantages and disadvantages of each of the listed solutions.

MARS

In the case of the “god of war”, experts noted the identity of the procedure for encrypting and decrypting data, however, its advantages were limited to this. The IBM algorithm turned out to be surprisingly gluttonous, which made it unsuitable for working with limited resources. There were also problems with parallelization of calculations. To work effectively, MARS needed hardware support for 32-bit multiplication and rotation by a variable number of bits, which again imposed restrictions on the list of supported platforms.

MARS also turned out to be quite vulnerable to time and power attacks, had problems with on-the-fly key expansion, and its excessive complexity made it difficult to analyze the architecture and created additional problems at the practical implementation stage. In a word, against the background of other finalists, MARS looked like a real outsider.

RC6

The algorithm inherited some of the transformations from its predecessor, RC5, which was carefully studied earlier, which, combined with a simple and visual structure, made it completely transparent for experts and excluded the presence of "bookmarks". In addition, RC6 demonstrated record data processing speeds on 32-bit platforms, and the encryption and decryption procedures were implemented in it in absolutely identical ways.

However, the algorithm had the same problems as the above-mentioned MARS: there is a vulnerability to attacks through third-party channels, and performance dependence on support for 32-bit operations, as well as problems with parallel computing, key expansion, and demands on hardware resources. In this regard, he was not suitable for the role of the winner.

Twofish

Twofish turned out to be quite nimble and well-optimized for low-power devices, coped well with key expansion and offered several implementation options, which made it possible to fine-tune it to specific tasks. At the same time, “two fish” turned out to be vulnerable to attacks through third-party channels (in particular, in terms of time and power consumption), were not particularly friendly with multiprocessor systems, and were characterized by excessive complexity, which, by the way, also affected the key expansion speed.

Serpent

The algorithm had a simple and understandable structure, which greatly simplified its audit, was not particularly demanding on the hardware platform, had support for on-the-fly key expansion, and was relatively easy to modify, which favorably differed from its opponents. Despite this, Serpent was, in principle, the slowest of the finalists, besides, the procedures for encrypting and decrypting information in it were radically different and required fundamentally different approaches to implementation.

Rijndael

Rijndael turned out to be extremely close to the ideal: the algorithm fully met the requirements of NIST, while not inferior, but in terms of the totality of characteristics it noticeably surpassed its competitors. Reindal had only two weaknesses: a vulnerability to power consumption attacks on the key expansion procedure, which is a very specific scenario, and certain problems with on-the-fly key expansion (this mechanism worked without restrictions only for two contestants - Serpent and Twofish). In addition, according to experts, Reindal had a slightly lower margin of cryptographic strength than Serpent, Twofish and MARS, which, however, was more than offset by resistance to the vast majority of types of side-channel attacks and a wide range of implementation options.

WD My Book External Drives Supporting AES-256 Hardware Encryption

Due to the combination of high reliability and performance, Advanced Encryption Standard quickly gained worldwide recognition, becoming one of the most popular symmetric encryption algorithms in the world and included in many cryptographic libraries (OpenSSL, GnuTLS, Linux's Crypto API, etc.). Currently, AES is widely used in enterprise and consumer applications, and its support is implemented in a wide variety of devices. In particular, AES-256 hardware encryption is used in Western Digital external drives of the My Book family to ensure the protection of stored data. Let's take a closer look at these devices.

The WD My Book line of desktop hard drives includes six models of 4, 6, 8, 10, 12, and 14 terabyte capacities, allowing you to choose the device that best suits your needs. By default, external HDDs use the exFAT file system, which ensures compatibility with a wide range of operating systems, including Microsoft Windows 7, 8, 8.1 and 10, as well as Apple macOS version 10.13 (High Sierra) and higher. Linux OS users have the ability to mount the hard drive using the exfat-nofuse driver.

My Book is connected to a computer using a high-speed USB 3.0 interface, which is backwards compatible with USB 2.0. On the one hand, this allows you to transfer files at the highest possible speed, because the bandwidth of USB SuperSpeed is 5 Gb / s (that is, 640 MB / s), which is more than enough. At the same time, the backward compatibility feature provides support for almost any device released in the last 10 years.

Although My Book does not require any additional software to be installed thanks to its Plug and Play technology for automatically detecting and configuring peripherals, we still recommend using the proprietary WD Discovery software package that comes with each device.

The set includes the following applications:

WD Drive Utilities

The program allows you to get up-to-date information about the current state of the drive based on SMART data and check the hard drive for bad sectors. In addition, with the help of Drive Utilities, you can quickly destroy all the data saved on your My Book: in this case, the files will not only be erased, but also completely overwritten several times, so that it will not be possible to restore them after the procedure is completed.

WD-Backup

Using this utility, you can set up scheduled backups. It is worth saying that WD Backup supports Google Drive and Dropbox, while allowing you to choose any possible source-target combinations when creating a backup. Thus, you can set up automatic transfer of data from My Book to the cloud, or import the necessary files and folders from the listed services both to an external hard drive and to a local machine. In addition, you can synchronize with your Facebook social network account, which allows you to automatically create backup copies of photos and videos from your profile.

W.D. Security

It is with the help of this utility that you can restrict access to the drive with a password and manage data encryption. All that is required for this is to specify a password (its maximum length can be up to 25 characters), after which all information on the disk will be encrypted, and only those who know the passphrase will be able to access the saved files. For more convenience, WD Security allows you to create a list of trusted devices, connecting to which My Book will be unlocked automatically.

We emphasize that WD Security only provides a convenient visual interface for managing cryptographic protection, while data encryption is carried out by the external drive itself at the hardware level. This approach provides a number of important advantages, namely:

the hardware random number generator, and not the PRNG, is responsible for generating encryption keys, which helps to achieve a high degree of entropy and increase their cryptographic strength;
during the encryption and decryption procedure, cryptographic keys are not uploaded to the computer's RAM, nor are temporary copies of processed files created in hidden folders on the system drive, which helps to minimize the likelihood of their interception;
file processing speed does not depend on the performance of the client device;
after activation of protection, file encryption will be carried out automatically, "on the fly", without requiring additional actions from the user.

All of the above guarantees data security and allows you to almost completely eliminate the possibility of confidential information theft. With additional storage options, this makes My Book one of the best secure storage available on the Russian market.

Source: habr.com

Best in class: the history of the emergence of the AES encryption standard