🥇Best in its class: the history of the AES encryption standard

Since May 2020, official sales of WD My Book external drives, supporting hardware AES encryption with a 256-bit key, have begun in Russia. Due to legal restrictions, similar devices could previously only be purchased from foreign electronics online stores or the gray market, but now anyone wishing to acquire a secure drive with a 3-year warranty from Western Digital can do so. To commemorate this significant event, we decided to take a brief look into the history and understand how the Advanced Encryption Standard came to be and what makes it so advantageous compared to competing solutions.

For a long time, the official standard for symmetric encryption in the U.S. was DES (Data Encryption Standard), developed by IBM and added to the list of Federal Information Processing Standards in 1977 (FIPS 46-3). The algorithm was based on developments obtained during a research project code-named Lucifer. When, on May 15, 1973, the National Bureau of Standards announced a competition aimed at creating an encryption standard for government agencies, the American corporation entered the cryptographic race with the third version of Lucifer, which used an updated Feistel network. Along with other competitors, it faced failure: none of the algorithms submitted for the first competition met the stringent requirements set by the NBS experts.

Of course, IBM couldn't simply accept defeat: when the competition was restarted on August 27, 1974, the American corporation submitted an application again, presenting an improved version of Lucifer. This time, the jury had absolutely no complaints: after thoroughly addressing the shortcomings, IBM successfully eliminated all flaws, leaving nothing to criticize. Achieving a convincing victory, Lucifer was renamed to DES and was officially published in the Federal Register on March 17, 1975.

However, during open symposia organized in 1976 to discuss the new cryptographic standard, DES faced severe criticism from the expert community. This was due to changes made to the algorithm by NSA specialists: in particular, the key length was reduced to 56 bits (originally Lucifer supported 64- and 128-bit keys), and the logic of how permutation blocks worked was altered. According to cryptographers, these 'improvements' were meaningless, and the sole intent of the National Security Agency in implementing these modifications was to gain the ability to easily view encrypted documents.

In light of these allegations, a special commission was established in the U.S. Senate to investigate the justification for NSA's actions. In 1978, a report was published as a result of the investigation, which stated the following:

NSA representatives only indirectly contributed to the refinement of DES, and their input was limited to altering the way permutation blocks operated;
the final version of DES turned out to be more resistant to attacks and cryptographic analysis than the original, thus the changes made were justified;
a key length of 56 bits is more than sufficient for the vast majority of applications, as breaking such encryption would require a supercomputer costing at least several tens of millions of dollars, and since ordinary criminals and even professional hackers do not possess such resources, there is nothing to worry about.

The commission's conclusions were partially confirmed in 1990, when Israeli cryptographers Eli Biham and Adi Shamir, while working on the concept of differential cryptanalysis, conducted an extensive study of block algorithms, including DES. The researchers concluded that the new model of permutations was significantly more resistant to attacks than the original, indicating that the NSA indeed helped eliminate several flaws in the algorithm.

Adi Shamir

At the same time, the key length limitation proved to be a serious issue, as convincingly demonstrated in 1998 by the Electronic Frontier Foundation (EFF) during the DES Challenge II experiment conducted under the auspices of RSA Laboratory. A supercomputer, code-named EFF DES Cracker, was specifically built to crack DES. John Gilmore, a co-founder of EFF and project leader of the DES Challenge, along with Paul Kocher, the founder of Cryptography Research, worked on its creation.

Processor EFF DES Cracker

The system they developed was able to successfully guess the key for the encrypted sample using a simple brute-force method in just 56 hours, or less than three days. To achieve this, the DES Cracker needed to check about a quarter of all possible combinations, which means that even under the most unfavorable circumstances, it would take around 224 hours to crack, or no more than 10 days. The cost of the supercomputer, considering the resources expended on its design, amounted to only $250,000. It is not hard to guess that today cracking such encryption is even easier and cheaper: not only has hardware become vastly more powerful, but thanks to advancements in internet technologies, a hacker does not even need to buy or rent the necessary equipment—it's enough to create a botnet from infected PCs.

This experiment clearly demonstrated how morally outdated DES had become. And since at that time the algorithm was used in about 50% of data encryption solutions (according to the same EFF), the need for finding alternatives was more urgent than ever.

New challenges — new competition

To be fair, it should be noted that the search for a replacement for the Data Encryption Standard began almost simultaneously with the preparation of the EFF DES Cracker: the U.S. National Institute of Standards and Technology (NIST) announced in 1997 the launch of a competition for encryption algorithms aimed at identifying a new "gold standard" in cryptographic security. Unlike in the past, when similar events were held exclusively for a select few, NIST decided to make the competition completely open this time, allowing any company and individual to participate, regardless of their location or nationality.

This approach proved effective even at the candidate selection stage: among the authors who submitted bids for the Advanced Encryption Standard competition were world-renowned cryptographers (Ross Anderson, Eli Biham, Lars Knudsen), small IT companies specializing in cybersecurity (Counterpane), large corporations (the German Deutsche Telekom), educational institutions (the Catholic University of Leuven, Belgium), as well as startups and small firms that were not widely known outside their countries (for instance, Tecnologia Apropriada Internacional from Costa Rica).

Interestingly, this time NIST approved only two main requirements for participating algorithms:

the block of data must have a fixed size of 128 bits;
the algorithm must support at least three key sizes: 128, 192, and 256 bits.

Achieving such results was comparatively simple, but as the saying goes, the devil is in the details: there were many more secondary requirements, which were much harder to meet. Meanwhile, it was precisely on their basis that NIST reviewers conducted the selection of contestants. Here are the criteria that candidates had to meet to win:

the ability to withstand any cryptanalytic attacks known at the time of the competition, including side-channel attacks;
the absence of weak and equivalent encryption keys (equivalent keys are those that, while having significant differences from each other, result in identical ciphertexts);
the speed of encryption is stable and approximately the same across all relevant platforms (from 8 to 64 bits);
optimization for multiprocessor systems, support for parallel operation;
minimum requirements for RAM capacity;
no restrictions for use in standard scenarios (as a basis for building hash functions, PRNGs, etc.);
the algorithm structure should be justified and easy to understand.

The last point may seem odd, but upon reflection, it makes sense because a well-structured algorithm is much easier to analyze, and it's also much harder to hide a 'backdoor' in it that a developer could use to gain unrestricted access to encrypted data.

The application process for the Advanced Encryption Standard competition lasted a year and a half. A total of 15 algorithms participated:

CAST-256, developed by the Canadian company Entrust Technologies based on CAST-128, created by Carlisle Adams and Stafford Tavares;
Crypton, created by cryptologist Cheong Hoon Lim from the South Korean company Future Systems, involved in cybersecurity;
DEAL, the concept of which was initially proposed by Danish mathematician Lars Knudsen, and whose ideas were later developed by Richard Outerson, who submitted an entry to the competition;
DFC, a joint project of the Paris Higher School of Education, the National Center for Scientific Research of France (CNRS), and the telecommunications corporation France Telecom;
E2, developed under the auspices of Japan's largest telecommunications company Nippon Telegraph and Telephone;
FROG, the brainchild of the Costa Rican company Tecnologia Apropriada Internacional;
HPC, conceived by American cryptologist and mathematician Richard Schaepe from the University of Arizona;
LOKI97, created by Australian cryptographers Lawrence Brown and Jennifer Seberry;
Magenta, developed by Michael Jacobson and Klaus Huber for the German telecommunications company Deutsche Telekom AG;
MARS from IBM, which involved Don Coppersmith — one of the authors of Lucifer;
RC6, written by Ron Rivest, Matt Robshaw, and Ray Sidney specifically for the AES competition;
Rijndael, created by Vincent Rijmen and Joan Daemen from Leuven Catholic University;
SAFER+, developed by the California corporation Cylink in collaboration with the National Academy of Sciences of the Republic of Armenia;
Serpent, created by Ross Anderson, Eli Biham, and Lars Knudsen;
Twofish, developed by Bruce Schneier's research team based on the Blowfish cryptographic algorithm proposed by Bruce back in 1993.

At the end of the first round, 5 finalists were determined, including Serpent, Twofish, MARS, RC6, and Rijndael. The jury members found flaws in almost every algorithm mentioned, except for one. Who emerged as the winner? Let's prolong the intrigue a bit and first examine the key advantages and disadvantages of each of the outlined solutions.

MARS

In the case of the 'God of War', experts noted the identical procedure for encrypting and decrypting data; however, this is where its advantages ended. The IBM algorithm turned out to be surprisingly resource-intensive, making it unsuitable for operation under resource constraints. There were also issues with parallel computation. For MARS to work effectively, it required hardware support for 32-bit multiplication and rotation by a variable number of bits, which imposed limitations on the list of supported platforms.

MARS also proved to be quite vulnerable to timing and power consumption attacks, had issues with 'on-the-fly' key expansion, and its excessive complexity complicated the architecture analysis and created additional problems during practical implementation. In short, against the backdrop of other finalists, MARS appeared to be a true outsider.

RC6

The algorithm inherited some transformations from its predecessor, RC5, which had been thoroughly researched earlier, and its simple and clear structure made it completely transparent to experts, ruling out the presence of 'backdoors'. Moreover, RC6 demonstrated record data processing speeds on 32-bit platforms, and its encryption and decryption procedures were implemented identically.

However, the algorithm faced the same issues as the previously mentioned MARS: susceptibility to side-channel attacks, performance dependence on 32-bit operation support, as well as challenges with parallel computation, key expansion, and demand for hardware resources. Therefore, it was not suited for the role of the winner.

Twofish

Twofish proved to be quite fast and well-optimized for operation on low-powered devices, excellently managing key expansion and offering several implementation options, allowing for fine adaptation to specific tasks. At the same time, 'two fish' turned out to be vulnerable to side-channel attacks (in particular, timing and power consumption), did not particularly cooperate well with multiprocessor systems, and exhibited excessive complexity, which, by the way, also affected the speed of key expansion.

Serpent

The algorithm had a simple and understandable structure, which significantly simplified its auditing, was not very demanding on the power of the hardware platform, supported 'on-the-fly' key expansion, and was comparatively easily modified, which gave it an advantage over its competitors. Despite this, Serpent was generally the slowest of the finalists, and the processes of encrypting and decrypting information in it were fundamentally different and required fundamentally different approaches to implementation.

Rijndael

Rijndael was extremely close to ideal: the algorithm fully met NIST requirements, while not only matching but significantly outperforming its competitors in a comprehensive set of characteristics. There were only two weak points in Rijndael: vulnerability to power consumption attacks during key expansion, which is a rather specific scenario, and certain issues with 'on-the-fly' key expansion (this mechanism worked without restrictions only for two competitors — Serpent and Twofish). Additionally, experts estimate that Rijndael had slightly less cryptographic strength than Serpent, Twofish, and MARS, which, however, was more than compensated for by its resistance to the overwhelming majority of varieties of side-channel attacks and a wide range of implementation options.

WD My Book external drives with support for AES-256 hardware encryption

Thanks to a combination of high reliability and performance, the Advanced Encryption Standard quickly gained global recognition, becoming one of the most popular symmetric encryption algorithms worldwide and integrating into numerous cryptographic libraries (OpenSSL, GnuTLS, Linux’s Crypto API, etc.). Today, AES is widely used in both enterprise and consumer-level applications, and its support is implemented across a variety of devices. In particular, AES-256 hardware encryption is employed in Western Digital's My Book external drives to ensure the protection of stored data. Let's take a closer look at these devices.

The WD My Book desktop hard drive lineup includes six models with varying capacities: 4, 6, 8, 10, 12, and 14 terabytes, allowing you to choose a device that best suits your needs. By default, external HDDs use the exFAT file system, ensuring compatibility with a wide range of operating systems, including Microsoft Windows 7, 8, 8.1, and 10, as well as Apple macOS version 10.13 (High Sierra) and later. Linux OS users can mount the drive using the exfat-nofuse driver.

Connecting My Book to your computer is done via the high-speed USB 3.0 interface, which is backward compatible with USB 2.0. This allows for file transfers at the maximum possible speed, as the USB SuperSpeed bandwidth is 5 Gbps (equivalent to 640 MB/s), which is more than sufficient. At the same time, backward compatibility ensures support for virtually any devices produced in the last 10 years.

Although My Book does not require additional software installation thanks to Plug and Play technology, which automatically detects and configures peripherals, we still recommend using the proprietary WD Discovery software package that comes with each device.

The bundle includes the following applications:

WD Drive Utilities

This program allows you to obtain current information about the drive's status based on S.M.A.R.T. data and check the hard drive for bad sectors. Additionally, with Drive Utilities, you can quickly erase all data stored on your My Book: the files will not just be deleted, but completely overwritten multiple times, making recovery impossible after the procedure.

WD Backup

Using this utility, you can set up backup scheduling. It should be noted that WD Backup supports integration with Google Drive and Dropbox, allowing you to choose any combinations of 'source-target' when creating backups. Thus, you can configure automatic data transfer from My Book to the cloud or import necessary files and folders from the listed services to either an external hard drive or a local machine. Additionally, there is an option to sync with a Facebook account, enabling automatic backups of photos and videos from your profile.

WD Security

This utility allows you to restrict access to the drive with a password and manage data encryption. All that is required is to set a password (which can be up to 25 characters long), after which all information on the disk will be encrypted, and only those who know the passphrase will have access to the saved files. For added convenience, WD Security allows you to create a list of trusted devices that will automatically unlock My Book when connected.

It is important to emphasize that WD Security merely provides a convenient visual interface for managing cryptographic protection, while the data encryption is carried out by the external drive at the hardware level. This approach offers several significant advantages, namely:

the hardware random number generator is responsible for creating encryption keys, rather than a pseudo-random number generator, which helps achieve high entropy and enhance their cryptographic strength;
during the encryption and decryption process, cryptographic keys are not loaded into the computer's RAM, nor are temporary copies of processed files created in hidden folders on the system disk, which helps minimize the risk of interception;
the speed of file processing does not depend on the performance of the client device;
once protection is activated, file encryption will occur automatically, 'on the fly', without requiring additional actions from the user.

All of the above ensures data security and virtually eliminates the likelihood of theft of confidential information. Given the additional features of the storage device, this makes My Book one of the best secure storage solutions available in the Russian market.

Source: habr.com