In In the cycle 'Introduction to SSDs', we discussed the history of the emergence of disks. The second part will discuss the interfaces for interacting with storage devices.
Communication between the processor and peripheral devices occurs according to predefined agreements known as interfaces. These agreements regulate the physical and software level of interaction.
An interface is a set of means, methods, and rules for interaction between elements of a system.
The physical implementation of the interface influences the following parameters:
- the bandwidth of the communication channel;
- the maximum number of simultaneously connected devices;
- the number of errors that occur.
Disk interfaces are built on input-output ports, which is the opposite of memory-mapped I/O and does not take up space in the processor's address space.
Parallel and Serial Ports
By the method of data exchange, input-output ports are divided into two types:
- parallel;
- serial.
As the name suggests, a parallel port sends a machine word consisting of several bits at once. A parallel port is the simplest way to exchange data, as it does not require complex circuitry solutions. In the simplest case, each bit of the machine word is sent over its own signal line, while two control signal lines are used for feedback: Data ready and Data received.
At first glance, parallel ports scale well: more signal lines mean more bits are transmitted at once, and therefore higher bandwidth. However, due to the increased number of signal lines, interference occurs between them, leading to distortion of transmitted messages.
Serial ports are the opposite of parallel ones. Data is sent one bit at a time, which reduces the total number of signal lines but complicates the input-output controller. The transmitter controller receives a machine word at once and must send one bit at a time, while the receiver controller must in turn receive bits and store them in the same order.
The small number of signal lines allows for interference-free increases in message transmission frequency.
SCSI
Small Computer Systems Interface (SCSI) was introduced back in 1978 and was originally developed to unify devices of various profiles into a single system. The SCSI-1 specification provided for the connection of up to 8 devices (including the controller), such as:
- scanners;
- tape drives (streamers);
- optical drives;
- disk drives and other devices.
Initially, SCSI was named Shugart Associates System Interface (SASI), but the standardization committee would not approve a name in honor of the company, and after a brainstorming session, the name Small Computer Systems Interface (SCSI) emerged. The 'father' of SCSI, Larry Boucher, intended for the abbreviation to be pronounced as 'sexy', but read it as 'sсuzzy'. Subsequently, the pronunciation 'sсuzzy' became firmly established for this standard.
In SCSI terminology, the connected devices are divided into two types:
- initiators;
- target devices.
The initiator sends a command to the target device, which then sends a response back to the initiator. Initiators and target devices are connected to a common SCSI bus, whose throughput in the SCSI-1 standard is 5 MB/s.
The used 'common bus' topology imposes several restrictions:
- special devices called terminators are required at the ends of the bus;
- the bus bandwidth is shared among all devices;
- the maximum number of simultaneously connected devices is limited.
Devices on the bus are identified by a unique number called SCSI Target ID. Each SCSI unit in the system is represented by at least one logical device, which is addressed by a number unique within the physical device Logical Unit Number (LUN).
Commands in SCSI are sent in the form of command descriptor blocks (Command Descriptor Block, CDB), consisting of an operation code and command parameters. The standard describes more than 200 commands, divided into four categories:
- Mandatory — must be supported by the device;
- Optional — may be implemented;
- Vendor-specific — used by specific manufacturers;
- Obsolete — outdated commands.
Among the many commands, only three are mandatory for devices:
- TEST UNIT READY — checks device readiness;
- REQUEST SENSE — requests the error code of the previous command;
- INQUIRY — requests the basic characteristics of the device.
After receiving and processing the command, the target device sends a status code back to the initiator, describing the result of the execution.
Further enhancements to SCSI (SCSI-2 and Ultra SCSI specifications) expanded the list of commands used and increased the number of devices that could be connected to 16, while the data transfer speed over the bus reached 640 MB/s. Since SCSI is a parallel interface, increasing the data transfer rate was associated with a reduction in the maximum cable length, leading to usability issues.
With the Ultra-3 SCSI standard, support for 'hot swapping' was introduced – connecting devices while powered on.
The first known SSD with a SCSI interface is the M-Systems FFD-350, released in 1995. The drive was expensive and did not achieve widespread adoption.
Currently, parallel SCSI is not a popular disk connection interface, but the command set is still actively used in USB and SAS interfaces.
ATA / PATA
The PerformanceResourceTiming ATA (Advanced Technology Attachment), also known as PATA (Parallel ATA) was developed by Western Digital in 1986. The marketing name of the IDE standard (Integrated Drive Electronics) emphasized a key innovation: the drive controller was integrated into the drive rather than being on a separate expansion board.
The decision to place the controller inside the drive resolved several issues at once. First, it reduced the distance from the storage device to the controller, positively affecting the performance of the storage. Second, the integrated controller was specifically designed for a certain type of drive, and therefore was cheaper.
ATA, like SCSI, uses a parallel input-output method, which is reflected in the cables used. To connect disks using the IDE interface, 40-wire cables, also known as ribbon cables, are necessary. In later specifications, 80-wire ribbon cables are used, over half of which are grounding wires to reduce interference at high frequencies.
The ATA cable features between two to four connectors, one of which connects to the motherboard, and the others connect to the drives. When connecting two devices with one cable, one must be configured as Master, and the other as Slave. A third device can only be connected in 'read-only' mode.
The jumper position determines the role of each device. The terms Master and Slave in relation to the devices are somewhat inaccurate, as all devices connected to the controller are Slaves.
A notable innovation in ATA-3 is the introduction of Self-Monitoring, Analysis and Reporting Technology (S.M.A.R.T.). Five companies (IBM, Seagate, Quantum, Conner, and Western Digital) collaborated to standardize the technology for assessing the health of drives.
Support for solid-state drives appeared with the fourth version of the standard, released in 1998. This version of the standard allowed data transfer speeds of up to 33.3 MB/s.
The standard imposes strict requirements on ATA cables:
- the cable must be flat;
- the maximum length of the cable is 18 inches (45.7 centimeters).
Short and wide cables were cumbersome and hindered cooling. Increasing the transfer rate with each subsequent version of the standard became increasingly difficult, and ATA-7 addressed the issue radically: the parallel interface was replaced with a serial one. Following this change, ATA adopted the term Parallel and became known as PATA, while the seventh version of the standard was renamed Serial ATA. The version numbering for SATA began with one.
SATA
The Serial ATA (SATA) standard was introduced on January 7, 2003, and resolved the issues of its predecessor with the following changes:
- the parallel port was replaced with a serial one;
- the wide 80-wire cable was replaced with a 7-wire one;
- the 'shared bus' topology was replaced with 'point-to-point' connections.
Although the SATA 1.0 standard (SATA/150, 150 MB/s) was only slightly faster than ATA-6 (UltraDMA/130, 130 MB/s), moving to a serial data transfer method was the 'groundwork' for increasing speeds.
Sixteen signal lines for data transfer in ATA were replaced with two twisted pairs: one for transmission and the other for reception. SATA connectors were designed for greater durability against multiple reconnections, and the SATA 1.0 specification made 'hot swapping' possible.
Some pins on the drives are shorter than all the others. This is designed to support 'hot swapping'. During replacement, the device 'loses' and 'finds' lines in a pre-defined order.
A little more than a year later, in April 2004, the second version of the SATA specification was released. In addition to increasing the speed to 3 Gbit/s, SATA 2.0 introduced technology Native Command Queuing (NCQ). Devices that support NCQ can organize the order of execution of incoming commands independently to achieve maximum performance.
For the next three years, the SATA Working Group worked on improving the existing specification, and version 2.6 introduced compact Slimline and micro SATA (uSATA) connectors. These connectors are a smaller version of the original SATA connector and are designed for optical drives and small disks in laptops.
Although the bandwidth of the second-generation SATA was sufficient for hard drives, solid-state drives required more. In May 2009, the third version of the SATA specification was released, increasing the bandwidth to 6 Gbit/s.
Particular attention was paid to solid-state drives in the SATA 3.1 revision. The Mini-SATA (mSATA) connector was introduced for connecting solid-state drives in laptops. Unlike Slimline and uSATA, the new connector resembled PCIe Mini, though it was not electrically compatible with PCIe. In addition to the new connector, SATA 3.1 boasted the ability to queue TRIM commands along with read and write commands.
The TRIM command notifies the solid-state drive of data blocks that carry no useful load. Prior to SATA 3.1, executing this command resulted in cache clearing and a suspension of input/output operations before executing the TRIM command. This approach degraded disk performance during delete operations.
The SATA specification could not keep up with the rapid growth in access speeds of solid-state drives, leading to the emergence of a compromise in 2013 called SATA Express in the SATA 3.2 standard. Instead of doubling the SATA bandwidth again, developers utilized the widely used PCIe bus, whose speed exceeds 6 Gbit/s. Drives supporting SATA Express acquired their own form factor called M.2.
SAS
The SCSI standard, which is 'competing' with ATA, also evolved and just a year after the introduction of Serial ATA in 2004, transformed into a serial interface. The new interface is named Serial Attached SCSI (SAS).
Despite SAS inheriting the SCSI command set, the changes were significant:
- a serial interface;
- a 29-pin cable with power;
- point-to-point connections.
The SCSI terminology was also inherited. The controller is still called the initiator, and the connected devices are called targets. All target devices and the initiator form a SAS domain. In SAS, the bandwidth of the connection does not depend on the number of devices in the domain, as each device uses its dedicated channel.
The maximum number of simultaneously connected devices in a SAS domain exceeds 16,000 as per specification, and instead of SCSI ID, a World-Wide Name (WWN) is used for addressing.
WWN is a unique identifier of 16 bytes in length, akin to a MAC address for SAS devices.
Despite the similarity of SAS and SATA connectors, these standards are not fully compatible. However, a SATA drive can be connected to a SAS connector, but not vice versa. Compatibility between SATA drives and a SAS domain is ensured through the SATA Tunneling Protocol (STP).
The first version of the SAS-1 standard has a bandwidth of 3 Gbit/s, while the most recent, SAS-4, improved this figure by 7 times to 22.5 Gbit/s.
PCIe
Peripheral Component Interconnect Express (PCI Express, PCIe) is a serial interface for data transmission that emerged in 2002. The development was initiated by Intel and later transferred to a special organization, the PCI Special Interest Group.
The PCIe serial interface was no exception and became the logical continuation of parallel PCI, which is designed for connecting expansion cards.
PCI Express is significantly different from SATA and SAS. The PCIe interface has a variable number of lanes. The number of lanes corresponds to powers of two and ranges from 1 to 16.
The term 'lane' in PCIe refers not to a specific signal line, but to an individual full-duplex communication channel consisting of the following signal lines:
- receive+ and receive-;
- transmit+ and transmit-;
- four ground wires.
The number of PCIe lanes directly affects the maximum bandwidth of the connection. The modern PCI Express 4.0 standard allows achieving 1.9 GB/s per lane, and 31.5 GB/s when using 16 lanes.
The "appetites" of solid-state drives are growing very quickly. Both SATA and SAS are unable to increase their bandwidth fast enough to keep up with SSDs, which has led to the emergence of PCIe-connected SSDs.
Although PCIe Add-In cards are secured with screws, PCIe supports "hot swapping." Short PRSNT pins ensure that the card is fully seated in the slot.
PCIe-connected solid-state drives are regulated by a separate standard. Non-Volatile Memory Host Controller Interface Specification and come in various form factors, which we will discuss in the next part.
Remote drives
With the creation of large data storage systems, there emerged a need for protocols that allow connecting drives located outside the server. The first solution in this area was Internet SCSI (iSCSI), developed by IBM and Cisco in 1998.
The idea of the iSCSI protocol is simple: SCSI commands are "wrapped" in TCP/IP packets and transmitted over the network. Despite the remote connection, clients are given the illusion that the drive is connected locally. A Storage Area Network (SAN) based on iSCSI can be built on existing network infrastructure. Using iSCSI significantly reduces the costs of organizing a SAN.
iSCSI has a "premium" variant— Fibre Channel Protocol (FCP). A SAN using FCP is built on dedicated fiber-optic communication lines. This approach requires additional optical network equipment, but is characterized by stability and high bandwidth.
There are many protocols for sending SCSI commands over computer networks. However, there is only one standard that addresses the opposite task and allows sending IP packets over the SCSI bus— .
Most protocols for organizing SAN use SCSI command sets to manage drives, but there are exceptions, such as the simple ATA over Ethernet (AoE). The AoE protocol sends ATA commands in Ethernet packets, but drives are represented in the system as SCSI.
With the advent of NVM Express storage, the iSCSI and FCP protocols ceased to meet the rapidly growing demands of solid-state drives. Two solutions emerged:
- extending the PCI Express bus beyond the server;
- creating the NVMe over Fabrics protocol.
Extending the PCIe bus requires the creation of complex switching equipment but does not alter the protocol.
The NVMe over Fabrics protocol has become a strong alternative to iSCSI and FCP. NVMe-oF uses fiber-optic communication lines and the NVM Express command set.
DDR-T
The iSCSI and NVMe-oF standards address the task of connecting remote disks as if they were local, while Intel took a different approach by bringing the local disk closer to the processor. The choice was made to use DIMM slots, where the RAM is connected. The maximum bandwidth of the DDR4 channel is 25 GB/s, significantly exceeding the speed of the PCIe bus. This is how the Intel® Optane™ DC Persistent Memory solid-state drive was created.
To connect the drive in DIMM slots, a protocol was invented DDR-T, physically and electrically compatible with DDR4, but requiring a special controller that recognizes the difference between a memory stick and a drive. The access speed to the drive is slower than that of RAM but faster than NVMe.
The DDR-T protocol is only available with Intel® Cascade Lake generation processors or newer.
Conclusion
Almost all interfaces have come a long way from serial to parallel data transmission. The speeds of solid-state drives are rapidly increasing; just yesterday, solid-state drives were a rarity, and today, NVMe is no longer particularly surprising.
In our laboratory, you can independently test SSD and NVMe drives.
Only registered users can participate in the survey. , please.
Will NVMe drives soon replace classic SSDs?
55.5%Yes 100
44.4%No 80
180 users voted. 28 users abstained.
Source: habr.com
