The system will reduce energy costs by half.
/ photo CC BY-SA
Why a new architecture is needed
, by 2030, electronic devices will consume 40% of all energy generated on the planet. Approximately 20% of this volume will be in the IT sector and data centers. By European analysts, data centers are already "taking" 1,4% of all electricity. This is expected .
A significant part of the electricity is consumed by SSD storage. Between 2012 and 2017, the share of SSDs in data centers . Although SSDs consume a third less power () than HDDs, electricity bills remain large at the scale of data centers.
To reduce the power consumption of SSDs in data centers, MIT engineers have developed a new SSD storage architecture. It was called LightStore and allows you to connect drives directly to a network of data centers, bypassing storage servers. By authors, the system will reduce electricity costs by half.
How it works
LightStore is a key-value flash store that maps user requests to drives as keys. They are then sent to the server, which releases the data associated with that key.
System built-in energy efficient processor, DRAM and NAND memory. It is controlled by a controller and special software. The controller is responsible for working with NAND arrays, and the software is responsible for processing KV requests and storing key pairs. The software architecture is based on , which is used in many modern DBMS.
The architecture diagram can be represented as follows:
The diagram shows the basic components of LightStore. The node cluster works with key-value pairs. Application servers are connected to the system using adapters. They convert client requests (such as fread() from the POSIX API) into KV requests. The architecture also has separate adapters for , block (based on the BUSE module) and file storages.
When distributing requests, the adapter uses . It is used in systems like Redis or Swift. Using the key of the KV request, the adapter generates a hash key, the value of which determines the target node.
The capacity of a LightStore cluster scales linearlyβjust add more nodes to the network. In some cases, you may need to purchase new switches. However, the developers have provided each node with additional slots for connecting NAND chips.
The potential of architecture
MIT engineers say the LightStore-based solution has a throughput of 620 Mbps for 10 Gigabit Ethernet. A single node consumes 10 watts instead of the usual 20 watts (in SSD systems used by data centers today). In addition, the equipment takes up half the space.
Now the developers are finalizing some points. For example, LightStore can't handle range queries and small queries. These features will be added in the future as LightStore uses LSM trees. Also, the system still has a limited set of adapters - YCSB and block adapters are supported. In the future, LightStore will be able to process SQL queries, etc.
Other developments
In the summer of 2018, Marvell, a storage developer, introduced a new line of AI-based SSD controllers. Developers have implemented NVIDIA deep learning accelerators into standard controllers for data centers and client applications. As a result, they created a standalone architecture that consumes less power than classic SSD controllers. The company hopes that the system will find application in edge computing, big data analytics and IoT.
The Western Digital Blue line of drives has recently been updated. In April, the developers presented a solution - WD Blue SSD based on SanDisk technology, which WD acquired a year ago. The updated WD Blue SSDs offer improved performance and power efficiency. The architecture is built on the basis of the specification , which provides access to SSDs connected via PCI Express.
This specification improves the performance of SSDs with a high number of concurrent requests and speeds up data access. Additionally, NVMe allows you to standardize the SSD interface - more hardware manufacturers to develop unique drivers, connectors and form factors.
Prospects
The data center SSD market is moving towards simplifying architecture, automating storage components, and improving energy efficiency. The development of engineers from MIT solves the latter problem. Authors that LightStore will become the industry standard for SSD storage in the data center. And it can be assumed that in the future new, even more efficient architectures will appear on its basis.
A few materials from the First Enterprise IaaS Blog:
Source: habr.com