The system will halve electricity costs.
/ фото CC BY-SA
Why a new architecture is needed
, by 2030 electronic devices will consume 40% of the world's generated energy. Approximately 20% of this will be attributed to the IT sector and data centers. According to European analysts, data centers already account for 1.4% of all electricity consumption. This number is expected .
A significant portion of electricity is consumed by SSDs. From 2012 to 2017, the share of solid-state drives in data centers . Although SSDs consume a third less power (), electricity bills in data centers remain substantial.
To reduce the energy consumption of solid-state drives in data centers, engineers from MIT have developed a new SSD storage architecture. It is called LightStore and allows the drives to connect directly to the data center network, bypassing the storage servers. According to the authors, the system will halve electricity costs.
How it works
LightStore is a key-value type flash storage, where user requests to the drives are represented as keys. These are then sent to a server that releases the data associated with that key.
System It includes an energy-efficient built-in processor, DRAM, and NAND memory. It is managed by a controller and special software. The controller handles the work with NAND arrays, while the software processes KV requests and stores key-value pairs. The software architecture is built on the basis of , which are used in many modern databases.
The architecture can be represented as follows:
The diagram shows the basic components of LightStore. A cluster of nodes operates with key-value pairs. Application servers connect to the system via adapters. They convert client requests (such as fread() from the POSIX API) into KV requests. The architecture also has separate adapters for , block (based on the BUSE module), and file storage.
By distributing requests, the adapter uses It is used in systems like Redis or Swift. Using the KV-query key, the adapter generates a hash key, the value of which determines the target node.
The capacity of the LightStore cluster scales linearly — simply connect additional nodes to the network. In some cases, it may be necessary to purchase new switches. However, developers equipped each node with extra slots for connecting NAND chips.
Architecture Potential
Engineers from MIT state that the bandwidth of the LightStore-based solution is 620 Mbps for 10-gigabit Ethernet. One node consumes 10 watts instead of the usual 20 watts (in SSD systems used by data centers today). Furthermore, the hardware occupies half the space.
Currently, developers are refining some aspects. For example, LightStore cannot handle range queries and small requests. These features will be added in the future, as LightStore utilizes LSM trees. Additionally, the system currently has a limited set of adapters — YCSB and block adapters are supported. In the future, LightStore will be able to process SQL queries and more.
Other Developments
In the summer of 2018, Marvell, a storage controller developer, introduced a new line of SSD controllers based on AI systems. The developers integrated NVIDIA deep learning accelerators into standard controllers for data centers and client applications. As a result, they created an autonomous architecture that consumes less energy compared to traditional SSD controllers. The company hopes that the system will find applications in edge computing, big data analytics, and IoT.
Recently, the line of Western Digital Blue drives was updated. In April, the developers introduced a solution — the WD Blue SSD based on SanDisk technologies, which WD acquired a year ago. The updated WD Blue SSDs are characterized by enhanced performance and energy efficiency. The architecture is built on the basis of the specification , which provides access to SSDs connected via PCI Express.
This specification improves the efficiency of SSDs under a high number of concurrent requests and accelerates data access. Additionally, NVMe standardizes the SSD interface — hardware manufacturers will no longer have to spend resources. for the development of unique drivers, connectors, and form factors.
Prospects
The SSD market for data centers is moving towards simplifying architectures, automating storage component operations, and enhancing energy efficiency. Engineers at MIT are addressing the latter challenge. The authors , that LightStore will become the industry standard for SSD storage in data centers. It can be assumed that, in the future, new and even more efficient architectures will emerge based on it.
Several materials from the First Blog about corporate IaaS:
Source: habr.com
