{"id":39302,"date":"2019-10-31T22:29:40","date_gmt":"2019-10-31T19:29:40","guid":{"rendered":"https:\/\/prohoster.info\/blog\/kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-serverov-i-bazy-dannyh\/"},"modified":"2019-10-31T22:29:40","modified_gmt":"2019-10-31T19:29:40","slug":"kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-serverov-i-bazy-dannyh","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-serverov-i-bazy-dannyh","title":{"rendered":"How AWS 'cooks' its elastic services. Scaling servers and databases","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Clouds are like a magical box \u2014 you specify what you need, and resources simply appear out of nowhere. Virtual machines, databases, networks \u2014 all of this belongs exclusively to you. There are other tenants in the cloud, but in your universe, you are the sole ruler. You are confident that you will always receive the required resources, without having to share with anyone else, and you alone determine how the network will function. How does this magic work that allows the cloud to flexibly allocate resources while completely isolating tenants from one another?<\/p>\n

\"How
\n
\nAWS Cloud is an incredibly complex system that has been evolving since 2006. Part of this development was witnessed Vasily Pantyukhin<\/strong> \u2014 by an architect at Amazon Web Services. As an architect, he sees not only the end result but also the challenges that AWS overcomes. The more you understand how the system works, the more trust you build. Therefore, Vasily will share secrets about AWS cloud services. Below, you'll find information on the architecture of AWS physical servers, the elastic scalability of databases, the custom Amazon database, and methods for enhancing the performance of virtual machines while simultaneously reducing their costs. Understanding Amazon's architectural approaches will help you use AWS services more effectively and may even inspire new ideas for building your own solutions.
\n<\/a><\/noindex>
\nAbout the speaker: Vasily Pantyukhin (Hen<\/a><\/noindex>) began his career as a Unix admin in Russian companies, spent 6 years working with substantial Sun Microsystems hardware, and spent 11 years preaching data-centricity at EMC. He naturally evolved into private clouds and, in 2017, moved into public clouds. Now he provides technical advice to help organizations thrive and grow in AWS cloud.<\/p>\n

Disclaimer: everything below is Vasily's personal opinion and may not reflect the views of Amazon Web Services. The video recording<\/a><\/noindex> of the presentation on which this article is based is available on our YouTube channel.<\/i><\/p>\n

Why I discuss the structure of Amazon<\/h2>\n

\nMy first car had a \"stick\" \u2014 it had a manual transmission. It was amazing because it gave me the feeling that I could control the car completely. I also liked that I could at least roughly understand how it worked. Naturally, I pictured the mechanics of the gearbox rather simply \u2014 somewhat like a bicycle's transmission.<\/p>\n

\"How
\n
\nEverything was great, except for one thing \u2014 being stuck in traffic. It feels like you're sitting still and doing nothing, but you're constantly shifting gears, pressing the clutch, accelerating, and braking \u2014 it really wears you out. The problem of traffic jams was partly resolved when my family got a car with an automatic transmission. Driving now gives me time to think about something, listen to an audiobook.<\/p>\n

Another mystery entered my life because I completely stopped understanding how my car works. A modern vehicle is a complex device. The car adapts simultaneously to dozens of different parameters: gas pedal pressure, braking, driving style, and road quality. I no longer comprehend how this functions.<\/p>\n

When I started working with Amazon Cloud, it was also a mystery to me. But this mystery is an order of magnitude greater because in a car, there's one driver, while in AWS, there are millions. All users are steering, pressing the gas and brakes simultaneously. It's amazing that they go where they want \u2014 to me, it's a miracle! The system automatically adapts, scales, and flexibly adjusts to each user, making them feel like they are the only one in this Universe.<\/p>\n

The magic faded a bit when I later came to work as an architect at Amazon. I saw the problems we faced, how we solved them, and how we developed services. As my understanding of the system grew, so did my trust in the service. That\u2019s why I want to share a picture of what\u2019s under the hood of the AWS cloud.<\/p>\n

What we'll talk about<\/h2>\n

\nI chose a diversified approach \u2014 I selected 4 interesting services worth discussing.<\/p>\n

Server Optimization<\/strong>. Ephemeral clouds with physical embodiment: physical data centers where physical servers hum, heat up, and blink their lights.<\/p>\n

Serverless Functions <\/strong>(Lambda) \u2014 probably the most scalable service in the cloud.<\/p>\n

Database Scaling<\/strong>. I will explain how we build our own scalable databases.<\/p>\n

Network Scaling<\/strong>. The final part, where I will reveal the structure of our network. It\u2019s a wonderful thing \u2014 every cloud user thinks they are alone in the cloud and cannot see other tenants.<\/p>\n

Note. This article discusses server optimization and database scaling. Scaling the network will be addressed in the next article. What about serverless functions? A separate transcription has been released on this topic.Small but mighty. Unboxing the Firecracker micro-virtual machine.<\/a><\/noindex>It details several different scaling methods and thoroughly examines the Firecracker solution\u2014a blend of the best qualities of virtual machines and containers.<\/i><\/p><\/blockquote>\n

<\/p>\n

Servers<\/h2>\n

\nThe cloud is ephemeral. Yet, this ephemerality has a physical embodiment\u2014servers. Initially, their architecture was classical: standard x86 chipset, network cards, Linux, and the Xen hypervisor on which virtual machines ran.<\/p>\n

\"How
\n
\nIn 2012, such an architecture adequately met its tasks. Xen is an excellent hypervisor but has one major drawback. It has relatively high overhead for device emulation.<\/strong>With the arrival of new, faster network cards or SSDs, these overheads became too high. How to tackle this problem? The solution was to work on two fronts\u2014 optimizing both the hardware and the hypervisor.<\/strong>This is a very serious task.<\/p>\n

Optimizing hardware and the hypervisor.<\/h3>\n

\nDoing everything well at once is not feasible. What constitutes 'well' was initially unclear.<\/p>\n

We decided to adopt an evolutionary approach\u2014changing one key architectural element and deploying it into production.<\/p><\/blockquote>\n

We tread on all the rakes, listen to complaints and suggestions, and then change another component. This way, through small increments, we fundamentally change the entire architecture based on user feedback and support.<\/p>\n

Transformations began in 2013 with the most challenging aspect\u2014the network. In the C3<\/strong> instances, a special Network Accelerator card was added alongside the standard network card. It connected literally with a short loopback cable on the front panel. Unsightly, but in the cloud, it's not visible. However, direct interaction with the hardware fundamentally improved jitter and network bandwidth.<\/p>\n

Next, we focused on enhancing access to Elastic Block Storage (EBS)\u2014a combination of network and storage. The challenge was that while Network Accelerator cards existed in the market, there was no option to simply buy Storage Accelerator hardware. Therefore, we turned to the startup Annapurna Labs.<\/strong>, which developed specialized ASIC chips for us. They allowed remote EBS volumes to be connected as NVMe devices.<\/p>\n

In the instances C4<\/strong> , we addressed two tasks. First, we laid the groundwork for the promising but new NVMe technology at the time. Second, we significantly relieved the CPU load by transferring EBS request processing to a new card. It worked out well, so now Annapurna Labs is part of Amazon.<\/p>\n

By November 2017, we realized it was time to change the hypervisor itself.<\/p>\n

The new hypervisor was developed based on enhanced KVM kernel modules.<\/p><\/blockquote>\n

It fundamentally reduced overhead for device emulation and enabled direct interaction with new ASICs. The instances C5<\/strong> were the first virtual machines powered by the new hypervisor. We named it Nitro<\/strong>.<\/p>\n

\"HowEvolution of instances on the timeline.<\/em><\/p>\n

All new types of virtual machines that appeared since November 2017 operate on this hypervisor. Bare Metal instances have no hypervisor<\/strong>, but they are also called Nitro, as they use specialized Nitro cards.<\/p>\n

Over the next two years, the number of Nitro instance types exceeded several dozen: A1, C5, M5, T3, and others.<\/p>\n

\"How
\nTypes of instances.<\/em><\/p>\n

How modern Nitro machines are structured<\/h3>\n

\nThey have three main components: the Nitro hypervisor (mentioned earlier), a security chip, and Nitro cards.<\/p>\n

The security chip<\/strong> is integrated directly into the motherboard. It controls many important functions, such as host OS boot control.<\/p>\n

Nitro cards<\/strong> \u2014 there are four types. All of them were designed by Annapurna Labs and are based on common ASICs. Some of their firmware is also shared.<\/p>\n

\"How
\nFour types of Nitro cards.<\/em><\/p>\n

One of the cards is designed to work with the network.<\/strong>VPC<\/strong>It is recognized in virtual machines as a network card ENA \u2014 Elastic Network Adapter<\/strong>. It encapsulates traffic when transmitting it over the physical network (which will be discussed in the second part of the article), controls the Security Groups firewall, and handles routing and other networking tasks.<\/p>\n

Separate cards work with block storage EBS<\/strong> and disks embedded in the server. To the guest virtual machine, they appear as NVMe adapters.<\/strong>They are also responsible for data encryption and disk monitoring.<\/p>\n

The system of Nitro cards, hypervisor, and security chip is unified in a SDN network or Software Defined Network.<\/strong>The Control Plane of this network is managed by map controller<\/strong>.<\/p>\n

Of course, we continue developing new ASICs. For example, at the end of 2018, we released the Inferentia chip, which allows for more efficient handling of machine learning tasks.<\/p>\n

\"How
\nInferentia Machine Learning Processor chip.<\/em><\/p>\n

Scalable database<\/h2>\n

\nA traditional database has a layered structure. To simplify, the following levels can be distinguished.<\/p>\n