{"id":39304,"date":"2019-10-31T22:29:40","date_gmt":"2019-10-31T19:29:40","guid":{"rendered":"https:\/\/prohoster.info\/blog\/kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-seti\/"},"modified":"2019-10-31T22:29:40","modified_gmt":"2019-10-31T19:29:40","slug":"kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-seti","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/kak-aws-varit-svoi-elastichnye-servisy-masshtabirovanie-seti","title":{"rendered":"How AWS 'cooks' its elastic services. Scaling the network","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

The scale of the Amazon Web Services network consists of 69 zones across 22 regions worldwide: the USA, Europe, Asia, Africa, and Australia. Each zone contains up to 8 data centers (DCs), and each DC holds thousands or hundreds of thousands of servers. The network is built to account for all unlikely scenarios of outages. For example, all regions are isolated from each other, and availability zones are distributed several kilometers apart. Even if a cable is cut, the system will switch to backup channels, and data loss will amount to mere packets. More about the principles on which the network is built and its structure will be explained by Vasily Pantyukhin.<\/p>\n

\"How
\n
\nVasily Pantyukhin<\/b> started as a Unix admin in .ru companies, spent 6 years working with large hardware from Sun Microsystems, and preached datacenter-centricity at EMC for 11 years. He naturally evolved into private clouds, then moved to public ones. Now, as an architect at Amazon Web Services, he provides technical advice to help thrive and grow in the AWS cloud.<\/p>\n

In the previous part of the trilogy about the structure of AWS, Vasily delved into the architecture of physical servers and database scaling. Nitro cards, a custom hypervisor based on KVM, and the Amazon Aurora database \u2014 all this is covered in the article \"How AWS 'cooks' its elastic services. Scaling servers and databases<\/a><\/noindex>\". Read it to immerse yourself in the context, or watch video recording<\/a><\/noindex> the presentations.<\/p>\n

This part will discuss network scaling \u2014 one of the most complex systems in AWS. The evolution from a flat network to a Virtual Private Cloud and its architecture, internal services like Blackfoot and HyperPlane, the noisy neighbor problem, and finally \u2014 the scale of the network, backbone, and physical cables. All this is covered below.<\/p>\n

Disclaimer: everything below is Vasily's personal opinion and may not reflect the views of Amazon Web Services.<\/i>
\n<\/a><\/noindex><\/p>\n

Network Scaling<\/h2>\n

\nAWS cloud was launched in 2006. Its network was quite primitive \u2014 with a flat structure. The range of private addresses was shared among all tenants of the cloud. When launching a new virtual machine, you would randomly receive an available IP address from this range.<\/p>\n

\"How
\n
\nThis approach was simple to implement but fundamentally limited cloud usage. In particular, it made it quite challenging to develop hybrid solutions that merged private networks on-premises and in AWS. The most common issue was overlapping IP address ranges.<\/p>\n

\"How
\n<\/p>\n

Virtual Private Cloud<\/h3>\n

\nThe cloud has proven to be in demand. It's time to consider scalability and the ability to use it for tens of millions of tenants. A flat network has become the main obstacle. Therefore, we thought about how to isolate users from each other at the network level so that they could independently choose IP ranges.<\/p>\n

\"How
\n
\nWhat comes to mind first when you think of network isolation? Of course, VLAN<\/b> and VRF \u2014 Virtual Routing and Forwarding<\/b>.<\/p>\n

Unfortunately, this didn't work. VLAN ID is only 12 bits, which gives us just 4096 isolated segments. Even in the largest switches, a maximum of 1-2 thousand VRFs can be used. The combination of VRF and VLAN gives us only a few million subnets. This is definitely insufficient for tens of millions of tenants, each of whom should be able to use multiple subnets.<\/p>\n

Moreover, we simply cannot afford to buy the required number of large boxes, for example, from Cisco or Juniper. There are two reasons: it\u2019s incredibly expensive, and we do not want to become dependent on their development and patching policies.<\/p>\n

One conclusion is clear \u2013 we need to build our own solution.<\/p><\/blockquote>\n

\nIn 2009, we announced VPC<\/b> \u2014 Virtual Private Cloud<\/b>. The name took hold, and now many cloud providers use it as well.<\/p>\n

VPC is a virtual network SDN<\/b> (Software Defined Network). We decided not to invent special protocols at L2 and L3 levels. The network operates on standard Ethernet and IP. To transmit, the traffic of virtual machines is encapsulated in a wrapper of our own protocol. It specifies the ID that belongs to the VPC tenant.<\/p>\n

\"How
\n
\nIt sounds simple. However, several serious technical challenges need to be addressed. For example, where and how to store data on the mapping of virtual MAC\/IP addresses, VPC IDs, and corresponding physical MAC\/IP. At the scale of AWS, this is a huge table that must work with minimal latency when accessed. This is handled by the mapping service<\/b>, which is spread thinly across the entire network.<\/p>\n

In next-generation machines, encapsulation is performed by Nitro cards at the hardware level. In older instances, encapsulation and decapsulation are done through software.\u00a0<\/p>\n

\"How
\n
\nLet's understand how this works in general terms. Starting with the L2 level, suppose we have a virtual machine with IP 10.0.0.2 located on a physical server 192.168.0.3. It sends data to another virtual machine 10.0.0.3, which resides on 192.168.1.4. An ARP request is formed and reaches the network Nitro card. For simplicity, let's assume both virtual machines are within the same 'blue' VPC.<\/p>\n

\"How
\n
\nThe card replaces the source address with its own and forwards the ARP frame to the mapping service.<\/p>\n

\"How
\n
\nThe mapping service returns the information necessary for transmission over the physical L2 network.<\/p>\n

\"How
\n
\nThe Nitro card replaces the MAC in the physical network with the address in the VPC in the ARP response.<\/p>\n

\"How
\n
\nWhen data is transmitted, we wrap logical MAC and IP in a VPC envelope. We send all of this over the physical network using the respective IPs of the source and destination Nitro cards.<\/p>\n

\"How
\n
\nThe physical machine intended for the packet performs a check. This is to prevent the possibility of address spoofing. The machine sends a special request to the mapping service asking, 'From the physical machine 192.168.0.3, I've received a packet destined for 10.0.0.3 in the\u00a0<\/p>\n

\"How
\n
\nThe mapping service checks its resource mapping table and either allows or denies the passage of the packet. In all new instances, additional validation is embedded in the Nitro cards. It cannot be circumvented even theoretically. Therefore, spoofing resources in another VPC will not work.<\/p>\n

\"How
\n
\nThen, the data is sent to the virtual machine for which it is intended.\u00a0<\/p>\n

\"How
\n
\nThe mapping service also acts as a logical router for data transmission between virtual machines in different subnets. Conceptually, it is straightforward, and I won't delve into details.<\/p>\n

\"How
\n
\nThus, with each packet transmission, servers refer to the mapping service. How do we handle the inevitable delays? By caching.<\/b>, of course.<\/p>\n

The beauty of it is that it's not necessary to cache the entire massive table. The physical server hosts virtual machines from a relatively small number of VPCs. Information needs to be cached only about these VPCs. Data transmission to other VPCs in 'default' configuration is still not legitimate. If functionalities like VPC peering are used, information about the corresponding VPCs is additionally loaded into the cache.\u00a0<\/p>\n

\"How
\n
\nWe've got a handle on data transmission in the VPC.<\/p>\n

Blackfoot<\/h3>\n

\nWhat to do when traffic needs to be sent externally, for example, to the Internet or through a VPN? Here we are helped by Blackfoot<\/b> \u2014 an internal AWS service. It was developed by our South African team. Thus, the service is named after the penguin that lives in South Africa.<\/p>\n

\"How
\n
\nBlackfoot decapsulates traffic and handles it as required. Data is sent to the Internet as is.<\/p>\n

\"How
\n
\nData is decapsulated and re-encapsulated in an IPsec wrapper when using a VPN.<\/p>\n

\"How
\n
\nWhen using Direct Connect, traffic is tagged and sent to the appropriate VLAN.<\/p>\n

\"How
\n<\/p>\n

HyperPlane<\/h3>\n

\nThis is an internal flow control service. Many network services require control of data flow state<\/b>. For example, when using NAT, flow control must ensure that each pair of \"IP: destination port\" corresponds to a unique outgoing port. In the case of a load balancer, NLB<\/b> \u2014 Network Load Balancer<\/b>, the data flow must always be directed to the same target virtual machine. Security Groups act as a stateful firewall. They monitor incoming traffic and implicitly open ports for outgoing packet flows.<\/p>\n

\"How
\n
\nIn the AWS cloud, latency requirements are extremely high. Therefore, HyperPlane<\/b> it is critical for the functionality of the entire network.<\/p>\n

\"How
\n
\nHyperPlane is built on EC2 virtual machines. There is no magic here, just cleverness. The trick is that these are virtual machines with large RAM. The transactions are conducted solely in memory. This allows achieving latencies of just a few microseconds. Working with disk would kill all performance.\u00a0<\/p>\n

HyperPlane is a distributed system made up of a vast number of such EC2 machines. Each virtual machine has a throughput of 5 GB\/s. Across the entire regional network, this provides immense terabits of bandwidth and allows processing millions of connections per second.<\/b>.<\/p>\n

HyperPlane works only with streams. VPC packet encapsulation is completely transparent to it. Any potential vulnerability in this internal service will still not breach the isolation of the VPC. Security is managed at lower levels.<\/p>\n

Noisy neighbor<\/h3>\n

\nThere is also the issue of the noisy neighbor<\/b> \u2014 noisy neighbor<\/b>Assuming we have 8 nodes. These nodes handle traffic for all cloud users. It seems fine, and the load should be evenly distributed across all nodes. The nodes are very powerful, and overloading them is difficult.<\/p>\n

However, we design our architecture even considering unlikely scenarios.\u00a0<\/p>\n

A low probability does not mean impossibility.<\/p><\/blockquote>\n

\nWe can imagine a situation where one or more users generate too much load. All HyperPlane nodes are involved in handling this load, and other users may potentially feel some performance degradation. This undermines the cloud concept, where tenants cannot influence one another.<\/p>\n

\"How
\n
\nHow can we solve the noisy neighbor problem? The first thing that comes to mind is sharding. Our 8 nodes are logically divided into 4 shards with 2 nodes each. Now the noisy neighbor will only affect a quarter of all users, but significantly.<\/p>\n

\"How
\n
\nLet's approach it differently. We'll allocate just 3 nodes per user.\u00a0<\/p>\n

\"How
\n
\nThe trick is to assign nodes to different users randomly. In the image below, the blue user shares nodes with one of two other users \u2014 the green and orange ones.<\/p>\n

\"How
\n
\nWith 8 nodes and 3 users, the probability of the noisy neighbor overlapping with one of the users is 54%. This is the likelihood that the blue user will impact other tenants, but only with part of their load. In our example, this influence will not be noticeable to everyone, but only to a third of all users. This is already a decent result.<\/p>\n

The number of users who will overlap<\/p>\n

The probability in percentage<\/p>\n

0<\/p>\n

18%<\/p>\n

1<\/p>\n

54%<\/p>\n

2<\/p>\n

26%<\/p>\n

3<\/p>\n

2%<\/p>\n

Let's bring the situation closer to reality \u2014 take 100 nodes and 5 users spread across 5 nodes. In this case, none of the nodes will overlap with a probability of 77%.\u00a0<\/p>\n

The number of users who will overlap<\/p>\n

The probability in percentage<\/p>\n

0<\/p>\n

77%<\/p>\n

1<\/p>\n

21%<\/p>\n

2<\/p>\n

1,8%<\/p>\n

3<\/p>\n

0,06%<\/p>\n

4<\/p>\n

0,0006%<\/p>\n

5<\/p>\n

0,00000013%<\/p>\n

In a real situation, with a vast number of HyperPlane nodes and users, the potential impact of a noisy neighbor on other users is minimal. This method is called shuffle sharding<\/b> \u2014 . It minimizes the negative effect of node failures.<\/b>Many services are built on HyperPlane: Network Load Balancer, NAT Gateway, Amazon EFS, AWS PrivateLink, AWS Transit Gateway.<\/p>\n

The scale of the network<\/p>\n

Network Scale<\/h3>\n

\nNow let's talk about the scale of the network itself. As of October 2019, AWS offers its services in 22 regions<\/b>, with 9 more planned.<\/p>\n