Many have probably heard about Anycast. With this method of network addressing and routing, a single IP address is assigned to multiple servers on a network. These servers can even be located in data centers that are remote from each other. The idea of Anycast is that, depending on the location of the request source, the data is sent to the nearest (according to the network topology, more precisely, the BGP routing protocol) server. Thus, it is possible to reduce the number of network transitions (hop) and delay (latency).
Essentially, the same route is advertised from multiple data centers around the world. Thus, clients will be sent to the "best" and "nearest" based on BGP routes, the data center. Why is it Anycast? Why use Anycast instead of Unicast?
Unicast is really suitable for a site with a single web server and a moderate amount of traffic. However, if a service has millions of subscribers, then it usually uses many web servers, each with the same IP address. These servers are distributed geographically to serve requests optimally.
In this scenario, Anycast will improve performance (traffic is sent to the user with minimal delay), ensure service reliability (due to redundant servers) and load balancing - routing to multiple servers will effectively distribute the load between them, improving the speed of the site.
Operators offer customers various types of load balancing based on Anycast and DNS. Clients can specify IP addresses to which requests will be sent based on the geographic location of the site. This makes it possible to distribute user requests more flexibly.
Suppose there are several sites between which you need to distribute the load (users), for example, an online store with 100 requests per day or a popular blog. To limit the region from which users access a particular site, you can use the Geo Community option. It allows you to limit the region within which the operator will announce the route.
Anycast and Unicast: differences
Anycast is often used in applications such as DNS (Domain Name System) and CDN (Content Delivery Networks) to make routing decisions that improve network performance. Content delivery networks use Anycast because they deal with large amounts of traffic, and Anycast provides a number of advantages in this case (more on them below). In DNS, Anycast allows you to significantly increase the level of reliability and fault tolerance of the service.
In Anycast IP, when using BGP, there are multiple routes to a particular host. They are actually copies of hosts in multiple data centers used to establish lower latency connections.
So, in the Anycast network, the same IP address is advertised from different places, and the network decides where to send the user's request based on the "cost" of the route. For example, BGP is often used to determine the shortest data path. When a user sends an Anycast request, BGP determines the best route for available Anycast servers on the network.
Benefits of Anycast
Latency Reduction
Anycast systems are able to reduce latency when processing user requests, as they allow you to receive data from the nearest server. That is, users will always connect to the "nearest" (in terms of the routing protocol) DNS server. As a result, Anycast reduces communication time by reducing the network distance between client and server. This not only reduces latency, but also provides load balancing.
Speed
Since traffic is directed to the nearest node, and the latency in data transfer between the client and the node is reduced, the result will be an optimization of the delivery rate, no matter where the client requests information from.
Increased stability and fault tolerance
If multiple servers around the world use the same IP, then if one of the servers fails or goes down, traffic will be redirected to the nearest server. As a result, Anycast makes the service more resilient and provides better network access/latency/speed.
Thus, by having multiple servers constantly available to users, Anycast, for example, improves the stability of the DNS. If a host fails, user requests will be redirected to another DNS server without any manual intervention or reconfiguration. Anycast provides an almost transparent switch to other sites by simply removing the routes of the problematic site.
Load balancing
In the Anycast system, network traffic is distributed to different servers. That is, it acts as a load balancer, preventing any single server from receiving the bulk of the traffic. Load balancing can be used, for example, when there are several network nodes at the same geographic distance from the request source. In this case, the load is distributed among the nodes.
Reducing the impact of DoS attacks
Another feature of Anycast is DDoS resistance. DDoS attacks are unlikely to be able to bring down the Anycast system, since it would have to suppress all the servers in such a network with an avalanche of requests.
DDoS attacks often use botnets that can generate so much traffic that it overloads the attacked server. The advantage of using Anycast in this situation is that each server is able to "absorb" part of the attack, which reduces the load on a particular server. A denial of service attack is likely to be localized to the server and not affect the entire service.
High horizontal scalability
Anycast systems are well suited for services with high traffic volumes. If a service using Anycast requires new servers to handle the growing traffic, new servers can be added to the network to handle it. They can be placed on new or existing sites.
If there is a large increase in traffic in a particular location, then adding a server will help balance the load for that site. Adding a server at a new site will help reduce latency by creating a new shortcut for some users. Both methods also help improve the stability of the service as new servers become available on the network. Thus, if a server is overloaded, one can simply deploy another one in a location that will allow it to accept some share of the overloaded server's requests. This does not require any configuration by the clients.
This is the only way to serve terabits of traffic and a very large number of users when there are only a few 10 or 25 Gb / s ports on the server. 100 hosts with one IP address will make it possible to process terabit volumes of traffic.
Ease of configuration management
As noted above, an interesting use of Anycast is DNS. It is possible to place several different DNS servers in the network nodes, but use one DNS address. Depending on where the source is located, requests are routed to the nearest node. This provides some traffic balancing and redundancy in the event of a DNS server failure. Thus, instead of having to set up different DNS servers depending on where they are located, a single DNS server configuration can be propagated to all hosts.
Anycast networks can be configured to route requests not only based on distance, but also based on parameters such as server availability, number of established connections. or response time.
No special servers, networks or special components are required to use Anycast technology on the client side. But Anycast also has disadvantages. It is believed that its implementation is a complex task that requires additional equipment, reliable providers and proper traffic routing.
From a pure source to a beautiful far away
While Anycast directs users based on the least hops, it doesn't necessarily mean the least latency. Latency is a more complex metric because one hop can have more than ten.
Example: Intercontinental communications may involve a single hop with very high latency.
Anycast is mainly used for UDP based services like DNS. User requests are routed to the "best" and "nearest" data center based on BGP routes.
Example: A DNS client workstation with an Anycast DNS IP address of 123.10.10.10 performs DNS resolution for the nearest of three DNS name servers deployed using the same Anycast IP address. If either R1 or Server A fails, DNS client packets will automatically be forwarded to the next closest DNS server via R2 and R3. In addition, the route to our server A will be removed from the routing tables, preventing further use of this nameserver.
Deployment scenarios
There are two general schemes that are used to determine which server a user is connecting to:
- Anycast network layer. Connects the user to the nearest server. The network path from the user to the server is important here.
- Application Level Anycast. There are more calculated metrics in this scheme, including server availability, response time, number of connections, etc. It depends on the external monitor that provides network statistics.
CDN based on Anycast
Let's return now to the use of Anycast in content delivery networks. Anycast is certainly an interesting networking concept and is gaining more acceptance from next generation CDN providers.
CDN is a distributed network of servers that deliver content to end users with high availability and low latency. Content delivery networks play an important role today as the backbone of numerous online multimedia services, and consumers are becoming less tolerant of slow download speeds. Video and voice applications are particularly sensitive to jitter and network latency.
CDN unites all servers into one network and provides faster content loading. Sometimes it is possible to reduce the user's waiting time by 5-6 seconds. The goal of a CDN is to optimize delivery by serving content from the server closest to the end user. This is very similar to Anycast, where the closest server is selected based on the location of the end user. It would seem that every CDN provider will use Anycast by default, but in reality this is not the case.
Applications that use protocols such as HTTP/TCP rely on the connection being established. If a new Anycast node is chosen (for example, if the server fails), the service may be interrupted. This is why Anycast was previously recommended for connectionless services such as UDP and DNS. However, Anycast works well for connection-oriented protocols, for example, TCP works fine in Anycast mode.
Some CDN providers use Anycast-based routing, others prefer DNS-based routing: the nearest server is chosen depending on where the user's DNS server is located.
Hybrid and multi-data center infrastructures are another use case for Anycast. The Load Balancing IP address received from the provider allows you to distribute the load between the IP addresses of different customer services in the provider's data center. With any device addressing technology, it provides better performance under heavy traffic, fault tolerance, and helps to optimize response time with a large number of users.
In hybrid multi-data center infrastructures, you can distribute traffic across servers or even virtual machines on dedicated servers.
Thus, there is a huge choice of technical solutions for building infrastructure. You can also set up IP load balancing across multiple data centers using any device in the group to optimize site performance.
You can distribute traffic according to your own rules, defining the "weight" of each of the distributed servers in each data center. This configuration is especially useful when there is a distributed server park, and the performance of services varies. This will allow more frequent distribution of traffic to improve server performance.
To create a monitoring system using the ping command, it is possible to configure probes. This allows the administrator to define their own control procedures and gain a clearer view of the status of each component in the infrastructure. In this way, accessibility criteria can be defined.
It is possible to build a hybrid infrastructure: sometimes it is convenient to leave the back office in the corporate network, and outsource the interface part to the provider.
It is possible to add SSL certificates for load balancing, encryption of transmitted data and security of communication between site visitors and corporate infrastructure. In the case of load balancing between data centers, SSL can also be used.
Anycast service with address load balancing can be obtained from your ISP. This feature will help improve the way users interact with apps based on location. It is enough to announce what services are available in the data center, and the traffic will be redirected to the nearest infrastructure. If there are dedicated servers, for example in France or North America, then the clients will be directed to the nearest server on the network.
One of the options for using Anycast is the optimal choice of the point of presence of the operator (PoP). Let's bring . LinkedIn (blocked in Russia) seeks not only to improve the performance and speed of its products - mobile and web applications, but also to improve the network infrastructure for faster content delivery. For this dynamic content delivery, LinkedIn makes heavy use of PoPs – Points of Presence. Anycast is used to direct users to the nearest PoP.
The reason is that in the case of Unycast, each LinkedIn PoP has a unique IP address. Users are then assigned PoP based on their geographic location using DNS. The problem is that when using DNS, about 30% of users in the United States were redirected to a sub-optimal PoP. Thanks to the phased introduction of Anycast, sub-optimal PoP assignment dropped from 31% to 10%.
The results of the pilot test are shown in the graph, where the y-axis is the percentage of optimal PoP assignment. As Anycast "scaled up" in many US states, there was an improvement in the percentage of traffic towards the optimal PoP.
Anycast network monitoring
In theory, Anycast networks are simple: multiple physical servers are assigned the same IP address, which BGP uses to determine the route. But the implementation and design of Anycast platforms is complex, especially for fault-tolerant Anycast networks. Even more difficult is effective monitoring of the Anycast network to quickly identify and isolate faults.
If services use a third-party CDN provider to serve their content, it is very important for them to monitor and verify network performance. Anycast CDN monitoring focuses on measuring end-to-end latency and penultimate hop characteristics to understand which data center is serving the content. Parsing HTTP server headers is another way to determine where the data is coming from.
Example: HTTP response headers indicating the location of the CDN server.
For example, CloudFlare uses its own CF-Ray header in HTTP Response messages, which includes an indication of the data center to which the request was made. In the case of Zendesk, the CF-Ray header for the Seattle region is CF-RAY: 2a21675e65fd2a3d-SEA and for Amsterdam it is CF-RAY: 2a216896b93a0c71-AMS. You can also use the HTTP-X headers from the HTTP response to determine where the content is located.
Other addressing methods
There are other addressing methods for routing user requests to a specific network endpoint:
unicast
Most of the internet today uses this method. Unicast - unicast transmission, the IP address is associated with only one specific node on the network. This is called one-to-one matching.
Multicast
Multicast uses a one-to-many-of-many or many-to-many relationship. Multicasting allows you to send a request from the sender simultaneously to different selected endpoints. This gives the client the ability to download a file in chunks from multiple hosts at the same time (which is useful for audio or video streaming). Multicast is often confused with Anycast. However, the main difference is that Anycast directs the sender to one specific node, even if multiple nodes are available.
Broadcast
A datagram from a single sender is routed to all endpoints associated with the broadcast address. The network automatically replicates the datagrams to be able to reach all recipients in the broadcast (usually on the same subnet).
geocast
Geocast is somewhat similar to Multicast: requests from a sender are sent simultaneously to multiple endpoints. However, the difference lies in the fact that the addressee is determined by its geographical location. This is a specialized form of multicasting used by some mobile peer-to-peer routing protocols.
Geo Router calculates its service area and approximates it. Georouters, exchanging service areas, build routing tables. The system of georouters has a hierarchical structure.
Unicast, Multicast and Broadcast.
The use of Anycast technology improves the reliability, resiliency and security of the DNS. Using this technology, operators offer their customers various types of DNS-based load balancing services. In the control panel, you can specify the IP addresses to which requests will be sent depending on the geographic location. This will give customers the ability to distribute user requests more flexibly.
Some carriers use per-point-of-presence (POP) route monitoring: the system automatically analyzes the shortest local and global routes for POPs and routes them through the lowest latency geographic locations with zero downtime.
At the moment, Anycast is the most stable and reliable solution for building high-load DNS services that have high requirements for stability and reliability.
The .ru domain supports 35 Anycast DNS servers grouped into 20 nodes distributed across five Anycast clouds. In this case, the principle of building on a geographical basis is used, i.e. geocast. When placing DNS nodes, it is planned to move them to geographically dispersed locations close to the most active users, the maximum concentration of Russian providers at the node location point, as well as the availability of free capacities and ease of interaction with the site.
How to build a CDN?
CDN is a network of servers that speeds up the delivery of content to users. unites all servers into one network and provides faster content loading. The distance from the server to the user plays an important role in download speed.
CDN allows you to use servers that are closest to the target audience. This reduces waiting time, helps speed up the loading of site content for all visitors, which is especially critical for sites with large files or multimedia services. Typical applications for CDNs are e-commerce and entertainment.
The network of additional servers created in the CDN infrastructure, which are located as close as possible to users, contributes to more stable and faster data delivery. According to statistics, the use of a CDN reduces the delay when accessing a site by more than 70% compared to sites without a CDN.
Как? Setting up a CDN using your own Anycast solution can be quite expensive, but there are cheaper options. For example, you can use GeoDNS and regular servers with unique IP addresses. With GeoDNS services, you can create a geolocation-enabled CDN where decisions are made based on the real location of the visitor rather than the location of the DNS resolver. You can set up your DNS zone to show US server IP addresses to US visitors, while European visitors will see an IP address from Europe.
With GeoDNS, you can return different DNS responses depending on the user's IP address. To do this, the DNS server is configured to return different IP addresses depending on the source IP address in the request. Typically, the GeoIP database is used to determine the region from which the request is made. Geolocation using DNS allows you to send content to users from the nearest site.
GeoDNS defines the IP address of the client that sent the DNS request, or the IP of the provider's recursive DNS server, which is used when processing the client request. The country/region is determined by the client's IP and GeoIP base. The client then gets the IP address of the nearest CDN server. Read more about configuring GeoDNS.
Anycast or GeoDNS?
While Anycast is a great way to deliver content on a global scale, it lacks specificity. This is where GeoDNS comes to the rescue. This service allows you to create rules that send users to unique endpoints based on their location.
Example: Users in Europe are routed to a different endpoint.
You can also deny access to domains by dropping all requests. This, in particular, is a quick way to cut off intruders.
GeoDNS gives more accurate answers than Anycast. If in the case of Anycast the shortest route is determined by the number of hops, then in GeoDNS, routing for end users occurs depending on their physical location. This reduces latency and improves accuracy when creating granular routing rules.
When switching to a domain, the browser accesses the nearest DNS server, which, depending on the domain, issues an IP address to load the site. Suppose an online store is popular in the United States and Europe, but there are DNS servers for it only in Europe. Then users from the US who want to use the services of the store will be forced to send a request to the nearest server, and since it is very far away, it will take a long time to wait for a response - the site will not load quickly.
When placing a GeoDNS server in the United States, users will already contact it. The response will be fast, which will affect the site loading speed.
In a situation with an existing US DNS server, when a user from the US navigates to this domain, he will turn to the nearest server, which will issue the desired IP. The user will be directed to the server that contains the content of the site, but since the servers with the content are far away, he will not get it quickly.
If you also place CDN servers with cached data in the US, then when loading, the client browser will send a request to the nearest DNS server, which will send back the correct IP address. The browser with the received IP contacts the nearest CDN server and the main server, and the CDN server sends the cached content to the browser. While the cached content is being loaded, the files missing to load the full site come from the main server. As a result, the site loading time is reduced, since much fewer files are sent from the main server.
Determining the exact location of a particular IP address is not always an easy task: there are many factors at play, and the owners of an IP address range may decide to announce it to the other side of the world (then you have to wait until the database is updated to get the correct location). Occasionally, VPS providers assign addresses that are believed to be in the US to VPS in Singapore.
Unlike using Anycast addresses, allocation is done at the time of name resolution, not at the time of connection to the cache server. If the recursive server does not support EDNS client subnets, then the location of that recursive server is used instead of the user who will connect to the cache server.
Client subnets in DNS is an extension of DNS (RFC7871) that defines how recursive DNS servers can send information about a client to a DNS server, specifically network information that a GeoDNS server can use to more accurately determine a client's location.
Most use their ISP's DNS servers, or DNS servers that are geographically close to them, but if someone in the US for some reason decides to use a DNS resolver located in Australia, they will most likely get an IP. server address closest to Australia.
If you want to use GeoDNS, it's important to be aware of these features, as in some cases it can increase the distance between the caching servers and the client.
Summary: If you want to combine multiple VPSs into a CDN, then the best deployment option is to use a DNS server bundle with the GeoDNS + Anycast feature out of the box.
Source: habr.com