A BGP route leak has led to widespread connectivity issues on the internet

Cloudflare released Report on yesterday's incident, which caused three hours of access problems to many global network resources, including Cloudflare, Facebook, Akamai, Apple, Linode, and Amazon AWS. Issues in the Cloudflare infrastructure, which provides CDN for 16 million websites, were observed from 14:02 to 16:02 (MSK). According to Cloudflare, approximately 15% of global traffic was lost during the outage.

The problem was is caused by a route leak through BGP, during which around 20,000 prefixes for 2,400 networks were incorrectly redirected. The leak's source was the provider DQE Communications, which used software BGP Optimizer to optimize routing. BGP Optimizer breaks IP prefixes into smaller parts; for example, it splits 104.20.0.0/20 into 104.20.0.0/21 and 104.20.8.0/21, resulting in DQE Communications holding a large number of specific routes that override more general routes (i.e., instead of general routes to Cloudflare, more granular routes to specific Cloudflare subnets were used).

These specific routes were announced to one of the clients (Allegheny Technologies, AS396531), which also had a connection through another provider. Allegheny Technologies broadcast the received routes to another transit provider (Verizon, AS701). Due to the lack of proper BGP announcement filtering and limits on the number of prefixes, Verizon picked up this announcement and broadcast the received 20,000 prefixes to the rest of the internet. The incorrect prefixes were perceived as more prioritized due to their granularity, as a specific route has a higher priority than a general one.

As a result, traffic for many major networks began to route through Verizon to the small provider DQE Communications, which was unable to handle the influx, leading to a collapse (the effect is comparable to replacing part of a busy highway with a dirt road).

To prevent similar incidents in the future,
are recommended:

• Use the verification of announcements based on RPKI (BGP Origin Validation, which allows receiving announcements only from network owners);
• Limit the maximum number of accepted prefixes for all EBGP sessions (the maximum-prefix setting would help immediately discard the transmission of 20,000 prefixes within a single session);
• Apply filtering based on the IRR (Internet Routing Registry, which defines the AS through which routing of specified prefixes is allowed);
• Use the settings recommended in RFC 8212 on routers for default blocking (‘default deny’);
• Cease reckless use of BGP optimizers.

Source: opennet.ru