Recently, the activity of web bots indexing traffic has significantly increased. In addition to properly functioning bots, there has been a rise in 'ferocious' bots that ignore the indexing rules of robots.txt, invade with tens of thousands of different IPs, impersonate legitimate users, and do not adhere to a reasonable request rate policy. These bots create massive parasitic loads on servers, disrupt the normal operation of systems, and consume administrators' time. The activity of such bots is seen by many as malicious actions.
As a measure to slow down these bots, as well as those scanning for unpatched vulnerabilities in typical web applications, one of the administrators proposed the 'zip bomb' method. The essence of the method is that a web bot receives, in response to a page request, content effectively compressed using the 'deflate' method, whose size after unpacking exceeds the transmitted network data many times over. For example, when using the 'deflate' method, content from /dev/zero packed into 10 MB will require 1 GB of disk space when unpacked. With the 'brotli' compression method, a ratio was achieved where sending 81 MB leads to unpacking 100 TB of data.
This type of protection can be activated by creating traps that are accessible via invisible links marked with the 'rel="nofollow"' flag, excluded from indexing via robots.txt, and triggered at a sufficiently high recursion level for bots trying to impersonate ordinary users. In practice, the proposed method is not recommended, as the site may be blacklisted by Google and start being flagged as harmful in the Chrome browser when 'Safe Browsing' mode is enabled.
Source: opennet.ru
