{"id":55733,"date":"2020-01-27T00:00:00","date_gmt":"2020-01-26T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/evolyutsiya-web-application-firewall-ot-setevyh-ekranov-do-oblachnyh-sistem-zashhity-s-mashinnym-obucheniem"},"modified":"2020-02-18T14:03:52","modified_gmt":"2020-02-18T11:03:52","slug":"evolyutsiya-web-application-firewall-ot-setevyh-ekranov-do-oblachnyh-sistem-zashhity-s-mashinnym-obucheniem","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/evolyutsiya-web-application-firewall-ot-setevyh-ekranov-do-oblachnyh-sistem-zashhity-s-mashinnym-obucheniem","title":{"rendered":"The Evolution of Web Application Firewall: From Network Firewalls to Cloud Protection Systems with Machine Learning","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

In our previous material on cloud topics, we talked<\/a><\/noindex>, discussed how to protect IT resources in the public cloud and why traditional antivirus solutions are not entirely suitable for these purposes. In this post, we will continue the theme of cloud security and talk about the evolution of WAF and what is better to choose: hardware, software, or cloud.\u00a0<\/b><\/p>\n

\"The
\n<\/p>\n

What is WAF<\/h3>\n

\nMore than 75% of hacker attacks target vulnerabilities in web applications and sites: such attacks are usually unnoticed by security infrastructure and security services. Web application vulnerabilities carry risks of account compromise and fraud involving user personal data, passwords, and credit card numbers. Additionally, vulnerabilities in a website serve as an entry point for intruders into corporate networks.<\/p>\n

A Web Application Firewall (WAF) is a protective barrier that blocks attacks on web applications: SQL injections, cross-site scripting, remote code execution, brute force attacks, and authentication bypass. This includes attacks that exploit zero-day vulnerabilities. Application firewalls provide protection by monitoring the content of web pages, including HTML, DHTML, and CSS, filtering potentially malicious requests over HTTP\/HTTPS.
\n<\/a><\/noindex><\/p>\n

What were the first solutions?<\/h3>\n

\nThe first attempts to create a Web Application Firewall were made back in the early 90s. At least three engineers are known to have worked in this field. The first was Professor of Computer Science Gene Spafford from Purdue University. He described the architecture of a proxy application firewall and published it in 1991 in the book \"Practical UNIX Security\"<\/a><\/noindex>.<\/p>\n

The second and third were security specialists William Cheswick and Marcus Ranum from Bell Labs. They developed one of the first prototypes of application firewalls. Its distribution was managed by DEC \u2014 the product was released under the name SEAL (Secure External Access Link). <\/p>\n

However, SEAL was not a full-fledged WAF solution. It was a classic network firewall with extended functionality \u2014 the ability to block attacks on FTP and RSH. For this reason, the first WAF solution is now considered to be the product from Perfecto Technologies (later Sanctum). In 1999, it was introduced<\/a><\/noindex> the AppShield system. At that time, Perfecto Technologies was developing cybersecurity solutions for e-commerce, and online stores became the target audience for their new product. AppShield could analyze HTTP requests and block attacks based on dynamic security policies.<\/p>\n

Around the same time as AppShield (in 2002), the first open-source WAF emerged. It was called ModSecurity<\/a><\/noindex>. It was created to promote WAF technologies and is still supported by the IT community (here is its repository on GitHub<\/a><\/noindex>). ModSecurity blocks application attacks based on a standard set of regular expressions (signatures) \u2014 tools for checking requests against patterns \u2014 OWASP Core Rule Set<\/a><\/noindex>.<\/p>\n

. As a result, developers achieved their goal \u2014 new WAF solutions began to appear on the market, including those built on ModSecurity.<\/p>\n

Three generations \u2014 already history<\/h3>\n

\nIt is common to distinguish three generations of WAF systems, which have evolved as technology has developed.<\/p>\n

The first generation<\/b>. It works with regular expressions (or grammars). ModSecurity belongs to this group. The system provider studies the types of attacks on applications and forms patterns that describe legitimate and potentially harmful requests. The WAF checks these lists to decide what to do in a specific situation \u2014 whether to block traffic or not.<\/p>\n

An example of regular expression-based detection is the aforementioned project Core Rule Set<\/a><\/noindex> open source. Another example is Naxsi<\/a><\/noindex>, which is also open source. Systems using regular expressions have several drawbacks, particularly since when a new vulnerability is discovered, the administrator must create additional rules manually. In the case of a large IT infrastructure, there can be thousands of rules. Managing such a large number of regular expressions is quite challenging, not to mention that their validation can reduce network performance.<\/p>\n

Regular expressions also have a relatively high level of false positives. The famous linguist Noam Chomsky proposed a classification of grammars, dividing them into four conditional levels of complexity. According to this classification, regular expressions can only describe firewall rules that do not allow deviations from the pattern. This means that attackers can easily \"fool\" first-generation WAFs. One method to combat this is to add special characters to application requests that do not affect the logic of malicious data but disrupt the signature rule.<\/p>\n

\"The
\n
\nSecond Generation<\/b>. To address the performance and accuracy issues associated with WAFs, second-generation application firewalls were developed. These include parsers that are responsible for identifying strictly defined types of attacks (on HTML, JS, etc.). These parsers work with special tokens that describe requests (for example, variable, string, unknown, number). Potentially malicious token sequences are placed on a separate list, which is regularly checked against the WAF system. This approach was first demonstrated at the Black Hat conference in 2012 in the form of C\/C++\u00a0libinjection library<\/a><\/noindex>, which enables the detection of SQL injection.<\/p>\n

Compared to first-generation WAFs, specialized parsers can operate faster. However, they did not resolve the difficulties related to manually tuning the system when new malicious attacks emerge. <\/p>\n

\"The
\n
\nThird Generation<\/b>. The evolution in detection logic in the third generation consists of applying machine learning methods that allow the grammar of detection to be closely aligned with the real grammar of the SQL\/HTML\/JS of the protected systems. This detection logic can adapt a Turing machine to encompass recursively enumerable grammars. Moreover, the task of creating an adaptable Turing machine was previously unsolvable until the first research on neural Turing machines was published.<\/p>\n

Machine learning provides a unique opportunity to adapt any grammar to cover any type of attack without manually creating signature lists, as was required in first-generation detection, and without developing new tokenizers\/parsers for new types of attacks such as Memcached, Redis, Cassandra, SSRF, as necessitated by second-generation methodology. <\/p>\n

By combining all three generations of detection logic, we can draw a new diagram, where the third generation of detection is outlined in red (see Fig. 3). This generation includes one of the solutions we are implementing in the cloud jointly with OnSec, the developer of the Valarm adaptive web application and API protection platform. <\/p>\n

Now, the detection logic uses feedback from the application for self-tuning. In the context of machine learning, this feedback loop is called 'reinforcement.' Typically, there are one or more types of such reinforcement:<\/p>\n