A year ago, on March 21, 2019, the on HackerOne received some very good from . When a null byte (ASCII 0) was injected into the POST parameter of one of the webmail API requests that returned an HTTP redirect, bits of uninitialized memory could be seen in the redirect data, which often revealed fragments from the GET parameters and headers of other requests to the same server.
This is a critical vulnerability as the requests also contain session cookies. A temporary fix was made a few hours later, which filtered the null byte (as it turned out, this was insufficient since the possibility of CRLF injection / ASCII 13, 10 remained, allowing for manipulation of HTTP response headers and data, which is less critical but still unpleasant). At the same time, the issue was transferred to security analysts and developers for investigation and resolution of the root cause of the bug.
Mail.ru Mail is a very complex application, involving a large number of different frontend/backend components in forming a response, both open-source (big thanks to all free software developers) and proprietary. All components were managed to be excluded except for nginx and openresty, and the problem was localized to the call of in the OpenResty script, which did not behave as expected (inserting a null byte or newline through GET parameters with rewrite in ngx_http_rewrite_module, which, according to the documentation, is used and should work exactly the same way, was not possible). Possible consequences were eliminated, maximum stringent filtering was added, and it was verified that the filtering eliminated all possible vectors. However, the mechanism that led to the memory content leak remained a mystery. A month later, the bug report was closed as resolved, and the investigation into the cause of the bug was postponed for a better time.
OpenResty is a very popular plugin that allows writing Lua scripts inside nginx, and it is used in several Mail.ru projects, so the problem was not considered resolved. After some time, it was revisited to understand the true causes, potential consequences, and to provide recommendations for developers. Contributing to the source code investigation were and It has been determined that:
- In nginx, when using rewrite with user data, there is a possibility of directory traversal (and possibly SSRF) in certain configurations. However, this is a known issue and should be detected by static configuration analyzers in and from Yandex (yes, we use it too, thank you). When using OpenResty, this vulnerability can be easily overlooked, but it did not affect our configuration.
example configuration:
location ~ /rewrite { rewrite ^.*$ $arg_x; } location / { root html; index index.html index.htm; }result
curl localhost:8337/rewrite?x=/../../../../../../etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
... - Nginx has a bug that leads to a memory leak if the rewrite string contains a null byte. When returning a redirect, nginx allocates a new memory buffer corresponding to the full length of the string but copies it using a string function where the null byte serves as a string terminator. As a result, the string is only copied up to the null byte, and the rest of the buffer contains uninitialized data. A detailed breakdown can be found .
example configuration (^@ null byte)
location ~ /memleak { rewrite ^.*$ "^@asdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdasdf"; } location / { root html; index index.html index.htm; }result
curl localhost:8337/secret -vv
...
curl localhost:8337/memleak -vv
...
Location: http://localhost:8337/secret
...
- Nginx protects GET parameters from injecting special characters and allows using only GET parameters in rewrite. Therefore, exploiting injection through user-controlled parameters in nginx is not possible. POST parameters, however, are not protected. OpenResty allows working with both GET and POST parameters, so when using POST parameters through OpenResty, there is a possibility of injecting special characters.
example configuration:
location ~ /memleak { rewrite_by_lua_block { ngx.req.read_body(); local args, err = ngx.req.get_post_args(); ngx.req.set_uri( args["url"], true ); } } location / { root html; index index.html index.htm; }result:
curl localhost:8337 -d "url=secret" -vv
...
curl localhost:8337 -d "url=asdfasdfasdfasdfasdfasdfasdfasdf" -vv
...
Location: http://localhost:8337/{...may contain secret...}
...
Further action
The issue was reported to the nginx and OpenResty developers. The developers do not consider the problem to be a security vulnerability in nginx since there is no possibility to exploit the bug through the injection of special characters in nginx. The fix was published on December 16. Four months after the report, no changes were made in OpenResty, although it was understood that a safe version of the ngx.req.set_uri() function was necessary. On March 18, 2020, we published information, and on March 21, OpenResty released , which adds URI verification.
Portswigger a good article and took comments from OpenResty and Nginx (although the comment that only a small fragment of memory is revealed is incorrect and misleading, this is determined by the length of the string following the null byte and, in the absence of explicit length constraints, can be controlled by the attacker).
So what was the error and what should be done to prevent it?
Was there an error in Nginx? Yes, there was, because a memory content leak is an error in any case.
Was there an error in OpenResty? Yes, at the very least, the security issue of the functionalities provided by OpenResty was not explored and documented.
Was there a configuration/error in the usage of OpenResty? Yes, because in the absence of explicit indication, an unchecked assumption was made about the security of the utilized functionality.
Which of these errors constitutes a security vulnerability with a bounty of $10,000? For us, it is generally not important. In any software, especially at the intersection of several components, particularly provided by different projects and developers, no one can ever guarantee that all aspects of their operation are known and documented and that there are no errors. Therefore, any security vulnerability arises where it affects security.
In any case, a good practice would be to normalize or restrict/filter as much as possible any input data that goes into any external module/API, if there are no explicit indications and unequivocal understanding that this is not required.
Errata
Based on experience , for the sake of maintaining the purity of the language:
bug bounty — a bug hunting competition
bug report — a bug report
redirect — redirection
open-source — with open code
errata — work on errors
Source: habr.com
