A year ago, March 21, 2019, at Very good came to HackerOne from . When embedding a null byte (ASCII 0) in the POST parameter of one of the webmail API requests that returned an HTTP redirect, chunks of uninitialized memory were visible in the redirect data, in which fragments from GET parameters and headers of other requests to the same server.
This is a critical vulnerability, because. requests contain, among other things, session cookies. A few hours later, a temporary fix was made that filtered the null byte (as it turned out later, this was not enough, because it remained possible to inject CRLF / ASCII 13, 10, which allows you to manipulate HTTP response headers and data, this is less critical, but still annoying). At the same time, the problem was transferred to security analysts and developers to find and eliminate the causes of the bug.
Mail.ru mail is a very complex application, a large number of different front-end / back-end components, both open-source (many thanks to all free software developers) and self-developed, can participate in the formation of a response. It was possible to exclude all components except nginx and openresty and localize the problem before the call in an OpenResty script that did not behave as expected (inserting a null byte or a line feed through GET parameters with rewrite into ngx_http_rewrite_module, which, according to the documentation, is used and, it would seem, should work in exactly the same way, will not work). Possible consequences have been eliminated, the most stringent filtering has been added, and it has been verified that the filtering eliminates all possible vectors. But the mechanism that led to the memory leak remained a mystery. A month later, the bug report was closed as resolved, and the analysis of the causes of the bug was postponed until better times.
OpenResty is a very popular plugin that allows you to write Lua scripts inside nginx, and it is used in several Mail.ru projects, so the problem was not considered solved. And after some time, they nevertheless returned to it in order to understand the true causes, possible consequences and make recommendations for developers. The excavation of the source code involved ΠΈ . It turned out that:
- In nginx, when using rewrite with user data, there is a possibility of directory traversal (and probably SSRF) in some configurations, but this is a known fact and should be detected by static configuration analyzers in ΠΈ from Yandex (yes, we also use it, thanks). When using OpenResty, this possibility is easy to miss, but this did not affect our configuration.
configuration example:
location ~ /rewrite { rewrite ^.*$ $arg_x; } location / { root html; index index.html index.htm; }result
curl localhost:8337/rewrite?x=/../../../../../../../etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
... - There is a bug in nginx that causes a memory leak if the rewrite string contains a null byte. When redirecting, nginx allocates a new memory buffer corresponding to the full length of the string, but copies the string there through a string function in which the null byte is the string terminator, so the string is copied only up to the null byte, the remainder of the buffer contains uninitialized data. A detailed breakdown can be found .
configuration example (^@ null byte)
location ~ /memleak { rewrite ^.*$ "^@asdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdasdf"; } location / { root html; index index.html index.htm; }result
curl localhost:8337/secret -vv
...
curl localhost:8337/memleak -vv
...
Location: http://localhost:8337/secret
...
- Nginx protects GET parameters from service character injection and makes it possible to use only GET parameters in rewrite. Therefore, exploiting the injection through user-controlled parameters in nginx does not work. POST parameters are not protected. OpenResty allows you to work with both GET and POST parameters, so when using POST parameters through OpenResty, it becomes possible to inject special characters.
configuration example:
location ~ /memleak { rewrite_by_lua_block { ngx.req.read_body(); local args, err = ngx.req.get_post_args(); ngx.req.set_uri( args["url"], true ); } } location / { root html; index index.html index.htm; }result:
curl localhost:8337 -d "url=secret" -vv
...
curl localhost:8337 -d "url=%00asdfasdfasdfasdfasdfasdfasdfasdf" -vv
...
Location: http://localhost:8337/{...ΠΌΠΎΠΆΠ΅Ρ ΡΠΎΠ΄Π΅ΡΠΆΠ°ΡΡ secret...}
...
Further reaction
The issue has been reported to the nginx and OpenResty developers, the developers are not treating the issue as a security bug in nginx, as in nginx itself there is no possibility of exploiting an error through the injection of special characters, fix was posted December 16th. For 4 months since the report, no changes have been made to OpenResty either, although there was an understanding that a safe version of the ngx.req.set_uri () function was needed. March 18, 2020 we published information, March 21 OpenResty released , which adds URI validation.
Portswigger good article and took comments from OpenResty and Nginx (although the comment that only a small piece of memory is revealed is incorrect and misleading, this is determined by the length of the string following the null byte and, in the absence of explicit restrictions on the length, can be controlled by the attacker).
So what was the mistake and what to do to prevent it?
Was there a bug in nginx? Yes, it was, because a memory leak is a bug anyway.
Were there any bugs in OpenResty? Yes, at least the issue of the security of the functionality offered by OpenResty has not been investigated and documented.
Has an OpenResty configuration / usage error been made? Yes, because in the absence of an explicit statement, an unverified assumption has been made about the safety of the functionality used.
Which of these errors is a $10000 bounty security vulnerability? For us, it doesn't really matter. In any software, especially at the junction of several components, especially those provided by different projects and developers, no one can ever guarantee that all the features of their work are known and documented and that there are no errors. Therefore, any security vulnerability occurs precisely where it affects security.
In any case, it is good practice to normalize or limit/filter as much as possible the input data that goes to any external module/API, unless there are explicit instructions and unequivocal understanding that this is not required.
Erratum
By experience , for the sake of preserving the purity of the language:
bug bounty - bug hunting contest
bug report - error notification
redirect - redirect
open source - open source
Wrong - work on mistakes
Source: habr.com