Daniel Stenberg, author of a utility for receiving and sending data over the network curl, criticized the use of AI tools when creating vulnerability reports. Such reports include detailed information, are written in normal language and look high-quality, but without thoughtful analysis in reality they can only be misleading, replacing real problems with quality-looking garbage content.
The Curl project pays rewards for identifying new vulnerabilities and has already received 415 reports of potential problems, of which only 64 were confirmed as vulnerabilities and 77 as non-security bugs. Thus, 66% of all reports did not contain any useful information and only took away time from developers that could have been spent on something useful.
Developers are forced to waste a lot of time parsing useless reports and double-checking the information contained there several times, since the external quality of the design creates additional confidence in the information and there is a feeling that the developer misunderstood something. On the other hand, generating such a report requires minimal effort from the applicant, who does not bother checking for a real problem, but simply blindly copies the data received from AI assistants, hoping for luck in the struggle to receive a reward.
Two examples of such garbage reports are given. The day before the planned disclosure of information about the dangerous October vulnerability (CVE-2023-38545), a report was sent via Hackerone that the patch with the fix had become publicly available. In fact, the report contained a mixture of facts about similar problems and snippets of detailed information about past vulnerabilities compiled by Google's AI assistant Bard. As a result, the information looked new and relevant, and had no connection with reality.
The second example concerns a message received on December 28 about a buffer overflow in the WebSocket handler, sent by a user who had already informed various projects about vulnerabilities through Hackerone. As a method of reproducing the problem, the report included general words about passing a modified request with a value larger than the size of the buffer used when copying with strcpy. The report also provided an example of a correction (an example of replacing strcpy with strncpy) and indicated a link to the line of code βstrcpy(keyval, randstr)β, which, according to the applicant, contained an error.
The developer double-checked everything three times and did not find any problems, but since the report was written confidently and even contained a correction, there was a feeling that something was missing somewhere. An attempt to clarify how the researcher managed to bypass the explicit size check present before the strcpy call and how the size of the keyval buffer turned out to be less than the size of the read data led to detailed, but not carrying additional information, explanations that only chewed on the obvious common causes of buffer overflow not related to specific Curl code. The answers were reminiscent of communicating with an AI assistant, and after spending half a day on pointless attempts to find out exactly how the problem manifests itself, the developer was finally convinced that there was in fact no vulnerability.
Source: opennet.ru