A group of researchers from Stanford University studied the impact of using intelligent assistants when writing code on the appearance of vulnerabilities in the code. We considered solutions based on the OpenAI Codex machine learning platform, such as GitHub Copilot, which allow you to generate fairly complex code blocks, up to ready-made functions. Concerns are related to the fact that since real code from public GitHub repositories, including those containing vulnerabilities, was used to train the machine learning model, the synthesized code may repeat errors and suggest code that contains vulnerabilities, and also does not take into account the need to perform additional checks when processing external data.
The study involved 47 volunteers with different experience in programming - from students to professionals with ten years of experience. Participants were divided into two groups - experimental (33 people) and control (14 people). Both groups had access to any libraries and Internet resources, including using ready-made examples from Stack Overflow. The experimental group was given the opportunity to use the AI assistant.
Each participant was given 5 tasks related to writing code in which it is potentially easy to make mistakes that lead to vulnerabilities. For example, there were tasks on writing encryption and decryption functions, using digital signatures, processing data involved in the formation of file paths or SQL queries, manipulating large numbers in C code, processing input displayed in web pages. To consider the impact of programming languages on the security of the code obtained when using AI assistants, the assignments covered Python, C and JavaScript.
As a result, it was found that participants who used an intelligent AI assistant based on the codex-davinci-002 model produced significantly less secure code than participants who did not use an AI assistant. In general, only 67% of the participants in the group using the AI assistant were able to provide a correct and secure code, while in the other group this figure was 79%.
At the same time, self-esteem indicators were reversed - participants who used the AI assistant believed that their code would be more secure than that of participants from the other group. In addition, it was noted that participants who trusted the AI assistant less and spent more time parsing and making changes to the issued hints made fewer vulnerabilities in the code.
For example, the code copied from the cryptographic libraries contained more secure default parameter values than the code suggested by the AI assistant. Also, when using the AI assistant, the choice of less reliable encryption algorithms and the absence of authentication of returned values were fixed. In the C number manipulation task, the code written using the AI assistant had more errors resulting in integer overflow.
In addition, a similar study by a group from New York University, conducted in November with the involvement of 58 students, who were asked to implement a structure for processing a shopping list in C, can be noted. The results showed a negligible impact of the AI assistant on code security - users who used the AI assistant made, on average, about 10% more security-related errors.
Source: opennet.ru