Camilla Moraes, a product manager at GitHub, initiated a discussion on adding a feature to GitHub to automatically block spammy pull requests generated by AI assistants, submitted without manual review and not meeting quality standards. Such changes create an additional burden for maintainers, who have to spend time sorting through useless code.
As short-term solutions to the problem, the possibility of quickly deleting pull requests through the web interface (deleting without it showing in history instead of marking them as closed) and the use of customizable permissions for submitting pull requests are being considered, allowing repository owners to permit contributions only from participants who have previously made changes.
Long-term solutions mentioned include expanding the permission model and providing maintainers with tools to flexibly define rules that determine who has the right to create and review pull requests and what criteria pull requests must meet. Additionally, there are proposals to involve AI in assessing whether submitted changes comply with the rules and quality standards of each project (for example, those specified in CONTRIBUTING.md), as well as identifying and marking changes prepared with the assistance of AI.
Among the suggestions discussed, the creation of a filter that prevents the submission of pull requests without first opening issue discussions explaining the reasons for the changes, and notifying maintainers about pull requests from newcomers only after they successfully pass tests in the continuous integration system can also be noted.
According to statistics from a key developer of the genkit framework, only one in ten changes prepared by AI meets the criteria for opening a pull request. A participant in the Azure Core Upstream project summarized the main concerns of the maintainers:
- Violation of the trust model in reviewing — reviewers cannot be certain that the person submitting the change wrote the code and understands its essence.
- AI-generated pull requests may appear structurally correct, but can be logically flawed, unsafe, or untested in practice.
- Line-by-line review practices remain mandatory, but they cannot scale in the face of increasing changes driven by AI assistants.
- Maintainers experience discomfort in accepting pull requests that they do not fully understand, while AI assistants simplify the transfer of large changes without deep comprehension.
- The cognitive load on maintainers increases, as they now need to not only review the code but also assess whether the author understands it.
- The emergence of AI tools has not reduced but rather increased the burden on maintainers.
Additionally, a study conducted by several European universities examined the impact of vibe coding on the open-source project ecosystem. Researchers developed a model of open-source ecosystem equilibrium, which showed that feedback loops, which previously drove explosive growth of open projects, create a counterproductive effect after the spread of vibe coding — resulting in a decrease in the number of developers willing to share code, a reduction in the diversity of open projects, and lowered quality. One proposed solution to this problem involves implementing a funding model reminiscent of Spotify, where AI platforms redistribute subscription revenue from their services to developers based on the extent of project utilization.
With vibe coding, developers stop analyzing available solutions, reading documentation, sending error messages, and interacting with teams developing open libraries. Open projects lose feedback from users. New projects find it harder to break through as AI assistants autonomously select the necessary open libraries based on information available at the time of the model's training. Due to the reduction of direct interaction with users, monetization of open projects relying on support services and advertising/donations on websites suffers. The quality also declines because of the lack of feedback. On the other hand, vibe coding increases productivity in creating new products based on foreign code and simplifies the integration of new libraries.
An example is the Tailwind CSS project, which continues to see a rise in downloads from the NPM repository, but traffic to the documentation has decreased by 40% since the beginning of 2023, and revenues have fallen by 80%. There has also been a noted reduction in discussion activity on Stack Overflow by approximately 25%, six months after the launch of ChatGPT.
Source: opennet.ru
