Camilla Moraes, a product manager at GitHub, has initiated discussions about adding a feature to GitHub that would automatically block spam pull requests generated by AI assistants, submitted without manual review, and that don't meet quality requirements. Such changes create additional burden on maintainers, who are forced to spend time reviewing useless code.
Short-term solutions being considered include the ability to quickly delete pull requests via the web interface (deleting them without leaving a mark in history instead of marking them as closed) and the use of customizable pull request permissions, which allow repository owners to restrict changes to only those who previously made the changes.
Long-term solutions include expanding the permissions model and providing maintainers with tools to flexibly define rules for who can create and review pull requests and what requirements pull requests must meet. Furthermore, it is proposed to use AI to determine whether a submitted change complies with the rules and quality standards of each project (for example, those defined in the CONTRIBUTING.md file), as well as to identify and specifically mark changes prepared with AI.
Among the suggestions made during the discussion, it is also worth noting the creation of a filter that prohibits the sending of pull requests without first opening an issue discussion explaining the reasons for implementing the changes, as well as informing maintainers about the receipt of pull requests from newcomers only after successful completion of tests in the continuous integration system.
According to statistics from one of the key developers of the genkit framework, only one in ten changes prepared in AI meets the criteria for opening a pull request. One of the Azure Core Upstream project participants summarized the main concerns of maintainers:
- Violation of the trust model in review - reviewers cannot be sure that the person submitting the change wrote the submitted code and understands its essence.
- Pull requests generated by AI assistants may appear structurally correct, but be logically incorrect, unsafe, or untested.
- The practice of line-by-line review remains mandatory, but it cannot scale in the face of increasing changes generated by AI assistants.
- Maintainers feel uncomfortable accepting pull requests they don't fully understand, while AI assistants make it easier to push out major changes without deep understanding.
- The cognitive load on maintainers increases, as they must now not only check the code but also assess whether the author understands it.
- The advent of AI tools has not reduced, but rather increased, the workload on support staff.
Additionally, a study conducted by several European universities on the impact of vibe coding on the open source ecosystem is noteworthy. The researchers developed a model of open source ecosystem equilibrium, which showed that the feedback loops that previously fueled the explosive growth of open source projects now have the opposite effect following the spread of vibe coding: the number of developers willing to share code decreases, the diversity of open source projects diminishes, and the quality declines. One suggested solution is the introduction of a Spotify-like funding model, in which AI platforms redistribute revenue from developer subscriptions among maintainers based on the projects' usage.
With vibe coding, developers stop analyzing available solutions, reading documentation, reporting bugs, and interacting with teams developing open-source libraries. Open-source projects lose user feedback. It becomes more difficult for new projects to gain ground, as AI assistants automatically select the necessary open-source libraries based on the information available at the time the model was trained. Reduced direct interaction with users affects the monetization of open-source projects, which rely on support services and advertising/donation campaigns on websites. Reduced feedback also affects quality. On the other hand, vibe coding increases productivity when creating new products based on third-party code and simplifies the implementation of new libraries.
As an example, the Tailwind CSS project is cited: the number of downloads from the NPM repository continues to grow, but traffic to the documentation has decreased by 40% since the beginning of 2023, and revenue has fallen by 80%. A decline in discussion activity on Stack Overflow of approximately 25% was also noted, six months after the launch of ChatGPT.
Source: opennet.ru
