Ondřej Surý, Director of DNS Engineering at ISC, previously involved in the CZ.NIC Labs project developing the Knot DNS server, summarized his experiments using large language models for analyzing, fixing, and modernizing the BIND 9 DNS server codebase, prototyping new projects, and preparing educational materials for students. The impressions from using AI concluded that large language models are well-suited for rapid prototyping, understanding unfamiliar code, and automating simple routine tasks, but it is doubtful that they save time for larger tasks, as significant time is spent on defining the problem, studying and verifying results, as well as subsequent refinement.
In the first experiment, the AI assistant Claude Code was tasked with identifying issues in the BIND 9 DNS server codebase, focusing on security problems and code modernization. None of the proposed fixes were accepted into the codebase, as Claude generated technically correct but practically useless code. For example, reserved identifiers and potential integer overflows were identified as issues, which were prevented by the compiler and did not require fixing. The experiment was deemed a waste of time.
In the second experiment, Ondřej asked Claude to write a telemetry system that integrates with various packages and minimizes metadata leaks. Claude Code prepared prototypes for the client and server, but without a proper understanding of the environment and issues arising during testing. Additionally, Google Gemini and ChatGPT were involved for verification, and each AI model identified errors in the output of the other models.
The method proved useful for quickly creating a prototype, but Ondřej noted that while working, he felt like the secretary of a robot overlord. Initially, the rapid prototyping was encouraging, but in the end, there was a sense that the entire AI-assisted development process took longer than writing the code manually from scratch. Much time was spent on analyzing the solution proposed by the AI, checking for pointless changes, and reworking it — the prototype had to be redone, as the code quality after AI involvement was mediocre and included a large number of repetitive constructs.
The third experiment involved generating a load balancer in Rust using the Domain and Tokio crate packages. Claude Code was able to generate the requested working prototype, but Ondřej is not familiar enough with Rust and the libraries involved to assess the quality of the work done.
The fourth experiment was related to preparing supplemental materials, tests, and questions for the course lectures that Ondřej teaches at the university using Google Gemini. This experiment was deemed the most successful, although it was not without problems — the AI invented a non-existent Czech word for a cryptography term. It was also noted that many student works generated by large language models were sent to him, which had to be returned for revision due to not meeting university-level standards and citing incorrect sources.
Additionally, there is another experiment conducted by the engineering director at Cloudflare. Using the Claude Code model, spending about a week and $1,100 on tokens, an alternative implementation of the Next.js API framework was prepared, formatted as a plugin for the Vite toolkit. The project was named vinext and published on GitHub. The new implementation is four times faster than building Next.js with Turbopack and generates frontend bundles that take up 57% less space.
The project implements 94% of the sixteen core APIs of Next.js and can be used as a transparent replacement for Next.js and for deploying projects on the Cloudflare Workers platform without additional layers like OpenNext and without binding to Node.js. The primary purpose of vinext is to provide the ability to use the Next.js API on platforms other than Vercel and without deploying a custom server. Currently, it only supports Cloudflare Workers and has a prototype for Vercel, but support for other serverless computing platforms compatible with Vite, including Netlify and AWS Lambda, is expected in the future.
Source: opennet.ru
