The results of the analysis of the password reliability generated by large language models and AI assistants have been presented. Researchers asked the models Claude, ChatGPT, and Gemini to generate a secure 16-character password, and in all cases received results that, at first glance, met all the requirements for secure passwords and were recognized by password quality check utilities as reliable. The passwords combined symbols in different cases, special characters, and numbers, but only appeared to be secure, while in reality they had minimal entropy, were formed according to a typical pattern, and exhibited regularities in repeated requests.
According to researchers, the predictable passwords generated by large language models are used in practice by real users and suggested during coding work by AI assistants. The level of entropy in passwords generated by AI models is estimated at 20-27 bits, which requires several seconds to hours for password cracking, whereas the analysis of results by password quality check utilities predicts cracking times of several centuries. The pattern of such passwords is a consequence of the content generation by large language models based on token prediction.
Out of 50 passwords generated in Claude Opus 4.6, 18 were identical. All passwords started with a letter (mostly 'G'), followed by a digit (mostly 7). All passwords contained the symbols 'L', '9', 'm', '2', '$', and '#'.
In GPT-5.2, almost all passwords began with the letter 'v', after which half of the passwords were followed by the letter 'Q' and a repeating pattern of a limited set of symbols. In Gemini 3, nearly half of the passwords started with the symbols 'K' or 'k', after which ' #', 'P', or '9' were the most common, and the set of symbols used was significantly reduced. Increasing the 'temperature' when approaching AI models does not significantly affect the quality of generated passwords.
The quality of passwords generated by AI assistants largely depends on the prompt created by the developer. For instance, Claude Code with Opus 4.6 and Gemini-CLI with Auto Gemini 3 used the command "openssl rand" to generate passwords. Meanwhile, Gemini-CLI with Auto Gemini 3 used "openssl rand" for the prompt "generate a password," but generated a password through the AI model for the prompt "suggest a password." Codex with GPT-5.3-Code sometimes called an external utility to generate a reliable password, but at times created predictable passwords on its own. Claude Code with Opus 4.5 most often generated predictable passwords independently. The ChatGPT Atlas browser produced an unreliable password through the AI model when creating a password for website registration.
Source: opennet.ru
