Microsoft security researchers developed a side-channel attack technique called Whisper Leak, which allows the classification of query topics to services based on large language models through passive analysis of encrypted traffic transmitted over TLS connections. Information about the size of network packets and the delays between their transmission proved sufficient to determine the topics of requests to AI chatbots with over 98% accuracy. In practice, the proposed method can be used to identify specific query topics in user transit traffic, such as attempts to obtain information about illegal activities, without decrypting the content.
The toolkit for extracting data from traffic dumps, training models, and testing the method has been published on GitHub. The ability to conduct the attack has been demonstrated for 28 popular large language models from major manufacturers. For instance, the accuracy of identifying requests about "money laundering" for many AI services reached 100%, with one relevant request in the analyzed traffic and 10,000 unrelated requests.
The reason for the information leak is that models generate responses to requests step by step, one token at a time, using the previous token as context to determine the next most likely word or phrase. Consequently, each token is sent in a separate network packet, and the delay between the packets corresponds to the delay in the model's determination of the next token.
In TLS, if data compression is not used, the ciphertext size equals the size of the plaintext plus a constant. By creating a model that correlates the desired token sets with packet sizes and delays between their transmissions, one can quite accurately identify the presence of the desired topics in the traffic. In the course of the study, three versions of such machine learning models were prepared, based on the architectures of neural networks LightGBM, Bi-LSTM, and BERT. Experiments were conducted for each model to determine the desired topic by analyzing either packet size, only delays between packets, or both criteria.
To reduce the effectiveness of passive topic analysis of queries, AI service developers are advised to attach random additional padding, buffer the transfer of tokens, or implement dummy packet substitutions.
Source: opennet.ru
