Researchers from JFrog have discovered malicious machine learning models in the Hugging Face repository, the installation of which could lead to execution of code by attackers to gain control over the user's system. The issue arises because certain model distribution formats allow the embedding of executable code; for example, models using the "pickle" format can include serialized Python objects as well as code that executes upon loading the file, while Tensorflow Keras models can execute code via Lambda Layer.
To prevent the spread of such malicious models, Hugging Face employs scanning for embedded serialized code. However, the identified malicious models demonstrate that existing checks can be bypassed. Moreover, Hugging Face mostly only labels models as dangerous without blocking access to them. Approximately 100 potentially malicious models have been identified, 95% of which are intended for use with the PyTorch framework, and 5% with Tensorflow. The most common malicious modifications include object capture, remote system access (reverse shell), application launch, and file writing.
It is noted that, judging by the actions taken, most of the identified malicious models are created by security researchers attempting to earn rewards for discovering vulnerabilities and methods to bypass Hugging Face's protections (for example, instead of a real attack, such models try to launch a calculator or send a network request with information about the attack's success). There are also instances that launch a reverse shell to connect the attacker to the system.
For instance, the models "baller423/goober2" and "star23/baller13" target attacks on systems that load the model file into PyTorch using the torch.load() function. To facilitate code execution, the "__reduce__" method from the pickle module is employed, allowing arbitrary Python code to be injected into the deserialization process carried out during model loading.
Source: opennet.ru
