Google has announced an experiment to integrate a large machine learning language model into Chrome. To access the model from web applications and browser extensions, an API Prompt is offered, allowing natural language requests in a manner similar to chatbots. The aim is that the integrated large language model in the browser will simplify the execution of AI tasks in web applications and remove the need for installing and managing language models.
The experiment utilizes the Gemini Nano model, the most compact of the Gemini family. There is also the option to install expert models that enhance the base model with additional knowledge needed for specific tasks, as well as to implement skills such as machine translation and summarization. Models run locally on the user's system without relying on external services.
The Runtime used for executing the models automatically utilizes available GPU and NPU in the system to accelerate model performance or switches to using CPU for model execution. Advantages of running the model on the user's system include the preservation of data privacy, the ability to continue working in offline mode when there is no network connection or when quality issues arise, reduced latency in sending requests, and avoidance of dependence on external services.
The developed API Prompt model allows not only for simple single requests in natural language but also for organizing the model's engagement in data processing and classification considering context, taking into account previously sent requests and data within a session, as well as using the model to select optimal options (for example, one can request to choose an emoji from a list for a specific comment on a website). Additionally, there are plans to develop the API for use in content creation, addressing tasks such as rephrasing, proofreading, and grammar correction.
In general, two types of APIs are being developed for interacting with the embedded AI model — Task and Exploratory. The first provides access to capabilities for solving specific tasks, such as translating text from one language to another (API Translation) or summarizing the essence of a text (API Summarization). The second type is focused on creating and testing experimental prototypes while developing new Task APIs. Work is also underway on the LoRA (Low-Rank Adaptation) API for adapting the weight coefficients of the base model to improve the effectiveness of solving specific tasks.
Access to participate in the experiment is granted after submitting an application. The API is actively being developed and will expand and change until a final version is accepted, taking into account user feedback and preferences. In the future, there are plans to organize more accessible testing using the Origin Trials mode, which provides the ability to work with experimental APIs from applications loaded from localhost or 127.0.0.1, or after registering and obtaining a special token that is valid for a limited time for a specific site. Work is being conducted in parallel with other browser manufacturers to standardize the developed APIs.
Source: opennet.ru
