An open-source extension called Explain This has been introduced, enabling instant explanations of selected text directly in the browser by leveraging local language model capabilities.
What happened
The developer introduced the Explain This tool, which operates entirely on the client side. The extension uses the WebLLM library and a quantized Llama 3.2 1B model of approximately 880 MB to process requests via WebGPU. This allows users to get text interpretation without sending data to external servers or using paid API keys.
Context
The project's technical implementation relies on the chrome.offscreen API, which helps bypass limitations imposed by the Manifest V3 specification in Chromium-based browsers. The use of WebGPU ensures efficient neural network inference, shifting the computational load from cloud servers to the user's graphics processing unit.
Why this matters for the industry
The project is an important precedent for implementing the Edge AI concept in a web environment. It demonstrates the viability of WebGPU inference for natural language processing tasks and paves the way for creating full-fledged local AI assistants that work within the browser. This could contribute to a shift from cloud-oriented architectures to hybrid models, where lightweight tasks are performed on the user's device.
Why this matters for users
For end users, this means the ability to quickly and securely understand complex terms or text fragments in offline mode. Thanks to local processing, a maximum level of privacy is ensured, as personal information and web page content do not leave the user's device.
What is still unknown / limitations
Discussions are shifting from research interest to questions of scalability pragmatism and corporate governance when implementing such solutions.
Sources
Author
Look at AI, editorial team