Your browser could run AI locally: What Opera's WebAI proposal means

Opera's WebAI proposal could let websites and browser extensions run AI models on users' devices. The shift could reduce reliance on cloud-based processing

Opera on October 27 introduced WebAI, an early-preview implementation of a proposed web standard that would allow websites and browser extensions to access AI models running on a user’s device. The approach supports models such as Google’s Gemma and Apple’s on-device foundation model on Mac computers.

 

The development adds to efforts by browser makers, including Google and Mozilla, to make AI available to web applications without requiring every request to be processed by a remote server. If adopted across browsers, such interfaces could allow developers to add features such as text summarisation, translation, text generation and semantic search without building separate implementations for different operating systems.

 

 

The proposal also raises questions about how browsers would manage AI models, allocate device resources and control access by websites. 

What is Opera’s WebAI proposal?

WebAI is an early implementation of a proposed interface through which websites and browser extensions can use AI models installed on a user’s device. Opera’s approach is based on the idea that web applications should be able to access local AI capabilities through a browser-managed interface.

 

At present, a website that offers AI-powered features will often send a user’s request to a cloud service, where a model processes it and returns a response. This requires a server-side AI system and an internet connection to communicate with it. Depending on the service, requests may also involve sending text, documents or other user-provided information to a remote system.

 

WebAI explores a different arrangement. A website could request an AI task through the browser, which would then use a compatible model available on the device. The website would not necessarily need to host the model or maintain its own inference infrastructure for that task.

 

Opera says the proposal can work with different models, including Google’s Gemma and Apple’s on-device foundation model on Macs. Support for models from different providers could give developers more options, although the models available would depend on the operating system, hardware and browser implementation.

 

The proposal is at an early stage. Its wider use would depend on how the interface develops, whether other browser makers adopt compatible implementations and how developers handle devices that cannot run the required models.

How browsers offer on-device AI, and what could change

Browser makers have been developing different ways to run AI models on users’ devices. Google Chrome offers built-in AI APIs for tasks such as summarisation, translation, writing assistance and proofreading. These interfaces allow websites and extensions to use models managed by the browser, without having to deploy their own AI infrastructure.

 

Mozilla has developed an experimental AI runtime for Firefox that uses Transformers.js and ONNX Runtime to run compatible machine learning models locally. The runtime is available for testing in Firefox Nightly and can support tasks such as text summarisation.

 

Opera's WebAI preview takes a similar approach, with APIs for text generation, classification and semantic search. However, the difference is in how these capabilities are made available to developers. Browser-specific implementations can require developers to account for different APIs and model support. Opera's proposal aims to provide a common interface through which websites and extensions can access local AI models.

 

If adopted across browsers, this could make it easier to add AI features to web applications without building separate integrations for different platforms.

Could the browser become a common runtime for AI?

A common runtime would allow developers to build AI features into websites without having to manage separate model deployments for each operating system. The browser would handle access to the model, while the website would define the task it needs to perform.

 

For example, a news website could offer summaries of long articles using a model installed on the user's device. A research tool could group documents by subject or search a collection of saved pages based on meaning rather than exact keywords. A browser extension could classify emails or identify relevant job listings without sending the content to a cloud AI service.

 

These functions require different AI capabilities. Text generation involves producing new content, while classification assigns content to categories. Semantic search uses numerical representations of text, known as embeddings, to identify related information even when the wording differs.

 

Opera's WebAI implementation includes Prompt, Decision and Semantic Embeddings APIs for these types of tasks. Developers could use them to build features that work with information already available in a webpage or extension, subject to the permissions and capabilities provided by the browser.

 

The browser could also manage the model's availability and the resources needed to run it. This would spare developers from having to build their own model-loading and inference systems for every application.

What does a browser need to run AI locally?

Running a model involves more than providing a website with an API. The browser must manage the model files, allocate memory, run inference and coordinate access to the device's processing hardware.

 

Model downloads are one consideration. AI models can occupy hundreds of megabytes or several gigabytes, depending on their size and configuration. A browser may need to download a model the first time a user enables a feature, store it locally and check whether an updated version is available.

 

Opera's WebAI preview is opt-in. Users must enable the feature before a compatible model is downloaded. Its initial availability is limited to 64-bit Windows PCs using Intel or AMD processors and Macs with Apple silicon. Intel-based Macs, Linux systems and Windows computers running Arm processors are not supported at launch, according to Opera.

 

Hardware is another constraint. AI inference can use the CPU, GPU or specialised accelerators, depending on the model and software implementation. The available memory, processing capacity and power limits will affect how quickly a response is generated.

 

A model that runs well on a laptop with sufficient memory may be slow or unavailable on an older computer. Mobile devices face their own limits around memory, battery consumption and heat. Browser makers will need to decide which models are suitable for different devices and how applications should respond when a device cannot run a requested task.

 

Developers must also account for differences in model output. Two models may support the same task but produce different results in terms of accuracy, language support and response quality. A common API can standardise how an application requests a task, but it cannot guarantee identical results across models.

Does local AI make browsing more private?

Local processing can reduce the amount of information sent to external AI providers. If a browser runs a summarisation model on the device, the text being summarised does not have to be sent to a separate AI server for inference.

 

This can be useful when working with personal documents, emails or other information that users may not want to share with an external service. Local models can also run without an active internet connection once the required files are available, provided the application does not depend on other online services.

 

However, local inference does not automatically make an entire website private. A website may still send browsing activity, account details, search queries or other information to its own servers. An application could also transmit content for purposes unrelated to its local AI feature.

 

Permissions will therefore be important. Websites and extensions should not receive unrestricted access to local models or sensitive content merely because a browser supports AI. Users need to know which sites can invoke a model, what information the model can process and whether any data is sent elsewhere.