The Prompt API

| Explainer | Web | Extensions | Chrome Status | Intent | |---|---|---|---|---| | GitHub | Origin trial | Chrome 138 | View | Intent to Experiment | | GitHub | Origin trial for sampling parameters | Chrome 148 | View | Intent to Experiment |

With the Prompt API, you can send natural language requests to Gemini Nano in the browser.

There are many ways you can use the Prompt API. For example, you could build:

AI-powered search: Answer questions based on the content of a web page.Personalized news feeds: Build a feed that dynamically classifies articles with categories and allow for users to filter for that content.Custom content filters. Analyze news articles and automatically blur or hide content based on user-defined topics.Calendar event creation. Develop a Chrome Extension that automatically extracts event details from web pages, so users can create calendar entries in just a few steps.Seamless contact extraction. Build an extension that extracts contact information from websites, making it easier for users to contact a business or add details to their list of contacts.

These are just a few possibilities, and we're excited to see what you create.

Review the hardware requirements

The following requirements exist for developers and the users who operate features using these APIs in Chrome. Other browsers may have different operating requirements.

The Language Detector and Translator APIs work in Chrome on desktop. These APIs do not work on mobile devices.

The Prompt API, Summarizer API, Writer API, Rewriter API, and Proofreader API work in Chrome when the following conditions are met:

Operating system: Windows 10 or 11; macOS 13+ (Ventura and onwards); Linux; or ChromeOS (from Platform 16389.0.0 and onwards) on Chromebook Plus devices. Chrome for Android, iOS, and ChromeOS on non-Chromebook Plus devices are not yet supported by the APIs which use Gemini Nano.Storage: At least 22 GB of free space on the volume that contains your Chrome profile.GPU or CPU: Built-in models can run with GPU or CPU.- GPU: Strictly more than 4 GB of VRAM. - CPU: 16 GB of RAM or more and 4 CPU cores or more. Note: The Prompt API with audio input requires a GPU.

Network: Unlimited data or an unmetered connection.

Gemini Nano's exact size may vary as the browser updates the model. To determine the current size, visit chrome://on-device-internals

.

Use the Prompt API

The Prompt API uses the Gemini Nano model in Chrome. While the API is built into Chrome, the model is downloaded separately the first time an origin uses the API. Before you use this API, acknowledge Google's Generative AI Prohibited Uses Policy.

To determine if the model is ready to use, call LanguageModel.availability()

.

const availability = await LanguageModel.availability({
// The same options in `prompt()` or `promptStreaming()`
});

To trigger the download and instantiate the language model, check for user activation. Then, call the create()

function.

const session = await LanguageModel.create({
monitor(m) {
m.addEventListener('downloadprogress', (e) => {
console.log(`Downloaded ${e.loaded * 100}%`);
});
},
});

If the response to availability()

was downloading

, listen for download progress and inform the user, as the download may take time.

Use on localhost

All of the built-in AI APIs are available on localhost

in Chrome. Set the following flags to Enabled:

chrome://flags/#optimization-guide-on-device-model

chrome://flags/#prompt-api-for-gemini-nano-multimodal-input

Then click Relaunch or restart Chrome. If you encounter errors, troubleshoot localhost.

Model parameters

The params()

function informs you of the language model's parameters. The object has the following fields:

defaultTopK

: The default top-K value.maxTopK

: The maximum top-K value.defaultTemperature

: The default temperature.maxTemperature

: The maximum temperature.

// Only available when using the Prompt API for Chrome Extensions.
await LanguageModel.params();
// {defaultTopK: 3, maxTopK: 128, defaultTemperature: 1, maxTemperature: 2}

Create a session

Once the Prompt API can run, you create a session with the create()

function.

const session = await LanguageModel.create();

Create a session with the Prompt API for Chrome Extensions

When you use the Prompt API for Chrome Extensions, each session can be customized with topK

and temperature

using an optional options object. The default values for these parameters are returned from LanguageModel.params()

.

// Only available when using the Prompt API for Chrome Extensions.
const params = await LanguageModel.params();
// Initializing a new session must either specify both `topK` and
// `temperature` or neither of them.
// Only available when using the Prompt API for Chrome Extensions.
const slightlyHighTemperatureSession = await LanguageModel.create({
temperature: Math.max(params.defaultTemperature * 1.2, 2.0),
topK: params.defaultTopK,
});

The create()

function's optional options object also takes a signal

field, which lets you pass an AbortSignal

to destroy the session.

const controller = new AbortController();
stopButton.onclick = () => controller.abort();
const session = await LanguageModel.create({
signal: controller.signal,
});

Add context with initial prompts

With initial prompts, you can provide the language model with context about previous interactions, for example, to allow the user to resume a stored session after a browser restart.

const session = await LanguageModel.create({
initialPrompts: [
{ role: 'system', content: 'You are a helpful and friendly assistant.' },
{ role: 'user', content: 'What is the capital of Italy?' },
{ role: 'assistant', content: 'The capital of Italy is Rome.' },
{ role: 'user', content: 'What language is spoken there?' },
{
role: 'assistant',
content: 'The official language of Italy is Italian. [...]',
},
],
});

Constrain responses with a prefix

You can add an "assistant"

role, in addition to previous roles, to elaborate on the model's previous responses. For example:

const followup = await session.prompt([
{
role: "user",
content: "I'm nervous about my presentation tomorrow"
},
{
role: "assistant",
content: "Presentations are tough!"
}
]);

In some cases, instead of requesting a new response, you may want to prefill part of the "assistant"

-role response message. This can be helpful to guide the language model to use a specific response format. To do this, add prefix: true

to the trailing "assistant"

-role message. For example:

const characterSheet = await session.prompt([
{
role: 'user',
content: 'Create a TOML character sheet for a gnome barbarian',
},
{
role: 'assistant',
content: '```toml\n',
prefix: true,
},
]);

Add expected input and output

The Prompt API has multimodal capabilities and supports multiple languages. Set the expectedInputs

and expectedOutputs

modalities and languages when creating your session.

type

: Modality expected.- For expectedInputs

, this can betext

,image

, oraudio

. - For expectedOutputs

, the Prompt API allowstext

only.

• For languages

: Array to set the language or languages expected. The Prompt API accepts"en"

,"ja"

, and"es"

. Support for additional languages is in development.- For expectedInputs

, set the system prompt language and one or more expected user prompt languages. - Set one or more expectedOutputs

languages.

• For

const session = await LanguageModel.create({
expectedInputs: [
{ type: "text", languages: ["en" /* system prompt */, "ja" /* user prompt */] }
],
expectedOutputs: [
{ type: "text", languages: ["ja"] }
]
});

You may receive a "NotSupportedError"

DOMException if the model encounters an unsupported input or output.