LanguageModel

Chromium only, and only on capable hardware Chromium is the only engine that ships this, and even there the device has to meet the model's requirements. A supporting browser can still answer Unavailable. Build a server-side or manual fallback and treat this as the fast path, not the only path.

Three steps, every time Ask Availability whether the options you want can be served, Create a session from a user gesture (the first creation downloads the model - gigabytes, minutes), then prompt it and dispose it. An undisposed session keeps hundreds of megabytes of model state alive.

Support and availability

IsSupported / Availability

IsSupported answers whether the API exists; Availability answers whether it can actually serve you - Available, Downloadable (a session will download the model first), Downloading, or Unavailable. Probe with the options you intend to create with, since an option set the model can't serve answers Unavailable.

@inject Bit.Butil.LanguageModel languageModel

var availability = await languageModel.Availability();

if (availability == AiAvailability.Unavailable) UseServerFallback();

Live sample

availability output

Results will appear here when you interact with the samples.

Create a session

Create

Creates a conversation, optionally with a system prompt and sampling settings. Call it from a user gesture: the first creation on a device triggers the model download, which the browser will not start without one. The progress handler receives a 0-1 fraction while that happens.

_session = await languageModel.Create(
    new LanguageModelOptions
    {
        SystemPrompt = "You answer in one short sentence.",
        Temperature = 0.7,
        TopK = 3,
    },
    onDownloadProgress: fraction =>
    {
        _progress = fraction;
        InvokeAsync(StateHasChanged);
    });

Live sample

System prompt

session output

Results will appear here when you interact with the samples.

Prompt and wait

LanguageModelSession.Prompt

Sends a turn and resolves with the whole answer. The session is stateful - every turn sees the ones before it - so this is a conversation, not a series of independent calls.

Razor

@code {
    // The session from Create above. It carries the conversation, so the model sees everything asked
    // through it - which is also what fills the context window up.
    private LanguageModelSession? _session;

    private async Task Ask()
    {
        var answer = await _session!.Prompt("Name three uses for a paperclip.");
    }
}

Live sample

Prompt

prompt output

Results will appear here when you interact with the samples.

Prompt and stream

LanguageModelSession.PromptStreaming

The same turn, reported as it is generated. Each chunk is the delta, not the text so far - append it. The handler runs on the interop dispatch, so a component has to call StateHasChanged itself.

Razor

<p>@_answer</p>

@code {
    private string _answer = "";
    private LanguageModelSession? _session;   // from Create

    private async Task Stream(string prompt)
    {
        _answer = "";
        await _session!.PromptStreaming(prompt, chunk =>
        {
            _answer += chunk;
            InvokeAsync(StateHasChanged);
        });
    }
}

Live sample

streaming output

Results will appear here when you interact with the samples.

Context, quota and forking

Append / MeasureInputUsage / GetUsage / Clone

Append adds turns the model should know about without asking for a reply. MeasureInputUsage says what a turn would cost before you spend it. GetUsage reports how much of the quota is gone - a session that runs out starts dropping the oldest turns. Clone forks the conversation, which is how 'regenerate this answer' keeps the original.

Razor

@code {
    private LanguageModelSession? _session;   // from Create

    private async Task Budget(string draft)
    {
        var cost = await _session!.MeasureInputUsage(draft);
        var usage = await _session.GetUsage();

        if (cost > usage?.Remaining) TrimTheConversation();

        // A fork shares everything said so far and diverges from here, which is how a branch of the
        // conversation is explored without spending the original session's context on it.
        var fork = await _session.Clone();
    }
}

Live sample

context output

Results will appear here when you interact with the samples.

API reference

Member

Signature

Description

IsSupported

ValueTask<bool> IsSupported()

True when the runtime exposes the LanguageModel API.

Availability

ValueTask<AiAvailability> Availability() / Availability(LanguageModelOptions options)

Whether a session can be created right now, and whether that means downloading the model first.

GetParams

ValueTask<AiModelParams?> GetParams()

The model's sampling knobs and their ceilings. Null when the runtime has no such API.

Create

ValueTask<LanguageModelSession?> Create(LanguageModelOptions? options = null, Action<double>? onDownloadProgress = null)

Creates a conversation. Call from a user gesture; the first creation downloads the model. Null when the runtime refused.

Session.Prompt

ValueTask<string?> Prompt(string input)

Sends a turn and waits for the whole answer.

Session.PromptStreaming

Task<string> PromptStreaming(string input, Action<string>? onChunk = null)

Sends a turn and reports the answer as it is generated. Returns the whole answer once the stream ends.

Session.Append

ValueTask<bool> Append(params AiPrompt[] prompts)

Adds turns to the conversation without asking for a reply.

Session.MeasureInputUsage

ValueTask<double> MeasureInputUsage(string input)

What a turn would cost, without sending it. -1 when the runtime can't measure.

Session.GetUsage

ValueTask<AiUsage?> GetUsage()

How much of the session's input quota is spent, and what remains.

Session.Clone

ValueTask<LanguageModelSession?> Clone()

Forks the conversation into a new session with the same history.

Session.DisposeAsync

ValueTask DisposeAsync()

Destroys the model instance and frees what it holds. Always dispose.