Workers AI

The AI binding in wrangler.jsonc gives the Worker access to Workers AI models: text generation, embeddings, image generation, speech and more.

Calling a model

import { env } from 'cloudflare:workers';

const result = await env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
  messages: [
    { role: 'system', content: 'You summarize notes in one sentence.' },
    { role: 'user', content: note.content },
  ],
});

Pick the model from the current Workers AI model catalog. Models are added and retired over time, so check the name before you ship.

Keep it on the server

Call models from endpoints or actions, never from the browser. Check the user and team first, like any other endpoint, and consider limiting how often each team can call it.