# The agent as a model: OpenAI- and Anthropic-compatible endpoints

Every agent can answer on the provider wire protocols. Point an SDK at this
platform, name the agent as the `model`, and a completion is a turn of that
agent — its own model, system prompt, tools, memory and observability.

It is a translation layer, not a proxy to OpenAI or Anthropic. Sampling
parameters a client sends (`temperature`, `max_tokens`, `seed`) are ignored,
because the agent's configuration decides those; the parameters that describe
the response rather than the model are honoured — `stream`, `stream_options`,
the tools the caller runs itself (`tools`), how much the model thinks
(`reasoning_effort` / `thinking`), whether it must or must not use a tool
(`tool_choice`), and the shape of the answer (`response_format`). Conversations
that arrive this way are stored, searchable and traced like one that arrived
over SMS.

## Turning it on

Agents → your agent → **API gateway** → *Turn on*. While it is off the agent is
not a model any client can name: `/models` omits it, and a call naming it is
refused with the provider's permission error.

The gateway holds no provider secret. Callers authenticate with the
organization's own developer API keys (`sk_live_…` / `sk_test_…`), so revoking a
key revokes this surface too.

## OpenAI protocol

Base URL: `https://<host>/api/v1/openai`

| Endpoint | What it is |
| --- | --- |
| `POST /chat/completions` | One turn of the agent, streaming or not. |
| `GET /models` | The org's agents whose gateway is on. |

```ts
import OpenAI from 'openai'

const client = new OpenAI({
  baseURL: 'https://<host>/api/v1/openai',
  apiKey: process.env.ORCHID_API_KEY,
})

const stream = await client.chat.completions.create({
  model: 'support', // the agent's name, or its id
  messages: [{ role: 'user', content: 'where is my order?' }],
  stream: true,
})
```

Anything speaking this protocol works the same way — OpenRouter clients, the
Vercel AI SDK, LangChain:

```ts
import { createOpenAI } from '@ai-sdk/openai'
import { streamText } from 'ai'

const orchid = createOpenAI({
  baseURL: 'https://<host>/api/v1/openai',
  apiKey: process.env.ORCHID_API_KEY,
})

// orchid.chat(), not orchid(): the AI SDK's default factory speaks the
// Responses API, and this surface is Chat Completions.
const result = streamText({ model: orchid.chat('support'), prompt: 'hey' })
```

A non-streaming completion carries an extra `orchid` object naming the
conversation, the agent and the turn's trace, so a caller can open what it just
ran in the dashboard.

## Anthropic protocol

Base URL: `https://<host>/api/anthropic`

| Endpoint | What it is |
| --- | --- |
| `POST /v1/messages` | The same turn, in Anthropic's message format. |
| `POST /v1/messages/count_tokens` | An estimate of the text you are about to send. |

```ts
import Anthropic from '@anthropic-ai/sdk'

const client = new Anthropic({
  baseURL: 'https://<host>/api/anthropic',
  apiKey: process.env.ORCHID_API_KEY,
})

const message = await client.messages.create({
  model: 'support',
  max_tokens: 1024, // accepted and ignored: the agent's config decides
  messages: [{ role: 'user', content: 'where is my order?' }],
})
```

The key may travel as `x-api-key` or as `Authorization: Bearer …` on either
surface. `count_tokens` is an estimate of the request's own text: what a turn
actually spends depends on the agent's prompt, its history window and the tools
it ends up calling, none of which exist until it runs.

## Conversations

Send `x-conversation-id: <your id>` to keep several calls in one conversation —
one thread in the dashboard, one thread in the agent's memory of it. Without the
header every call is a new conversation.

The client's `messages` array is the context of the turn: an SDK re-sends the
whole thread on every call, and that transcript is what the agent answers
against. A client `system` message is carried in as context but does not replace
the agent's own prompt.

Because a conversation runs one turn at a time, a second call into a
conversation whose turn is still running is refused as busy rather than queued
behind an answer the caller will never read. Use a fresh conversation id for
concurrent calls.

## Streaming and tools

Streaming is real: tokens are forwarded as the model produces them, from the
first one, and continue across the agent's tool calls. The answer on this surface
is the completion's own text — the agent has no send tool here, because the
response you are reading *is* the delivery — and a message another outward tool
sends (an email, a text to someone else) arrives in the stream too.

Streaming from token one has one consequence worth knowing: the agent's reply
guards run on the finished completion, so a completion the agent then rejects
(blocked, degenerate, or an echo of a tool's output) has already been read by the
client. Such a stream ends in the protocol's error event rather than a normal
stop, since no provider protocol can unsend bytes; the stored turn holds the
answer the agent settled on. A non-streaming call never shows the rejected text
at all — it returns the settled answer.

Requests are retried upstream only before the first byte has reached the client.
After that a failure travels as the protocol's in-stream error — an `error` frame
on the OpenAI surface, an `error` event on the Anthropic one — which is what both
SDKs raise on. Hanging up mid-answer stops the turn.

## Reasoning

The agent has a configured reasoning level, and a request may replace it for
that call — the hidden thinking phase runs before the first token, so it is the
difference between an answer that starts in a couple of seconds and one that
starts in ten. A chat UI usually wants `low` or none; an offline batch can
afford `high`.

```ts
await client.chat.completions.create({
  model: 'support',
  reasoning_effort: 'none', // none | minimal | low | medium | high
  messages: [{ role: 'user', content: 'where is my order?' }],
})

await anthropic.messages.create({
  model: 'support',
  max_tokens: 1024,
  thinking: { type: 'enabled', budget_tokens: 8000 }, // or { type: 'disabled' }
  messages: [{ role: 'user', content: 'where is my order?' }],
})
```

OpenRouter's `reasoning: { effort }` / `reasoning: { enabled: false }` is read
the same way as `reasoning_effort`. A turn takes a level rather than a budget, so
a token budget is bucketed onto one (under 4k → `low`, under 16k → `medium`,
above → `high`, zero → off), and OpenAI's `minimal` is read as off. Send nothing
and the agent's own setting stands. A level that is not one of these is a 400
rather than a silently different answer.

## Your own tools

The tools you declare in `tools` are added to the agent's own, and the model is
offered one list. Who runs what differs:

- the agent's tools (its capabilities, MCP servers, authored tools) run on the
  platform, inside the turn, and you never see them;
- yours run in your process, so a call to one ends the response with
  `finish_reason: "tool_calls"` (Anthropic: `stop_reason: "tool_use"`) and waits
  for you to send the results.

Send them back the way you would to the provider — a `tool` message per call on
the OpenAI surface, `tool_result` blocks on the Anthropic one — with the same
`x-conversation-id`:

```ts
const first = await client.chat.completions.create({
  model: 'support',
  messages: [{ role: 'user', content: 'is my order out for delivery?' }],
  tools: [{ type: 'function', function: { name: 'get_location', parameters: {} } }],
})

const call = first.choices[0].message.tool_calls![0]
await client.chat.completions.create({
  model: 'support',
  messages: [
    { role: 'user', content: 'is my order out for delivery?' },
    first.choices[0].message,
    { role: 'tool', tool_call_id: call.id, content: 'Paris' },
  ],
  tools: [{ type: 'function', function: { name: 'get_location', parameters: {} } }],
})
```

The second call continues the *same* turn rather than starting one: the suspended
run is saved server-side and resumed with your results, so the platform tools it
had already called before suspending are not called a second time (nothing it
did is repeated, and its trace is one turn). A conversation holds one suspended
run, resumable for 24 hours and answerable once — sending the same results twice
gets you a 400 rather than a replay.

Details worth knowing:

- a tool named the same as one of the agent's tools stays the agent's, and your
  declaration is dropped (visible on the trace);
- names must match `[A-Za-z0-9_-]{1,64}`, a name is declared once, and there is a
  cap of 64 tools per request;
- keep declaring your tools on the follow-up request — an SDK does this for you;
- a result you leave out is answered for you with an error, and a result over
  32k characters is truncated before the model reads it;
- your tools cannot reach anything of the platform's: they are names and schemas
  in a prompt, executed entirely by you.

## Forcing, forbidding, and shaping the answer

`tool_choice` is about this request rather than the agent, so it is honoured:
`"none"` makes the agent answer without calling anything, `"required"` (Anthropic
`{type:'any'}`) makes it open with a tool call, and naming a tool picks which.
A turn is a loop, so a forced choice constrains the *first* completion only —
re-forcing a tool at every step is a run that could never reach an answer —
while `"none"` holds throughout. A named tool must be one you declared in
`tools`: the agent's own tools are its configuration, not yours to select, and
naming something you did not declare is a 400.

`response_format` shapes the answer, on the OpenAI surface (the Anthropic
protocol has no equivalent field):

```ts
await client.chat.completions.create({
  model: 'support',
  response_format: {
    type: 'json_schema',
    json_schema: { name: 'order', schema: { type: 'object', properties: { id: { type: 'string' } } } },
  },
  messages: [{ role: 'user', content: 'where is my order?' }],
})
```

`{type:'json_object'}` asks for JSON, `{type:'json_schema'}` holds the model to
your schema, and `{type:'text'}` is the default. It applies to every completion
of the turn, so the answer keeps its shape however many steps the agent took to
get there.

## Passthrough mode

Agents → your agent → **API gateway** → *Passthrough mode*. Off, a turn is the
agent as configured: your `system` message is context that rides along, and the
agent's own prompt, tools and memory decide the answer. On, the caller owns the
prompt. Your `system` (or `developer`) messages become the turn's system
prompt, only the tools you declared in the request are offered, and nothing of
the agent's skills, memory or reply guards enters the turn. The agent still
picks the model and reasoning level, and the turn is stored and traced like any
other — which is what makes it a way to run a plain model call you already
wrote the prompt for through this platform's observability.

The one thing of the agent's that does enter the turn is its stored system
prompt, appended after yours. It is empty until someone writes it: the
dashboard, or an eval-driven fix you accept. That is how a passthrough agent
is improved without touching your code — your prompt stays the contract and
keeps winning on every request, the platform's amendments ride behind it, and
the evals measure the two together exactly as production runs them. Your
prompt itself is only recorded (as the spec the evals are generated from),
never written back to the agent.

Passthrough is a property of the gateway, not of the agent: the same agent keeps
answering as itself over SMS, Slack or the dashboard. It applies to the managed
harness; an agent running on your own server (`executionMode: 'sdk'`) receives
the turn as it always did. A turn parked on one of your tools finishes in the
mode it started in, even if the toggle is flipped before the results come back.

## Errors

Errors use each provider's envelope, so an SDK raises the exception type a client
already handles:

| Situation | Status |
| --- | --- |
| Missing or invalid API key | 401 |
| Key without the `agents:chat` scope | 403 |
| No such agent in the org | 404 |
| The agent's gateway is off | 403 |
| A turn is already running for the conversation | 409 |
| The turn itself failed | 500 |

A `tool_choice` naming a tool you did not declare, or a `response_format` that
is not one of the three shapes, is a 400 — the turn never runs, rather than
quietly answering something else.