Batch API

A batch is up to 10,000 chat completions submitted together to the provider’s own batch API — OpenAI’s Batch API, Anthropic’s Message Batches, Google’s Gemini batch mode, Mistral’s batch jobs, Alibaba Model Studio’s and xAI’s Batch APIs — on your own key, at the provider’s batch price, with an answer within 24 hours. Any model with a :batch row in the catalogue takes one; the row is the model’s price in a batch, typically half the live price.

Submitting

{
  "model": "openai/gpt-5.4",
  "requests": [
    {"custom_id": "doc-1", "body": {"messages": [{"role": "user", "content": "Summarise: ..."}], "max_tokens": 300}},
    {"custom_id": "doc-2", "body": {"messages": [{"role": "user", "content": "Summarise: ..."}], "max_tokens": 300}}
  ]
}

• One model per batch, named with or without the suffix (openai/gpt-5.4 or openai/gpt-5.4:batch). provider takes the same routing preferences as a chat completion.
• Each request is {"custom_id", "body"}: an id of 1 to 64 letters, digits, _ or -, unique in the batch, and a chat-completion body without model or stream. Routing fields in a body (models, provider, plugins, session_id, run_id) are dropped; a server tool is refused, because it runs inside a live request.
• The same rules as live traffic. Every message text passes your guardrails and content filters, and what a filter redacts is what is submitted. The hold placed on your balance is the worst case of every request; it is released when the batch is settled.
• endpoint may only be /v1/chat/completions and completion_window only 24h; both are optional. The X-Autonomous-Relay-Agent header attributes the batch to an agent, as on chat completions.

Following it

A batch is OpenAI’s batch object: status (in_progress, completed, failed, cancelled or expired), request_counts, created_at, expires_at and, once settled, usage.cost. Nothing runs between your calls: reading a batch asks the provider when the last answer is more than 30 seconds old, and the first read that finds it ended settles it. /results streams one JSON line per request — {"custom_id", "response": {"status_code": 200, "body": {...}}, "error": null}, the body a chat completion — read from the provider as it is sent; while the batch is still running it is a 409. /cancel stops a running batch; one that has ended is a 409.

Billing and data

A batch is billed once, when it ends, on the sum of its successful requests’ usage at the :batch price — the way one completion is metered, with the same ledger lines. Nothing of the requests or results is stored here: the provider holds them under your account, for its own retention, and the gateway keeps only the bookkeeping (whose batch, which provider, how many items, what it cost), deleted about 30 days after settlement (What we store).