Batches
Reference for client.batches, BatchHandle and JobError.
batches.submit()
batch = await client.batches.submit(
requests: list[dict] | str,
completion_window: str = "batch",
*,
providers: list[int] | None = None,
metadata: dict[str, str] | None = None,
validate_params: bool = True,
) -> BatchHandle| Parameter | Meaning |
|---|---|
requests | OpenAI batch lines, or a path to a JSONL file of them. Each line is {"custom_id", "method", "url", "body": {"model", ...}}. |
completion_window | "async" / "1h" or "batch" / "24h", like submit(sla=...). A "24h" batch usually takes minutes to a few hours; 24 hours is the maximum. |
providers | An allow-list for every line without its own body.provider: those lines are planned only onto these providers, cheapest first. Non-empty, unique, each an integer from 1 to 4294967295. |
metadata | Stored in plaintext on the batch record. |
validate_params | Check each line's input against its model's schema, as submit does. |
Line rules, all checked before anything is sealed (violations raise ValidationError):
body.modelis required.urlis/v1/responses(the default) or/v1/embeddings, and every line uses the same one.custom_idis optional, 1–64 characters and unique in the batch. It travels sealed.max_rate_in,max_rate_out,units_outandproviderinbodyset the line's order terms; the rest ofbodyis the model input. The ceilings are USD per 1M units, as decimal strings ("0.05") orDecimal, each optional; anintorfloatis refused.provideris a provider id, an integer from 1 to 4294967295: the line is planned onto that provider only, overridingproviders=.providers=must be a non-empty list of unique provider ids.- An empty batch is refused.
Every line is planned: one POST /v1/batches with no file sends, per model, pair of
ceilings and set of allowed providers, the line count and summed units; a restricted entry names
its set as providers. The coordinator answers which providers within the ceilings take how
many lines at which ask, cheapest first, and plans a restricted entry onto its allowed providers
only. No provider gets more lines than its on-chain capacity leaves free. Each planned line signs
its provider's ask and is designated to it.
A restricted line (providers=, or body.provider) never rests. If the plan leaves any
over, because the allowed providers' asks do not clear the ceilings or they lack the capacity,
the whole batch raises ValidationError before anything is sealed or signed, naming the model,
how many lines could not be placed and the allowed providers.
An unrestricted line the plan cannot place rests at its ceilings as an open order sealed
to the coordinator's verified escrow key, which requires a client built with verifier= (else
EscrowKeyUnverified, before anything is sealed). A side with no ceiling rests at the market
rate, the cheapest live ask's, read by one more plan. If lines with no ceiling at all do not all
fit in the window, or there is no live ask to take a market rate from, ValidationError is
raised and nothing is signed.
Then every line is sealed, signed and paid for in your process. The fees on top of each line's
cap are read once, from one quote. The file is uploaded (POST /v1/files, purpose=batch) and
the batch created (POST /v1/batches).
batches.get()
batch = client.batches.get(batch_id: str) -> BatchHandleRe-attaches to a batch. Makes no network call.
BatchHandle
batch.id -> str
batch.job_ids -> list[str] | None
batch.output_file_id -> str | None
batch.error_file_id -> str | None
batch.request_counts -> dict | None
await batch.status() -> str
await batch.results(timeout: float | None = None) -> list[TextResult | MediaResult | EmbeddingResult | JobError]
await batch.consume(on_result, on_error=None, timeout=None) -> None
await batch.cancel() -> None| Member | Meaning |
|---|---|
job_ids | Each line's job id, in input order. Set by submit; None on a re-attached handle. |
output_file_id, error_file_id, request_counts | Copied from the batch object on each read. |
status() | One GET /v1/batches/{id}; returns the batch status. |
results() | Waits until the batch is terminal, then returns every line's result or JobError. |
consume() | Same wait, then calls on_result or on_error per line. Callbacks may be plain functions or coroutines; coroutine callbacks run concurrently. |
cancel() | Cancels lines that are still open. A line a provider has already claimed runs to its end. |
results() and consume():
- Terminal statuses are
completed,failed,expiredandcancelled. - Results arrive in file order (every settled line, then every failed one), not input order.
Match on
.custom_id, or on.job_idagainstjob_ids. - Waiting polls once a minute for the first 15 minutes, every 3 minutes for the rest of the
first hour, and every 10 minutes after that. Each poll is one
GET /v1/batches/{id}, whatever the line count. - The default timeout is the batch's completion window. On expiry they raise
WaitTimeout, whose.job_idis the batch id. - A batch that ends
failedhad its input file refused, and raisesBatchFailedwith.batch_id.
JobError
A line that never delivered.
| Field | Meaning |
|---|---|
.message | Human-readable reason. |
.type | For a line that became a job: provider_fail, reclaim, cancelled or expired. For a line refused before it became one: the coordinator's refusal code. |
.job_id | The line's job id, when it has one. |
.custom_id | None: the label is sealed, and a failed line has no sealed result to read it from. Match on .job_id. |
.raw | The error object as received. |