Submit a batch
Seal many requests into one batch file, submit it, and read every line's result.
A batch is many jobs submitted as one file. Every line is sealed and paid for locally before anything is uploaded, so the file the coordinator stores holds routing terms and ciphertext only.
Write the lines
Each line is an OpenAI-style batch request. Put the model in body:
const requests = prompts.map((prompt, i) => ({
custom_id: `row-${i}`, // optional, 1–64 characters, unique in the batch
url: "/v1/responses", // or "/v1/embeddings"; one endpoint per batch
body: { model: "moonshotai/kimi-k3.1-pro:mxfp4", input: prompt, max_output_tokens: 512, max_rate_in: "1.5" },
}));max_rate_in on a line protects it from being overcharged: the line never signs an input rate
above it, and pays less when its provider asks less. It is worth setting on every batch,
because a batch signs and pays for all of its lines in one call, before you see what any of
them was priced at. It caps the input side only; add max_rate_out to cap the output side too.
Set it too low and no provider matches: the line rests and may expire without being
served.
body.modelis required.body.max_rate_in,body.max_rate_out,body.units_outandbody.providerare order terms, taken out of the body; everything else inbodyis the model input. The ceilings and the provider are optional.- Before sealing, the client asks the coordinator for a plan: which providers within each line's ceilings take how many lines, and at which ask. A provider is never given more lines than its on-chain capacity leaves free, and each planned line signs its provider's ask and is pinned to it.
- A line the plan cannot place rests at its ceilings, with the market rate on a side that names
none. A line with no ceiling cannot rest: if the network cannot take all such lines of a
model in the window,
submitraisesValidationErrorbefore anything is signed. body.providerrestricts a line to exactly one provider id, overriding the batch'sproviders. A restricted line never rests (see below).urldefaults to/v1/responses.methodis ignored.requestsmay also be a JSONL string of such lines.
Submit
const batch = await client.batches.submit(requests, "24h");
await store.put(batch.id, { jobIds: batch.jobIds }); // keep both before waitingprovidersis optional: an allow-list ({ providers: [3, 7] }) for every line without abody.providerof its own. Those lines are planned only onto the listed providers, cheapest first. A restricted line never rests: if the allowed providers' asks do not clear its ceilings, or they lack the capacity, the whole batch raisesValidationErrorbefore anything is signed, naming the model, how many lines could not be placed and the allowed providers.- Without a list or a
body.provider, a line goes where the plan puts it, and a line the plan cannot place rests as an open order, which needs a client built with averifier. - The window is
"1h"/"24h", or the tier names"async"/"batch"; it defaults to"24h". A"24h"batch usually takes minutes to a few hours; 24 hours is the maximum. metadata(at most 16 string pairs) is optional and stored in plaintext on the batch record.- Every distinct model's input is checked against its published schema first, as
submitdoes;{ validateParams: false }turns that off.
batch.jobIds lists each line's job id in input order. Until results are opened, the job id is
the only way to correlate a line: your custom_id travels sealed inside the line and the
coordinator never sees it.
Read the results
import { JobError } from "@vorq-ai/client-sdk";
for (const line of await batch.results()) {
if (line instanceof JobError) console.error(line.jobId, line.type, line.message);
else console.log(line.customId, line.cost);
}results() polls until the batch is terminal (completed, failed, expired or cancelled),
then returns every line: a TextResult, MediaResult or EmbeddingResult for a settled line,
or a JobError for one that produced nothing. Lines come back in file order (all settled
lines, then all failed ones), not input order; match them on .customId or .jobId.
To process lines as they are read instead of collecting them:
await batch.consume(
async (result) => save(result.customId, result),
async (error) => logFailure(error.jobId, error.type),
);A batch whose input file was refused raises BatchFailed. A failed line is never an exception.
Re-attach, list, cancel
const batch = client.batches.get(batchId); // no network call
console.log(await batch.status(), batch.requestCounts);
const page = await client.batches.list({ limit: 20 });
const next = page.hasMore ? await client.batches.list({ after: page.lastId! }) : null;
await batch.cancel(); // lines no provider has claimed are cancelled; claimed ones finishA re-attached handle has jobIds: null, which is why you store them at submit time.
With the openai package
After submission, the stock openai client can read the batch too: batches.retrieve,
batches.list and files.content(output_file_id) all pass through
sealingFetch. Creating the batch must go through
client.batches.submit, which seals each line.