Batch API

Batch API — in more detail

A batch API is an asynchronous processing mode offered by AI providers such as OpenAI, Anthropic or Google. Instead of answering each request individually in real time, users upload a bundle file with hundreds or thousands of requests; the provider processes them off-peak and returns the results within a service-level window (commonly 24 hours). In exchange, the token price typically drops by around 50 % compared to the real-time API.

Example / practical context

Common use cases are non-interactive workloads: classifying a large corpus, generating embeddings for a knowledge base, analyzing customer feedback at scale, or producing synthetic training data. The workflow is always similar — prepare a JSONL file of requests, upload it, wait for the completed status, then download the results file.

Delineation from similar terms

A batch API differs from the regular (synchronous) API in latency and price: synchronous means answers in seconds at full tariff; batch means answers in hours at a discounted tariff. It should also not be confused with streaming, which delivers a single response token by token within one request.

More on this topic

See everything in one place:API Usage→LLM Pricing→

Glossary$/MTok (Cost per Million Tokens)Standard pricing unit for AI APIs — cost in US dollars per one million processed tokens. Listed separately for input, output and sometimes cache.GlossaryInput TokenTokens you send to an AI model in an API call — your prompt, the context, attached documents. Billed separately from output tokens and usually much cheaper.GlossaryOutput TokenTokens an AI model produces as its response. Billed separately and usually three to five times more expensive than input tokens because the model has to actively generate them.