Batch Jobs

1. Overview

Send batch jobs through the Batch API to the SiliconFlow cloud platform. They are not subject to online rate limits, are typically expected to finish within 24 hours, and are priced 50% lower. This works well for workloads that do not need an immediate response, such as large-scale evaluations, classification and extraction, and document processing. URLs for batch result files are valid for one month—download and archive them in time to avoid disruption.

2. Workflow

2.1 Prepare the batch input file

Batch jobs use a .jsonl input file: each line is a full API request body. Requirements:

• Each line must include custom_id, and every custom_id must be unique within the file.
• Each line’s body must include a messages array. Each message has role of system, user, or assistant, and the array must end with a user message.
• You may set the same or different inference parameters per line (e.g. different temperature, top_p).
• If you use the OpenAI SDK with SiliconFlow batch inference, keep model the same across all lines in one input file.
• Per batch: the input file for a single batch may be at most 1 GB.
• Batch input limits: for a single batch job, the input file is at most 1 GB and at most 5,000 lines.

Example input file with two requests:

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "deepseek-ai/DeepSeek-V3.2", "messages": [{"role": "system", "content": "You are a highly advanced and versatile AI assistant"}, {"role": "user", "content": "How does photosynthesis work?"}], "stream": true, "max_tokens": 1514, "thinking_budget": 32768}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "deepseek-ai/DeepSeek-V3.2", "messages": [{"role": "system", "content": "You are a highly advanced and versatile AI assistant"}, {"role": "user", "content": "Imagine a world where everyone can fly. Describe a day in this world."}], "stream": true, "max_tokens": 1583, "thinking_budget": 32768}}

custom_id and body.messages are required; other fields are optional.

For reasoning models, use thinking_budget to cap chain-of-thought length, for example:

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "deepseek-ai/DeepSeek-V3.2", "messages": [{"role": "system", "content": "You are a highly advanced and versatile AI assistant"}, {"role": "user", "content": "How does photosynthesis work?"}], "stream": true, "max_tokens": 1514,"thinking_budget": 32768}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "deepseek-ai/DeepSeek-V3.2", "messages": [{"role": "system", "content": "You are a highly advanced and versatile AI assistant"}, {"role": "user", "content": "Imagine a world where everyone can fly. Describe a day in this world."}], "stream": true, "max_tokens": 1583,"thinking_budget": 32768}}

2.2 Upload the batch input file

Upload the input file first so you can start a batch job later. Example using the OpenAI SDK with SiliconFlow:

from openai import OpenAI
client = OpenAI(
    api_key="YOUR_API_KEY", 
    base_url="https://api.siliconflow.cn/v1"
)

batch_input_file = client.files.create(
    file=open("batch_file_for_batch_inference.jsonl", "rb"),
    purpose="batch"
)
print(batch_input_file)
# ID returned after upload
file_id = batch_input_file.data['id']
print(file_id)

Save the returned id; you will pass it when creating the batch.

2.3 Create a batch job

After a successful upload, create a batch job with the file object’s ID and your parameters.

• For chat models, the endpoint is /v1/chat/completions.
• Completion window can be set from 24 to 336 hours (14 × 24 hours).
• We recommend setting the model via extra_body, e.g. extra_body={"replace":{"model": "deepseek-ai/DeepSeek-V3.2"}}, unless your file already meets OpenAI’s convention with the same model on every line.
• If the model in extra_body does not match the file’s model, the model in extra_body wins.
• metadata can hold extra notes (e.g. job description).

Example (get input_file_id from the previous step):

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY", 
    base_url="https://api.siliconflow.cn/v1"
)

batch_input_file_id = "file-abc123"
client.batches.create(
    input_file_id=batch_input_file_id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
    metadata={
        "description": "nightly eval job"
    },
    extra_body={"replace":{"model": "deepseek-ai/DeepSeek-V3.2"}}
)

This creates a batch job and returns its status.

2.4 Check batch status

You can poll status at any time:

from openai import OpenAI
client = OpenAI(
    api_key="YOUR_API_KEY", 
    base_url="https://api.siliconflow.cn/v1"
)

batch = client.batches.retrieve("batch_abc123")
print(batch)

Example response:

{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "validating",
  "output_file_id": null,
  "error_file_id": null,
  "created_at": 1714508499,
  "in_progress_at": null,
  "expires_at": 1714536634,
  "completed_at": null,
  "failed_at": null,
  "expired_at": null,
  "request_counts": {
    "total": 0,
    "completed": 0,
    "failed": 0
  },
  "metadata": null
}

status may be:

• in_queue: queued
• in_progress: running
• finalizing: finished processing; preparing results
• completed: done; results ready
• expired: not finished within the completion window
• cancelling: cancellation in progress (waiting for in-flight requests)
• cancelled: cancelled

2.5 Cancel an in-progress batch

If needed, cancel a running batch. Status becomes cancelling until in-flight work finishes, then cancelled.

from openai import OpenAI
client = OpenAI(
    api_key="YOUR_API_KEY", 
    base_url="https://api.siliconflow.cn/v1"
)
client.batches.cancel("batch_abc123")

2.6 Retrieve batch results

1. Result files are split by outcome:
   • output_file_id: successful responses.
   • error_file_id: errors for failed requests.
2. Files are kept for 30 days. Download and back them up before expiry; after that they are deleted permanently and cannot be recovered.

2.7 List all batch jobs

List all batch jobs for the account, with pagination:

from openai import OpenAI
client = OpenAI(
    api_key="YOUR_API_KEY", 
    base_url="https://api.siliconflow.cn/v1"
)
response = client.batches.list(limit=20) 
print(response.to_json())
    next_page_response = client.batches.list(limit=20, after=last_batch_id)
    print(next_page_response.to_json())

2.8 Batch expiry (expired)

If a batch does not finish in time, it ends in expired. Pending requests are cancelled; any completed responses are still delivered via the batch output file. Tokens for completed requests are still billed.

Requests that did not run before expiry are written to the error file with content like below. Use custom_id to match them to your input.

{"id": "batch_req_123", "custom_id": "request-3", "response": null, "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}}
{"id": "batch_req_123", "custom_id": "request-7", "response": null, "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}}

3. Supported models

Only /v1/chat/completions is supported. Models:

• deepseek-ai/DeepSeek-V3.1-Terminus
• deepseek-ai/DeepSeek-V3
• deepseek-ai/DeepSeek-R1

4. Input limits

Batch job input limits are separate from per-model online rate limits:

1. Per batch: input file size at most 1 GB.

Note: Batch jobs do not consume your online inference rate limits (requests or tokens at the user × model level for standard API usage).

5. Billing

Batch usage is paid only from your prepaid balance. Pricing:

SiliconFlow inference pricing (unit: CNY per million tokens)