Rate Limits and Quotas

Inference Space performs a unified rate-limit check (checkGatewayLimit) before processing requests on each capability route (LLM / images / ASR / TTS / OCR) to protect backend capacity and control organization usage.

Private deployment uses the same rate-limit mechanism; manage thresholds in your own console → Enterprise private deployment.

Rate-limit dimensions

Rate limits are evaluated across the following dimensions. Manage exact quotas and alert thresholds in the console:

• API key: An independent rate and quota for each key, making application isolation, auditing, and revocation easier.
• Organization: Organization-wide limits and budget driven by balance or quota, aggregated across all keys in the organization.
• Application configuration: Configure different limits and alerts in the console for different use cases.

Rate-limit response: 429

Rate-limited requests return 429, with the error type set to rate_limit_exceeded and a recommended wait time in the retry-after response header:

HTTP/1.1 429 Too Many Requests
retry-after: 12
content-type: application/json

{
  "error": {
    "message": "Rate limit exceeded",
    "type": "rate_limit_exceeded"
  }
}

• Streaming requests also return 429: when rate-limited, they do not enter a streaming response and instead return a 429 JSON error body.
• Rate-limit errors for capabilities such as TTS also include retry-after.

Backoff and retry

# When 429 is returned, read retry-after and sleep for the specified seconds before retrying.
resp=$(curl -s -D - -o /dev/null "https://ai.inf.space/v1/chat/completions" \
  -H "Authorization: Bearer $TOS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen-plus","provider":"dashscope","messages":[{"role":"user","content":"hi"}]}')
echo "$resp" | grep -i '^retry-after:'

import os, time, requests

def call_with_retry(payload, max_retries=3):
    url = "https://ai.inf.space/v1/chat/completions"
    headers = {"Authorization": f"Bearer {os.environ['TOS_API_KEY']}"}
    for attempt in range(max_retries + 1):
        resp = requests.post(url, headers=headers, json=payload)
        if resp.status_code != 429:
            return resp
        # Back off using retry-after; fall back to exponential backoff when absent.
        wait = int(resp.headers.get("retry-after", 2 ** attempt))
        time.sleep(wait)
    return resp

async function callWithRetry(payload, maxRetries = 3) {
  const url = "https://ai.inf.space/v1/chat/completions";
  const headers = {
    Authorization: `Bearer ${process.env.TOS_API_KEY}`,
    "Content-Type": "application/json",
  };
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const resp = await fetch(url, {
      method: "POST",
      headers,
      body: JSON.stringify(payload),
    });
    if (resp.status !== 429) return resp;
    const wait = Number(resp.headers.get("retry-after") ?? 2 ** attempt);
    await new Promise((r) => setTimeout(r, wait * 1000));
  }
}

Capacity planning recommendations

• Estimate tokens, image counts, and durations from real business samples before setting quotas and alerts in the console. Include tool calls, retrieved context, and reasoning (extended thinking) tokens in the budget; see Pricing and billing.
• Centralize concurrency and retries, such as in the callWithRetry function above, and consistently follow retry-after.
• For long-running requests such as images and long contexts, set the client timeout to at least 600 seconds. Distinguish 429 (backoff and retry) from transient 5xx errors (limited retries); see Error codes and handling.