Azure OpenAI Rate Limit RPM, TPM & Token Bucket Calculator

Verified Nov 17, 2026 Quality 100/100 Confidence 100% 0 Evaluations
VERIFIED QUICK ANSWER

Azure OpenAI enforces dual metering via Tokens-Per-Minute (TPM) and Requests-Per-Minute (RPM). RPM is often derived proportionally at 6 RPM per 1,000 TPM, though ratios vary by model. A request's estimated token count for rate limiting includes prompt tokens, the max_tokens parameter, and the best_of parameter. Exceeding either limit, or violating sub-minute sliding window distribution, triggers an HTTP 429 Too Many Requests response.

Interactive Parameters & Live Verdict

Rule Diagnostics & Recommendations

    Engineering Formula & Verification Notes

    Calculations follow Azure OpenAI technical architecture specifications. Estimated token counts for rate limiting include prompt tokens, max_tokens, and best_of settings. RPM limits are derived proportionally using the standard baseline formula: 1,000 TPM = 6 RPM, though actual model ceilings should be verified via ARM API (properties.callRateLimit.count).

    Worked Engineering Example

    Consider a deployment with a TPM quota of 100,000 tokens per minute. This yields an automatic RPM limit of 600 requests per minute (100k * 6 / 1000). If your application sends requests with a 500-token prompt and a max_tokens setting of 500, each request consumes 1,000 estimated tokens toward the rate limit counter. Dividing your 100,000 TPM quota by 1,000 tokens per request permits a theoretical maximum of 100 requests per minute.

    • max_tokens counts in full toward rate limits regardless of actual response length.
    • Short 1-to-10-second sub-windows can throttle bursts exceeding 10% of the RPM ceiling.

    Common Sizing & Compatibility Mistakes

    Developers often assume rate limits match actual billed token consumption. However, Azure OpenAI calculates rate-limit tokens at request reception using prompt length plus max_tokens and best_of parameters. Another common pitfall is dispatching batch jobs instantly, causing sub-minute sliding window 429 throttling errors.

    • Failing to account for the max_tokens parameter setting in rate limit math.
    • Ignoring sub-minute rate evaluation windows (1 to 10 seconds).

    Azure OpenAI Deployment Baseline Ratios

    Azure OpenAI Deployment Baseline Ratios
    TPM Quota Derived RPM Ratio Max RPM Ceiling
    10,000 TPM 6 RPM per 1,000 TPM 60 RPM
    100,000 TPM 6 RPM per 1,000 TPM 600 RPM
    1,000,000 TPM 6 RPM per 1,000 TPM 6,000 RPM

    Frequently Asked Technical Questions

    Why am I getting a 429 error when my total minute token usage is low?

    Azure OpenAI evaluates requests over short 1 or 10-second sub-windows. Spiking requests instantaneously violates the expected even distribution over the minute.

    Does billing token count match rate limit token count?

    No. Rate limit tokens are estimated upon request arrival using prompt text, max_tokens, and best_of. Billing tokens are computed after completion processing finishes.

    How can I check my exact deployment RPM programmatically?

    You can query the Azure Resource Manager (ARM) API via GET on /accounts/{account}/deployments/{name}?api-version=2024-10-01 and check properties.callRateLimit.count.

    Did this interactive specification answer your scenario accurately?