Azure OpenAI Rate Limit RPM, TPM & Token Bucket Calculator
Azure OpenAI enforces dual metering via Tokens-Per-Minute (TPM) and Requests-Per-Minute (RPM). RPM is often derived proportionally at 6 RPM per 1,000 TPM, though ratios vary by model. A request's estimated token count for rate limiting includes prompt tokens, the max_tokens parameter, and the best_of parameter. Exceeding either limit, or violating sub-minute sliding window distribution, triggers an HTTP 429 Too Many Requests response.
Interactive Parameters & Live Verdict
Rule Diagnostics & Recommendations
Engineering Formula & Verification Notes
Calculations follow Azure OpenAI technical architecture specifications. Estimated token counts for rate limiting include prompt tokens, max_tokens, and best_of settings. RPM limits are derived proportionally using the standard baseline formula: 1,000 TPM = 6 RPM, though actual model ceilings should be verified via ARM API (properties.callRateLimit.count).
Worked Engineering Example
Consider a deployment with a TPM quota of 100,000 tokens per minute. This yields an automatic RPM limit of 600 requests per minute (100k * 6 / 1000). If your application sends requests with a 500-token prompt and a max_tokens setting of 500, each request consumes 1,000 estimated tokens toward the rate limit counter. Dividing your 100,000 TPM quota by 1,000 tokens per request permits a theoretical maximum of 100 requests per minute.
- max_tokens counts in full toward rate limits regardless of actual response length.
- Short 1-to-10-second sub-windows can throttle bursts exceeding 10% of the RPM ceiling.
Common Sizing & Compatibility Mistakes
Developers often assume rate limits match actual billed token consumption. However, Azure OpenAI calculates rate-limit tokens at request reception using prompt length plus max_tokens and best_of parameters. Another common pitfall is dispatching batch jobs instantly, causing sub-minute sliding window 429 throttling errors.
- Failing to account for the max_tokens parameter setting in rate limit math.
- Ignoring sub-minute rate evaluation windows (1 to 10 seconds).
Azure OpenAI Deployment Baseline Ratios
| TPM Quota | Derived RPM Ratio | Max RPM Ceiling |
|---|---|---|
| 10,000 TPM | 6 RPM per 1,000 TPM | 60 RPM |
| 100,000 TPM | 6 RPM per 1,000 TPM | 600 RPM |
| 1,000,000 TPM | 6 RPM per 1,000 TPM | 6,000 RPM |
Frequently Asked Technical Questions
Why am I getting a 429 error when my total minute token usage is low?
Azure OpenAI evaluates requests over short 1 or 10-second sub-windows. Spiking requests instantaneously violates the expected even distribution over the minute.
Does billing token count match rate limit token count?
No. Rate limit tokens are estimated upon request arrival using prompt text, max_tokens, and best_of. Billing tokens are computed after completion processing finishes.
How can I check my exact deployment RPM programmatically?
You can query the Azure Resource Manager (ARM) API via GET on /accounts/{account}/deployments/{name}?api-version=2024-10-01 and check properties.callRateLimit.count.