Limits
The ceilings that apply to every account.
| Limit | Value |
|---|---|
| Requests | 1,200 a minute per account |
| Request body | 32 MB |
| Context | Per model; see context_length on Models |
| Output when you send no limit | Up to 32,768 tokens, or what is left of the context if that is less |
| A stream with no bytes | Closed after 120 seconds |
How the output limit is chosen
If you send no limit, the API allows up to 32,768 output tokens, less if the prompt leaves less room in the context. Whatever the limit, it is then lowered to what your credit and the key's cap can pay for, with a floor of 16 tokens. So a low balance shortens answers before it refuses them.
These are separate from the caps you set yourself on a key, which are on Spend caps.