A retry budget for OpenAI-compatible gateway clients

APIClaw serves Claude, GPT, Gemini, DeepSeek and Qwen behind one base URL, so the client-side behaviour that matters is not the provider SDK but the retry policy you wrap around it.

Four things break when you repoint an OpenAI-compatible client: the model name, the tool-call schema, the streaming chunk shape, and the retry policy. The first three fail loudly on the first request. The fourth fails quietly, a week later, on a bill.

The budget we settled on: two retries, jittered exponential backoff starting at 500 ms, a 30 second wall-clock cap per user action, and a hard fail on 401 and 403 so a bad key never spends the budget. Only 429, 408, 5xx and socket resets are retryable.

Why a cap instead of a retry count alone: an agent that retries forever turns a provider outage into a local outage. A wall-clock cap keeps the failure bounded and visible, and it makes the retry count a per-attempt decision rather than a global one.

Re-reads dominate agent token spend, so the retry budget is only half the design. The other half is caching the system prompt and the tool schema between turns on the same base URL.

References: APIClaw documentation, Claude Code API setup, flat-rate pricing.