Cloudflare AI Gateway 2026: One API Key for 100+ Models

Cloudflare's AI Gateway gives you one endpoint, caching, and fallbacks across every major LLM provider - with a generous free tier.

💡 What You Will Learn

Cloudflare's AI Gateway gives you one endpoint, caching, and fallbacks across every major LLM provider - with a generous free tier.

📜 Table of Contents

• The Zero-Ops Gateway
• What You Get
• The Quickstart
• When to Use It vs LiteLLM
• FAQ

The Zero-Ops Gateway

Self-hosting an LLM gateway (like LiteLLM) means running a server. Cloudflare AI Gateway is the managed version: you get the same benefits - one endpoint, key management, observability - with zero infrastructure, running on Cloudflare's edge. For small teams and prototypes, it removes the 'run a proxy' step entirely.

What You Get

• One endpoint (gateway.ai.cloudflare.com/v1/) for OpenAI, Anthropic, Gemini, Mistral, Groq, and 100+ others.
• Automatic retries and fallbacks: if the primary provider fails or rate-limits, the gateway tries the next.
• Caching: repeated identical requests return cached responses - big cost saver for stable prompts.
• Logging and analytics: every request logged, tokens counted, cost tracked.
• Rate limiting: per-key or per-IP controls.
• Free tier: generous monthly requests at no cost - enough for development and small apps.

The Quickstart

1. Create a Cloudflare account, open AI Gateway, create a gateway.
2. Add a provider: OpenAI (paste your key) or any of the 100+.
3. Point your code at the gateway URL with the provider's model name:

from openai import OpenAI
client = OpenAI(
    base_url="https://gateway.ai.cloudflare.com/v1/YOUR_ACCOUNT/YOUR_GATEWAY/openai",
    api_key="YOUR_OPENAI_KEY",
)
resp = client.chat.completions.create(model="gpt-4o-mini", messages=[{"role":"user","content":"hi"}])

1. Set fallbacks in the dashboard: if openai fails, route to groq or anthropic.

When to Use It vs LiteLLM

• Cloudflare AI Gateway: no server to run, edge caching, generous free tier - great for startups and small apps.
• LiteLLM (56,266 stars): full control, self-hosted, more routing logic - for teams with specific compliance or custom needs.
• Both: they are complementary; some teams run LiteLLM inside and Cloudflare in front.

FAQ

Is it really free? The free tier covers a substantial number of requests monthly; paid plans start when you scale.

Does caching affect correctness? Only identical requests are cached; dynamic prompts bypass it or use short TTLs.

Can I use my own keys? Yes - you bring your provider keys; Cloudflare proxies and manages them.

Is my data seen by Cloudflare? Requests pass through Cloudflare's edge; enterprise plans offer data localization options.