Building a Multi-Model AI Router in Python

[chawkian]

Building a Multi-Model AI Router in Python

Most applications send every request to the same model. A simple classifier that routes easy requests to cheap models and hard requests to capable ones can cut costs by 50-70% without measurable quality loss.

OpenRouter does this as a service. You can build the same pattern yourself with any OpenAI-compatible API — including AIWave, which provides all the models below through one endpoint.

The Routing Strategy

The idea: classify each request by estimated complexity, then route to the appropriate model.

Tier	Model	Use for	Cost (vs GPT-4o)
1 (Cheap)	DeepSeek V4 Flash	Simple queries, summaries, classification	94% cheaper
2 (Mid)	GLM-5	Chinese content, moderate reasoning	94% cheaper
3 (Premium)	DeepSeek V4 Pro	Complex coding, multi-step reasoning	83% cheaper

Implementation

from openai import OpenAI

client = OpenAI(api_key="***", base_url="https://aiwave.live/v1")

def classify_complexity(prompt: str) -> str:
    """Route based on simple heuristics."""
    indicators = [
        "debug", "fix", "refactor", "optimize",
        "complex", "architecture", "multi-step",
        "analyze", "compare", "benchmark"
    ]
    if len(prompt) > 3000:
        return "premium"
    if any(w in prompt.lower() for w in indicators):
        return "premium"
    if len(prompt) > 1000:
        return "mid"
    return "cheap"

MODELS = {
    "cheap": "deepseek-chat",
    "mid": "glm-4-plus",
    "premium": "deepseek-reasoner"
}

def route_chat(prompt: str) -> str:
    tier = classify_complexity(prompt)
    response = client.chat.completions.create(
        model=MODELS[tier],
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

Cost Impact

A typical app: 10,000 API calls/month.

• 70% simple (Tier 1): 7,000 calls × DeepSeek V4 Flash
• 20% moderate (Tier 2): 2,000 calls × GLM-5
• 10% complex (Tier 3): 1,000 calls × DeepSeek V4 Pro

vs all on GPT-4o ($2.50/$10.00):

Approach	Monthly Cost
All GPT-4o	$400+
All DeepSeek V4 Pro	$33
Routed (3-tier)	$19

The 3-tier router saves roughly 95% compared to GPT-4o. Even compared to using DeepSeek V4 Pro for everything, routing saves 42%.

Production Considerations

• Latency classification: For real-time apps, use token count or prompt length instead of keyword matching
• Fallback: If the cheap model fails or times out, escalate to the next tier
• Monitoring: Track which tier handles what percentage of requests — adjust thresholds over time
• A/B testing: Compare routed vs single-model quality with a sample of requests

All models above are available through AIWave with a single API key and PayPal payments. The $1 free credit covers thousands of tier-1 requests to test the pattern before committing.

---

Compare routing approaches? Check OpenRouter for a hosted solution, or build your own with AIWave models.

---

AIWave - 50+ models, one key. $1 free.

---

Resources

• AIWave developer blog
• AIWave Python SDK