Building a Multi-Model AI Chatbot with Python — Route by Task Complexity (2026 Guide)

[chawkian]

Why Multi-Model Routing Matters

Most AI chatbots use a single model for everything — simple but expensive. A smarter approach: route different tasks to different models based on complexity, and let your routing logic handle cost optimization automatically.

The cost difference between models isn't marginal — it's orders of magnitude:

Model	Input (1M tokens)	Output (1M tokens)	Context	Use Case
ERNIE Tiny 8K	$0.1781	$0.1781	8K	Simple Q&A, greetings
DeepSeek V4 Pro	$0.42	$0.84	1M	Complex reasoning, long docs

A greeting ("Hi, how are you?") costs ~$0.000024 with ERNIE Tiny. Sending it to DeepSeek V4 Pro costs ~$0.000058 — 2.4x more expensive for zero quality gain. At scale, routing becomes a real lever.

The Complete Project

Here's a production-ready multi-model chatbot using AIWave's OpenAI-compatible API:

multi_model_bot/
├── bot.py          # Main entry point
├── router.py       # Task complexity classifier
├── models.py       # Model configurations
├── config.py       # API key and settings
└── requirements.txt

config.py

API_KEY = "your-aiwave-api-key"
BASE_URL = "https://aiwave.live/v1"

MODELS = {
    "simple": "ernie-tiny-8k",
    "complex": "deepseek-v4-pro",
}

PRICING = {
    "ernie-tiny-8k": {"input": 0.1781, "output": 0.1781},
    "deepseek-v4-pro": {"input": 0.42, "output": 0.84},
}

router.py — The Intelligence Layer

import re
from typing import Literal

RouteDecision = Literal["simple", "complex"]

COMPLEX_PATTERNS = [
    r"\b(analyze|analysis|compare|evaluate|explain why)\b",
    r"\b(algorithm|architecture|design|refactor|optimize)\b",
    r"\b(debug|troubleshoot|fix|error|exception)\b",
    r"\b(write|implement|create|build|develop)\b",
    r"\b(summarize|extract|translate).{0,20}(long|document|paper)\b",
    r"\b(what if|how would|assuming|given that)\b",
]

SIMPLE_PATTERNS = [
    r"^(hi|hello|hey|thanks|bye|ok)\b",
    r"^(what is|who is|where is|when is|define)\b",
    r"\b(how do i|how to)\b",
]

def classify_complexity(message: str) -> RouteDecision:
    msg_lower = message.lower().strip()
    
    if len(msg_lower) < 30:
        if any(re.search(p, msg_lower) for p in COMPLEX_PATTERNS):
            return "complex"
        return "simple"
    
    complex_score = sum(1 for p in COMPLEX_PATTERNS if re.search(p, msg_lower))
    simple_score = sum(1 for p in SIMPLE_PATTERNS if re.search(p, msg_lower))
    
    if complex_score >= 1:
        return "complex"
    if simple_score >= 1 and complex_score == 0:
        return "simple"
    
    return "simple" if len(msg_lower) < 200 else "complex"

models.py — Model Interaction

import openai
from config import API_KEY, BASE_URL, MODELS, PRICING

client = openai.OpenAI(api_key=API_KEY, base_url=BASE_URL)

conversation_history: dict[str, list[dict]] = {
    "ernie-tiny-8k": [],
    "deepseek-v4-pro": [],
}

def call_model(model_key: str, message: str, system_prompt: str = "") -> dict:
    model_name = MODELS[model_key]
    history = conversation_history[model_name]
    
    if system_prompt and not history:
        history.append({"role": "system", "content": system_prompt})
    
    history.append({"role": "user", "content": message})
    
    response = client.chat.completions.create(
        model=model_name,
        messages=history,
        temperature=0.3 if model_key == "complex" else 0.7,
        max_tokens=2048 if model_key == "complex" else 512,
    )
    
    assistant_msg = response.choices[0].message.content
    history.append({"role": "assistant", "content": assistant_msg})
    
    if len(history) > 20:
        history[:] = history[-16:]
    
    pricing = PRICING[model_name]
    cost = (
        response.usage.prompt_tokens * pricing["input"] / 1_000_000
        + response.usage.completion_tokens * pricing["output"] / 1_000_000
    )
    
    return {
        "text": assistant_msg,
        "model": model_name,
        "route": model_key,
        "input_tokens": response.usage.prompt_tokens,
        "output_tokens": response.usage.completion_tokens,
        "cost_usd": cost,
    }

bot.py — Putting It Together

from router import classify_complexity
from models import call_model

SYSTEM_PROMPTS = {
    "simple": "You are a helpful assistant. Keep responses concise and friendly.",
    "complex": (
        "You are a senior software engineer and technical analyst. "
        "Provide thorough, well-reasoned responses with code examples "
        "where appropriate. Think step-by-step."
    ),
}

class MultiModelChat:
    def __init__(self):
        self.total_cost = 0.0
        self.request_count = 0
        self.route_counts = {"simple": 0, "complex": 0}

    def chat(self, message: str) -> dict:
        self.request_count += 1
        route = classify_complexity(message)
        self.route_counts[route] += 1
        
        result = call_model(
            model_key=route,
            message=message,
            system_prompt=SYSTEM_PROMPTS[route],
        )
        
        self.total_cost += result["cost_usd"]
        result["running_total_usd"] = self.total_cost
        result["route_distribution"] = dict(self.route_counts)
        
        return result

if __name__ == "__main__":
    bot = MultiModelChat()
    
    test_messages = [
        "Hi, what can you help me with?",
        "What is a REST API?",
        "Compare ERNIE Tiny vs DeepSeek V4 Pro for a production chatbot "
        "handling 10K daily requests.",
        "Thanks!",
        "Write a Python function that implements a concurrent "
        "task queue with priority scheduling and retry logic.",
    ]
    
    for msg in test_messages:
        result = bot.chat(msg)
        route_label = "SIMPLE" if result["route"] == "simple" else "COMPLEX"
        print(f"Route: {route_label} -> {result['model']}")
        print(f"Cost: ${result['cost_usd']:.6f}")
        print(result["text"])

Understanding the Cost Flow

A realistic 5-message session:

Message	Route	Model	Est. Cost
"Hi, what can you help me with?"	Simple	ERNIE Tiny	$0.000003
"What is a REST API?"	Simple	ERNIE Tiny	$0.000012
Complex comparison question	Complex	DeepSeek V4 Pro	$0.000137
"Thanks!"	Simple	ERNIE Tiny	$0.000004
Concurrent task queue code	Complex	DeepSeek V4 Pro	$0.000182

Session total: ~$0.000338 for 5 real queries.

At 10,000 daily requests where ~60% are simple: $6/day vs $10/day — that's $120/month saved by routing intelligently.

Production Considerations

1. Replace heuristic router with a classifier. Use a free model like GLM-4.7-Flash ($0 on AIWave) to classify complexity with higher accuracy:

def ml_classify(message: str) -> str:
    response = client.chat.completions.create(
        model="glm-4.7-flash",  # Free on budget tier
        messages=[
            {"role": "system", "content": (
                "Classify the user message. Reply ONLY 'simple' or 'complex'."
            )},
            {"role": "user", "content": message}
        ],
        max_tokens=10,
        temperature=0.0
    )
    return response.choices[0].message.content.strip().lower()

2. Add caching. Identical queries shouldn't hit the API twice. Use hash-based cache with TTL.

3. Monitor routing distribution. If 90% route to "complex," your threshold is too low.

Get Started

All models in this article are available on AIWave with an OpenAI-compatible API. New accounts get $5 free credit.

• Full model catalog: aiwave.live/models
• Pricing details: aiwave.live/pricing
• API docs: aiwave.live/docs/chat-completions

---

Discussion: Have you implemented multi-model routing in your projects? What heuristics or classifiers work best for you? Share your experience below!

---

Resources

• AIWave developer blog
• AIWave Python SDK