Model Tier Router — Cost-to-Capability Routing Framework for GPT Models

Model Tier Router is a cost-to-capability routing framework for anyone shipping GPT-based products on hosted APIs, where every request currently gets routed to one model tier by default — either the flagship model for everything (safe, but expensive), or the cheapest model for everything (cheap, but inconsistent).

Why this exists

In 2026, Andrej Karpathy's nanochat became one of the most-watched repos of the year — a full, hackable LLM pipeline (tokenization, pretraining, fine-tuning, evaluation, inference, chat UI) that trains a GPT-2-capability chat model for about $48 on an 8xH100 node. The trick isn't just the price — it's that the entire cost/capability tradeoff collapses into a single parameter, --depth, which automatically retunes every other hyperparameter for the compute budget you choose. That's a genuine advance — if you're training your own model from scratch on a rented GPU cluster. It doesn't help if you're shipping features on hosted GPT-style APIs, where you're not choosing transformer depth — you're choosing, request by request, which model tier to call, and most teams never formalize that decision at all.

Model Tier Router is the prompt- and process-layer answer to the same problem: one deliberate control point that routes each request to the cheapest tier that can actually handle it, instead of a single global default. This is original methodology — not a copy of nanochat's code, training pipeline, or --depth mechanism. It's built for teams calling hosted APIs, not teams training infrastructure.

What's inside

• Task Complexity Scoring Rubric — classify every request type your product handles into a capability tier (fast / standard / reasoning) using a repeatable checklist instead of gut feel
• Tier Routing Decision Tree — the actual routing logic: which signals (input length, ambiguity, multi-step reasoning, stakes of a wrong answer) trigger escalation to a stronger tier, and which signals justify a downgrade
• Escalation & Fallback Pattern — a retry pattern where a fast-tier response that fails a confidence or validation check auto-escalates to a stronger model, without doubling latency on the happy path
• Cost-per-Outcome Tracking Sheet — a template for logging cost per resolved task by tier, so tiering decisions are backed by dollars instead of assumptions
• Prompt Compression Checklist — trims context and instructions per tier so a fast-tier call isn't quietly paying premium-tier prompt-length costs
• Monthly Tier Audit — a recurring review to catch tier drift as usage patterns, features, and provider pricing change

Best for

• Teams shipping GPT-based products on hosted APIs (OpenAI, Anthropic, or others) that default every call to one model tier
• Builders who suspect they're overpaying for capability most requests don't need
• Teams that tried "just use the cheap model everywhere" and got inconsistent quality complaints instead

What you'll need

• Your current API usage or billing export (even a rough one)
• A list of the distinct request types or features your product handles
• Access to at least two model tiers from your provider
• 2–3 hours for a first full pass

Format

Delivered as an 8-page PDF with copy/paste rubrics, the routing decision tree, and fillable templates.

FAQ

Is this affiliated with Andrej Karpathy, nanochat, OpenAI, or Anthropic? No. Model Tier Router is an independent, provider-agnostic framework and isn't affiliated with or endorsed by any AI lab or open-source project it references.

Does this require training my own model? No — it's built for teams calling hosted GPT-style APIs, not teams running training infrastructure.

Will this give me nanochat's code or its exact --depth mechanism? No — this is original methodology inspired by its cost/capability tradeoff philosophy, not a copy of its code, training pipeline, or architecture.

Does this guarantee cost savings? No guarantee — but most teams see meaningful savings once they stop defaulting every call to the top-tier model. Actual savings depend on your usage pattern and provider pricing.