Skip to content
Razorpay AI Playbook

Effort settings, model routing, fall-backs

Status: drafted · Time: 30 min · Audience: platform-builder Outcome: Tune per-call settings — effort level, model choice, fall-back path — to match the work’s stakes.

The closing module of Part B. Where B.10 measured cost at scale, B.11 is the per-call lever that shapes the cost-quality trade-off. Three named knobs (effort settings, model routing, fall-backs) that a Black Belt builder tunes to match the work’s stakes. The bar is “cheapest setting that meets the quality bar,” not “highest setting available.”

If you’re short on time

Section titled “If you’re short on time”
  • Effort settings tell the model how eagerly to spend tokens on a single call. Higher effort costs more; sometimes pays off, sometimes does not.
  • Model routing picks which model handles which call. Different models for different jobs; cost-quality differs across model families.
  • Automatic routing is a policy under test, not a magic model. Prove which route actually ran, whether multi-turn sessions behave coherently, and whether quality held before making it a default.
  • Fall-backs define what happens when the primary path fails: rate limit, timeout, classifier flag. A clean fall-back beats a noisy retry loop.
  • Estimate → Execute → Expand. Start with the minimum route that can reliably do the job. Expand context, model capability, or effort only when verification gives you a reason.
  • The discipline: pick by stakes. A 5x cost increase for a 5% quality lift on a low-stakes call is the wrong trade.

The mental model

Section titled “The mental model”
   ┌────────────────────────────────────────────────┐
   │              PER-CALL TUNING                     │
   ├────────────────────────────────────────────────┤
   │                                                  │
   │   1. EFFORT SETTING                              │
   │      How much "thinking" does this call         │
   │      deserve? Low / medium / high (or the      │
   │      provider's named scale).                    │
   │                                                  │
   │   2. MODEL ROUTING                               │
   │      Which model in the family for this call?  │
   │      Smaller models for narrow tasks; larger   │
   │      for open-ended reasoning.                   │
   │                                                  │
   │   3. FALL-BACK PATH                              │
   │      If the primary call fails (rate-limited,  │
   │      timeout, classifier-flagged), what next?  │
   │                                                  │
   └────────────────────────────────────────────────┘

The three knobs interact: a high-effort call on a smaller model may cost more than a default-effort call on a larger model and produce less. A Black Belt builder reads the cost-quality trade-off across knobs, not knob by knob.

Route the task before you route the model

Section titled “Route the task before you route the model”

A recent internal example separates quick repository lookups, planning, and implementation instead of sending all three through the slowest route. Fresh research calls the broader pattern Estimate → Execute → Expand (E3): judge the work’s stakes and minimum evidence, run the smallest credible path, then widen the path only when verification fails.

This is not “always use the cheap model first.” A security review can justify a capable model and high effort on its first pass. The rule is minimum sufficient execution, not minimum possible spend.

Copyable routing card

Section titled “Copyable routing card”

Fill this in before a repeated workflow or expensive agent run:

## Task route
- Outcome: <what must be true when this finishes?>
- Failure cost: <low / medium / high, and why?>
- Minimum evidence: <files, sources, tests, or human checks required>
- Initial route:
  - Context: <smallest relevant folder / source set>
  - Model tier: <fast / balanced / deep-reasoning>
  - Effort: <low / medium / high / provider-supported level>
  - Stop limit: <time, tool calls, or steps>
- Verify with: <test, rubric, source comparison, or reviewer>
- Expand only if: <named failed check or unresolved uncertainty>
- Expansion: <more context, stronger model, higher effort, or human escalation>

Run the same representative task at two plausible routes. Record quality, latency, token/cost usage, and failure shape. Pick the cheaper route only when it passes the same acceptance bar. That small trial is the interactive exercise; a model leaderboard would age faster and teach less.

Knob 1 — Effort settings

Section titled “Knob 1 — Effort settings”

What it is: a per-call parameter that asks the model to invest more or less reasoning time. Anthropic describes effort as how eager Claude is to spend tokens, with supported named levels depending on the model. Other providers may expose a reasoning budget or thinking-token control. Temperature is not an effort setting: it changes sampling variability, not how much reasoning the task deserves. Treat effort as a behavioural signal rather than a guaranteed token ceiling.

When to dial up effort.

  • Hard reasoning tasks. Multi-step proofs, edge-case design decisions, security-shaped reviews where missed nuance costs real money.
  • High-stakes one-shots. A call you cannot easily re-run because the side effect is irreversible.
  • Eval calls. A judging prompt (per B.9) deserves the most thought you can afford; the cost is amortised over many evaluations.

When to dial down effort.

  • Narrow, well-bounded tasks. “Pick the right Blade primitive for this Figma element” does not need the highest effort setting.
  • High-frequency calls. A skill invoked 1000 times a day pays the effort multiplier 1000 times. Keep it low unless the quality lift justifies it.
  • Calls that follow up on prior reasoning. When the previous turn already spent the high-effort budget, the follow-up usually does not need to repeat it.

The trap: defaulting to highest effort because “more is better.” More is more expensive; sometimes more is also better; sometimes it is the same. Measure (per B.9) before assuming.

Knob 2 — Model routing

Section titled “Knob 2 — Model routing”

What it is: which model in the family handles this call. Anthropic’s Claude family includes models tuned for different cost-quality points (Opus, Sonnet, Haiku, etc., with the exact model strings in the program’s pinned configuration). Other providers have analogous tiers.

When to route to a smaller / cheaper model.

  • Narrow tasks with bounded outputs. Classification, extraction, simple transformations. Smaller models often handle these as well as larger ones for a fraction of the cost.
  • High-frequency triage. First-pass classification of incoming work; the second pass can use the larger model on the hard cases.
  • Simple tool calls. A model picking among a small enum of tools rarely needs the full power of the largest model.

When to route to a larger / more capable model.

  • Open-ended reasoning. Architectural decisions, multi-step debugging, ambiguous human input.
  • Tasks that span many sub-skills. A workflow that combines coding + design + review benefits from a model that holds it all in one head.
  • Adversarial cases. When prompt injection or output-classifier shapes are at stake, the larger model’s safer behaviour is usually worth the cost.

The routing layer. LiteLLM (per G.23) is the public open-source proxy that handles routing in the program’s pinned setup. The rule: every call routes through the proxy; the proxy enforces policy on which model handles which call. A Black Belt builder authoring a custom agent (per B.4) routes through the proxy too — same path, same policy enforcement.

Evaluate an automatic router before adopting it

Section titled “Evaluate an automatic router before adopting it”

Razorpay’s LiteLLM gateway now offers auto-router-0.1 as an opt-in experiment. Instead of pinning one model, a caller requests the router and the gateway assigns each call to a tier from prompt complexity. The current tier-to-model mapping is rollout configuration, not curriculum: it can change without changing the lesson.

The promise is useful — cheaper routes for simple work, stronger routes for harder work — but the router adds a second system to evaluate. A good final answer does not prove the right route ran, and a cheap first turn does not prove a later complex turn escalated correctly.

Test it against a fixed-route control before using it for a repeated workflow:

  1. Choose task slices, not favourite prompts. Include short factual or formatting work, bounded everyday reasoning, technical multi-part work, and open-ended trade-offs. Add one safety- or policy-shaped case if the workflow can encounter one.
  2. Set the answer key first. Reuse B.9’s golden set and hard guardrails. Record the fixed route that currently meets the bar; that is the control, not an assumed “best model.”
  3. Prove the executed route. For every call, capture the requested router, executed model, route cause or tier when telemetry exposes it, fall-back or retry, latency, token/cost usage, and outcome score. Missing executed-route evidence is BLOCKED, not an auto-router pass.
  4. Test a conversation, not only isolated prompts. Run a simple → complex → simple sequence in one session. Check whether later turns see enough prior context, whether the complex turn reaches a sufficient route, and whether switching routes destroys useful prompt-cache savings. Do not assume session affinity is enabled.
  5. Compare by slice and promote narrowly. Require the same outcome and guardrail bar as the control, then compare latency, cost, route coverage, and failure rate for each slice. Approve the router for the tested workflow and traffic shape — not “all agent calls.”
Copy this auto-router canary into an eval note 
# Auto-router canary: <workflow>
Control route: <fixed route that currently meets the bar>
Candidate route: auto-router-0.1
Golden set / hard gates: <B.9 suite and required pass conditions>


| Task slice / turn | Expected outcome | Requested route | Executed model + cause/tier | Fallback / retry | Cache read/write | Latency | Cost | Outcome |
|---|---|---|---|---|---|---|---|---|
| Simple, isolated | <answer or state> | <route> | <telemetry receipt> | <none / path> | <tokens> | <ms> | <amount> | PASS / FAIL |
| Complex, isolated | <answer or state> | <route> | <telemetry receipt> | <none / path> | <tokens> | <ms> | <amount> | PASS / FAIL |
| Turn 1: simple | <answer or state> | <route> | <telemetry receipt> | <none / path> | <tokens> | <ms> | <amount> | PASS / FAIL |
| Turn 2: complex | <answer or state> | <route> | <telemetry receipt> | <none / path> | <tokens> | <ms> | <amount> | PASS / FAIL |
| Turn 3: simple | <answer or state> | <route> | <telemetry receipt> | <none / path> | <tokens> | <ms> | <amount> | PASS / FAIL |


Route coverage gate: <N/N calls have executed-model evidence>
Quality and guardrail gate: <threshold; hard failures allowed = 0>
Latency / cost acceptance: <thresholds versus control, by slice>
Session and cache decision: <observed behaviour and accepted trade-off>
Promotion scope: <named workflow, traffic share, owner, rollback trigger>
Missing or mixed route evidence: BLOCKED

Start with a canary share, watch B.10’s team-level cost and outlier signals, and keep the fixed route as a rollback. An automatic router earns wider traffic through evidence; it does not inherit trust from the models behind it.

Knob 3 — Fall-backs

Section titled “Knob 3 — Fall-backs”

What it is: what happens when the primary call fails. Three named failure modes:

Rate-limited. The provider declined the call because the volume is too high. Fall-back: retry after a backoff window. The proxy usually handles this transparently; a Black Belt builder confirms the retry policy is sensible (exponential backoff, capped retries, named upstream alert if it persists).

Timeout. The call took too long. Fall-back: usually retry once with a longer timeout, or fail fast and surface the issue. For interactive calls, fail fast — a 60-second wait is worse than a clear “try again” message. For batch calls, retry with backoff.

Classifier-flagged. The proxy’s output classifier (per G.25) flagged the response. Fall-back: the proxy refuses or redacts; the calling code surfaces the refusal cleanly. Do not retry blindly; reword the prompt or escalate.

The principle. A fall-back path is a named path, not a “whatever happens, retry.” The agent’s failure modes should be understood at design time, with a clear plan for each. The G.21 hard-debugging discipline applies here too: a fall-back that hides the failure is worse than one that surfaces it.

Worked example — tuning a “pre-ship-check” invocation

Section titled “Worked example — tuning a “pre-ship-check” invocation”

The in-repo pre-ship-check definition is a useful routing example. Where a team distributes it as a runtime workflow, route one invocation with E3:

Estimate. The required evidence is bounded: redlines, design-system compliance, tests, PR craft, prompt-craft trace, and behaviour preservation. Layers 1–4 are mostly deterministic checks; Layers 5–6 require judgement. The failure cost is high enough that every layer must run, but not every layer needs the same model or effort.

Execute. Start Layers 1–4 with bounded context, a smaller model, and low or default effort. Route Layers 5–6 to a more capable model at medium effort because they need deeper reading. Cap retries and require each layer to return named evidence rather than a generic pass.

Verify. Run the routed version and the current single-route version against the same golden PR set. Compare missed findings, false positives, latency, and cost. Do not promote the routed version unless it meets the existing quality bar.

Expand. If a deterministic layer returns ambiguous evidence, rerun that layer with more context or the stronger route. Rate-limited: back off and retry within the cap. Timeout: fail with a clear “pre-ship-check did not complete; reviewer should run manually” message. Classifier-flagged: surface the flag; do not retry blindly.

The result is not a promised savings percentage. It is a measured route: cheaper only when the golden set proves quality held, and explicit about when to spend more.

What this is not

Section titled “What this is not”

Not a magic multiplier. The cheapest setting that meets the bar is the right setting; the cheapest setting that does not meet the bar is a regression. Per B.9, evaluate.

Not a substitute for prompt quality. A poorly-crafted prompt does not get rescued by higher effort; the prompt is the lever. Effort and routing are second-order.

Not a way to bypass safety. Routing to a smaller model does not weaken the redline reflex (per G.22), the proxy’s scan (per G.23), or the output classifiers (per G.25). All three apply at every routing decision.

Not free to change. Effort and routing decisions are part of the skill’s contract for cost-attribution purposes (per B.10). Changing them changes the cost rollups; document in the skill’s CHANGELOG.

Common failure modes

Section titled “Common failure modes”

Defaulting to highest effort everywhere. Cost compounds; quality lift is often illusory. Fix: measure (per B.9); default to the cheapest setting that meets the bar.

Routing decisions hardcoded in skill bodies. Changing the model means editing every skill. Fix: route at the proxy layer; let policy live there.

No fall-back path. A skill that fails opaquely on rate-limiting blocks the team. Fix: name the fall-back paths at design time.

Retry loops that hide failure. A skill that retries silently for 10 minutes is worse than one that fails in 30 seconds. Fix: cap retries, surface the failure, log to the proxy’s audit.

Effort tuning without eval. “We dialled up effort and it feels better” is the vibes-driven update B.9 forbids. Fix: A/B against the golden set.

Treating the auto-router alias as route evidence. The request says auto-router-0.1, but the executed model, reason, retry, or cache behaviour is absent. Fix: capture per-call routing receipts and block promotion when coverage is incomplete.

Expanding before evidence asks for it. The agent reads the whole repo or jumps to the strongest route before trying the bounded path. Fix: name the initial scope, verification check, and expansion trigger on the routing card.

Forgetting the cost cascade. A skill that runs 1000x/day with a 2x effort multiplier is paying 1000x more than the team thinks. Fix: B.10’s per-team rollup catches this; B.11 is what you do once you find it.

GREEN / YELLOW / RED self-check

Section titled “GREEN / YELLOW / RED self-check”
  • 🟢 GREEN: I estimate the task’s stakes and minimum evidence, execute the smallest credible route, expand only after a named verification failure, evaluate fixed and automatic route changes against the golden set with executed-route evidence, and surface fall-back failures cleanly.
  • 🟡 YELLOW — I understand the knobs but my skills default to the highest setting available without evaluation.
  • 🔴 RED — I route by model reputation or habit; I have not defined an acceptance bar or an expansion trigger.

What you can say after this module

Section titled “What you can say after this module”

“I estimate the task, execute the minimum sufficient route, verify it, and expand only when evidence asks me to — while keeping fall-backs explicit and quality measured against the golden set.”

Where to go next

Section titled “Where to go next”

You have finished Black Belt Part B. Quest B-2 (Component contribution or full-stack feature) is the practical test of cross-layer ownership. Either submit a Blade component via the contribution pipeline or ship a full-stack feature you own end-to-end.

Previous: ← B.10 Cost + observability · Next: → Quest B-2

Further reading