You shipped a skill. It worked. You closed the tab.
That’s the whole problem. Model choice is a decision you make once, at the moment you’re least equipped to make it: before the skill is even authored. Then you never revisit it, because the skill stopped being interesting the day you got it working.
So go back and check. Here’s how.
TL;DR: how do you right-size a skill you already shipped?
Point model-right-sizer at the skill file instead of at a new idea. It reads the stages you already wrote, scores each one, and hands back a routing table that marks the rows where your current model is bigger than the work needs. You hand those rows to Claude. That’s it. Five steps, read-only until you ask Claude to apply it, no rebuild.
- Install the plugin.
- Run the dry-run against what you built, not what you’re about to build.
- Review the whole blueprint, row by row.
- Ask Claude to change the model pin.
- Ask Claude to gate the agentic ones, then re-run for real.
Takeaway: Aim the rightsizer at a skill that’s already in production. It scores the stages you shipped.
Report
Finance needs to prove AI’s return: CloudZero report
260 senior finance leaders (more than half CFOs) told us why the speed of seeing AI spend, not the size of it, separates who pulls ahead on AI from who gets burned.
Why would a skill that already works be overpaying?
Because you never priced it. You picked a model when the skill was an idea, and an idea has no measured stages, so you picked big. Opus on everything, on the theory that you’d tune it once it ran. That’s the rational move when you don’t yet know which parts are hard, and it stays in the file forever because the file works.
Now the skill has run two hundred times. Some of those stages turned out to be a lookup; one is a string extraction. They are all still running on whatever you guessed in the first ten minutes.
We made this same argument when we open-sourced Model Rightsizer: a default is not a decision. That post was written for work that hadn’t started yet, which left the harder case alone: the skill that’s been running for months on a pin nobody remembers choosing.
What happened when we ran this on someone else’s repo
I sat down with Larry Advey, CloudZero’s AI Economics Director — his Good Morning skill has been posting a live FinOps briefing every morning for months — for a walkthrough. The plan was simple: don’t demo the tool on my repo, where I already know where the fat is. Point it at his.
Two adoption modes came out of the walkthrough, and the distinction matters more than I expected going in:
Manual, retroactive. You point the rightsizer at a skill that already exists and ask what it would have routed differently. Read-only. No edits, no rebuild, no commitment. This is the mode this post is about, and it’s the one nobody was using.
Auto, standing mandate. Run model-right-sizer-install in a repo — it stamps a marker-delimited block into CLAUDE.md telling every future session to consult the rightsizer before and after substantive work. Also real, also demoed, and out of scope here.
We ran both live during the walkthrough, and I went in expecting the standing mandate to be the interesting half.
Where do the cheap wins actually hide?
Inside sub-agent dispatch, on stages that never got a model decision of their own.
The rightsizer pulls three levers, and only one of them is the model name. This is the short version — how the routing actually works has the full mechanics:
- Model tier. Scored on effectiveness need against efficiency pressure: how badly the outcome degrades on a weaker model, versus how much cost, volume, and latency actually bite.
- Difficulty-adaptive effort and an explicit token budget. How hard the chosen model reasons, tuned stage by stage. Over-thinking an easy stage costs real money; the IBPO and BudgetThinker results are what ground that lever.
- The agent-to-agent message schema. What each stage hands the next one. A cheap model reading a bloated handoff can cost more than an expensive model reading a tight one, so every seam gets named fields and an explicit list of what not to pass.
Here’s the mechanical reason a retroactive pass is the only thing that finds these. In standing-mandate mode, the top-level agent running your session will not downgrade itself to Haiku, and it shouldn’t: switching the orchestrator’s model mid-session invalidates its prompt cache, which is a real cost, not a preference. So the rightsizer’s actual lever is per-sub-agent overrides — the model each dispatched build unit runs on.
An overprovisioned sub-agent still returns the right answer, so the skill’s behavior gives you no signal at all. You have to go look.
The second takeaway: The cheap wins sit in sub-agent and build-unit dispatch, one level below anything you’d notice from the outside.
What did the retroactive pass actually find?
We pointed it at Larry’s Good Morning skill. It flagged a retrieval step as overprovisioned and recommended dropping it to Haiku.
That’s one row out of a table with one row per stage, and it’s the row worth walking through. I’m going to leave it exactly that size. What came out of the session is a recommendation, not a measured saving. The rightsizer is read-only by design, so nothing was applied and nothing was re-run. Quoting a percentage here would be inventing one. The honest version is better anyway: a stage nobody had ever looked at individually turned out not to need the tier it was running on, and finding that took one read-only pass over a file that already worked.
Retrieval is a predictable place for a flag like that to land. Nobody sits down and picks a model tier for a lookup, so a stage like that inherits whatever the rest of the skill was pinned to and keeps it for as long as the skill runs.
Back to that first takeaway, now with a price on it: the pass that found this read one file that was already in production. If every row had come back keep, we’d have spent ten minutes learning the skill was fine. That’s the entire downside of looking.
How to run a retroactive pass on your own skill
Five steps. You need a skill file and about ten minutes.
Step 1: Install the plugin
/plugin marketplace add cloudzero/cloudzero-claude-marketplace
/plugin install model-right-sizer@cloudzero
That gives you the model-right-sizer agent plus three skills: model-right-sizer-dryrun (preview a routing map, build nothing), model-right-sizer-install (stamp the standing mandate), and model-right-sizer-audit (a one-shot version of the walkthrough below — it finds every real call site in a target repo on its own and opens a PR with the blueprint). For the manual pass this post walks through, you only need the first one; reach for model-right-sizer-audit once you’d rather sweep a whole repo in one pass than read one skill file at a time. The agent is Apache-2.0 and the repo is public.
Step 2: Run the dry-run against what you built, not what you’re about to build
The dry-run skill takes a free-text intent. It was written for something you’re planning:
/model-right-sizer-dryrun build a slack bot that summarizes standup threads
For a retroactive pass you hand it the file instead of the idea. Same skill, different input:
/model-right-sizer-dryrun audit the existing skill at .claude/skills/finops-briefing/SKILL.md — score its stages as they're written today, don't design a new one
Plain English works identically if the slash command is namespaced differently in your install. “Dry-run the right-sizer on this skill file” is a documented trigger phrase. Either way you get Pass A only. The blueprint, then a stop. Nothing gets built, nothing gets edited, no closing report, because there’s no “actual” to reconcile against yet.
One thing to state explicitly in your prompt: score these stages as written. Otherwise you’ll get a redesign, which is a different and much less useful answer.
Step 3: Review the whole blueprint, row by row
You get back one row per stage — the whole skill, scored. Review all of it, not just the rows with a flag on them. This is probably the first stage-level opinion anyone has written down about this skill, and the clean rows are part of that opinion.
Each row carries three scores with a one-clause reason each: effectiveness need, efficiency pressure, difficulty. After that, a primary pick with a confidence percentage, a runner-up with its own percentage, a token budget, and a handoff schema. The field that orients you fastest is keep_or_override.
keep means the agent thinks your current pin was right. Read the reason anyway, then move on. A keep you’ve read is worth something; a keep you scrolled past isn’t.
override means the stage is running on a bigger model than the work needs, and the row tells you what to move it to. Pass A now emits this as a strict JSON contract — shipped in PR #35 — so the shape below is exactly what the agent prints today, not a preview of where it’s headed. Here’s an overprovisioned row, sketched — the labels below are mine for readability, not a field-for-field transcript of what the agent prints:
stage: resolve briefing recipients
current pin: claude-opus-4-8
effectiveness: 20 — a wrong lookup misroutes one message, cheap to catch
efficiency: 85 — runs every fire, pure key-value lookup, no reasoning
difficulty: 10 — single indexed lookup, no ambiguity
primary pick: deterministic query layer (confidence 90%)
runner-up: claude-haiku-4-5 (confidence 10%)
keep_or_override: override
what flips it: if recipient routing ever has to interpret free-text
preferences instead of reading a config row
Read it in this order:
keep_or_override. Is there anything to do at all?- The pick itself. The agent flags any data-query-shaped stage; when the pick that comes back is a deterministic query rather than a tier, stop shopping for a cheaper model. The stage doesn’t need one at all. Route it through a real query. Deterministic execution can’t hallucinate the join it just computed. This is the biggest single win available and it’s the one people skim past looking for a model name.
- The three scores. Do you actually believe them? You know this skill. If effectiveness reads 20 and you know a wrong answer here pages someone at 3am, the score is wrong and you should say so rather than take the pick.
what flips it. The condition that would invalidate the recommendation. This is your regression test. Write it down.
If a row comes back with no runner-up and no flip condition, the weighing didn’t happen, and the pick isn’t worth acting on until someone re-runs it.
Step 4: Ask Claude to change the model pin
“Drop to Haiku” is a one-line edit, not a project. It’s also not your line to type. Claude Code is already open in this repo, it can already see the skill file, and you’re holding a row that says exactly what to change — so hand it the row and tell it which shape the stage is.
If the stage runs as a sub-agent with its own file, it’s a frontmatter change. Here’s what should come back, so you’re reading the diff instead of writing it:
---
name: briefing-retriever
description: Pulls the day's cost rows for the morning briefing.
-model: opus
+model: haiku
---
If the stage is dispatched inline from the skill, tell Claude to pin it at the dispatch site instead, and to pass the budget the blueprint gave you along with it. The budget is what makes the stage self-truncate; leave it off and the stage runs to whatever length the model lands on:
Agent(briefing-retriever, model=haiku, budget=8k)
Two things Haiku will not do, both of which the blueprint should have already warned you about. Check these before you ask, because Claude will make the edit correctly whether or not the pin was a good idea:
- No effort dial. Setting effort on Haiku errors. A Haiku stage that turns out harder than expected has exactly one move: escalate the model. It cannot think longer.
- Smaller context. If your retrieval step hands a large payload downstream, check it fits before you commit. This is a common
what flips iton exactly the retrieval-shaped stages that get flagged.
Change one row at a time — one row per request, one row per commit. Move two pins together, and if the skill gets slower you’ll spend the next hour figuring out which one did it.
Step 5: Ask Claude to gate the agentic ones, then re-run for real
People skip this step. It’s the one that keeps a right-sizing pass from becoming a regression.
A down-pin is safe on a single-shot or low-tool-turn stage. It’s genuinely risky on an agentic one: three or more tool turns, unattended. A smaller model on a long loop tends to take more turns to converge, and each turn pays a full round trip. The wall-clock you add can erase the per-token saving you just banked, or invert it.
So for any agentic row, don’t ask for the edit — ask for the gate. Tell Claude the down-pin is measurement-required: keep the existing pin live, run a real sample, compare wall-clock against the baseline. Hand it the thresholds too. At or under baseline × 1.15, promote the down-pin. Above baseline × 1.25, revert. Anything in between, hold the pin where it is and sample again. Claude runs the gate; the thresholds are yours, so tune both numbers to your own latency tolerance before you hand them over.
Once a row clears the gate, run the skill for real and ask the rightsizer for Pass B: recommended versus actual per stage, budget adherence, schema adherence, cost. That’s the number you’re allowed to quote, because it happened.
One more thing while you’re already in the repo. Run model-right-sizer-install, so the next stage someone adds gets scored on the way in instead of waiting six months for another retroactive pass.
And back to the second one, now that you’ve seen the steps: every row worth changing here lived in a dispatched stage. So tell Claude to start your pass in the sub-agent frontmatter and at the dispatch sites. That’s a much shorter list than the whole skill file, which is why this is a ten-minute job and not an afternoon.
How do you know the recommendation holds up?
You gate it, then you measure it. That’s what step 5 is for.
The rightsizer is advisory by design. It scores and recommends. You apply. If a wrong answer on a given stage costs you a rework cycle, the cheap pick is the expensive one, and the scoring says so. Sizing up is also right-sizing.
Larry’s next step out of the walkthrough was to run it across more repos and report back. That’s the next step I’d give you too.
So pick one skill you already shipped. Run the dry-run on it today, read the flagged rows, and see what savings are sitting in a file you stopped thinking about months ago.
Get it: cloudzero-claude-marketplace · docs.cloudzero.com/docs/ai-model-right-sizer · Apache-2.0, free, no CloudZero account required.
Background reading:
Why a default is not a decision
How the routing actually works
