What AI sees in your LLM bill that your finance team misses

A founder running a mid-size support-automation product pulled up her May invoice expecting the usual number. It had jumped 40% with no change in traffic. The cause: GPT-5.5's API price doubled to $5 per million input tokens and $30 per million output tokens that same month, and her routing config sent every ticket — from "reset my password" to a genuinely gnarly multi-step debugging thread — through the same frontier model. Nobody had touched that config since 2025, back when the same workload cost half as much and nobody had a reason to look.

That's the story playing out across AI-product teams right now. LLM pricing spent 2025 falling roughly 80% year over year as providers competed on rate cards. Then 2026 reversed it: frontier prices are creeping back up, GitHub Copilot moved to token-based billing on June 1, and the assumption that AI infrastructure behaves like a stable-rate API is gone. If your unit-economics model was built on last year's price sheet, it's already wrong — and the finance team reviewing your invoice line-by-line has no way to know it, because the waste isn't a billing error. It's a pattern.

The 2026 price shock: what changed and why your 2025 margin math is wrong

Most AI SaaS unit economics models assume a roughly flat per-token cost, adjusted maybe once a year. That assumption broke this year. OpenAI's GPT-5.5 jumped to $5/$30 per million tokens and, layered on top of Copilot's shift to metered billing, two of the most common cost lines in an AI product's stack moved in the same direction at once — which is exactly why cost optimization stopped being a finance afterthought and became a product concern. Reporting on the "hidden economics of AI SaaS" in 2026 puts realistic gross margins somewhere around 60–70% for products running complex operations on expensive models — well below the 80%+ margins software investors expect from a SaaS line item.

Finance teams read this as "the vendor raised prices, nothing we can do." That's the wrong read. The invoice is a single number; the spend behind it is a distribution of thousands of individual calls, most of which didn't need the model they got.

Three token waste patterns AI identifies in a real spend log

Point a classifier at a token log instead of a P&L line and three patterns show up almost immediately, every time.

Oversized models on simple queries. A frontier model answering a sub-100-token question — "what's my current plan," "reset my key" — costs the same per token whether the question is trivial or genuinely hard. A mini or nano-tier model handles the trivial case at a fraction of the cost with no measurable quality loss, because the task doesn't require frontier reasoning in the first place.

Repeated prompts with no cache hit. The same system prompt, the same few-shot examples, the same boilerplate instructions get re-sent and re-billed on every single call, even when nothing in them changed.

Context bloat. System prompts grow every time a new feature ships and nobody trims the old instructions. A prompt that started at 400 tokens ends up at 4,000 a year later, and every one of those tokens gets billed on every call, whether the feature it supports fires once a day or never.

None of these show up on an invoice. They show up in a query-by-query log, which is exactly what a finance team doesn't normally read — and exactly what a cost-routing layer reads by default.

How model routing works (and what it routes on)

A model-routing layer sits at the gateway, in front of your model calls, and does one job: read each incoming query, estimate its complexity, and route it to the cheapest model capable of handling it correctly. The classifier looks at things like query length, whether it matches a known simple-task pattern, and whether the task requires multi-step reasoning or tool use.

Reported outcomes vary by how aggressively teams tune the routing rules. Conservative distributions — routing only the clearest simple-task queries down a tier — cut blended token cost 35–50% on typical SaaS workloads. Teams willing to route more aggressively, backed by evaluation data showing quality holds, report cuts as steep as 70–90%. The gap between those numbers is confidence, not magic: it's how much of your traffic you've actually verified doesn't need the expensive model.

This is the job description of an AI gateway — tools like LiteLLM, OpenRouter, and Portkey exist specifically to sit in this position. Past roughly 10 million tokens a month, the math on standing up a routing layer stops being a nice-to-have and starts paying for itself within weeks, because the savings compound with every call.

Caching: the cheapest cost cut you are probably not making

If routing is the sophisticated fix, caching is the boring one that most teams still haven't shipped. A prompt-cache hit costs roughly 10% of a full inference call, because the model doesn't have to re-process context it's already seen.

Take a support bot answering the same 200 or so recurring questions. A meaningful share of its system prompt — the persona instructions, the product FAQ, the tool definitions — renders byte-for-byte identical on every single call, and a reasonable estimate for that overlap runs around 40% of the prompt. One caching layer recovers that entire 40% at a tenth of the cost. No model change, no quality risk, no A/B test required. It's the cost cut that sits there unclaimed because nobody assigned it to anyone.

What 40% savings actually looks like in your P&L — and where SideKyk fits

The honest counter-argument here is "I need frontier quality for everything my product does." Test that claim instead of assuming it: pull your last 1,000 production queries, route them through a cheaper model, and score the outputs against what the frontier model actually produced. Most operators who run this test find that 60% or more of their traffic is served identically well by a model two tiers down — the frontier model was never doing anything the cheaper one couldn't.

Run the math at scale and a 40% reduction in blended token cost on a product spending $30,000 a month on inference is $12,000 a month back into gross margin, every month, without a single feature change. That's not a rounding error on a P&L — that's the difference between a 65% margin AI product and a 75% one, which is the difference investors actually price.

The reason this waste survives so long is structural: your token spend usually lives in a billing dashboard, and your business outcomes live in a completely different one, and nobody's job is to hold both open at the same time. This is the gap SideKyk's Ops & PM specialist is built to close — it tracks model cost per conversation alongside the business outcome that conversation produced, in the same place, so you catch the oversized-model call or the missing cache hit the week it happens instead of the quarter your investor asks about margin. Sign up at sidekyk.ai/ai-business and it starts reading your spend pattern from day one.