When the AI bill starts climbing, the easiest mistake to make is rushing straight to cheaper models.
This closely resembles the early days of cloud cost governance. When the bill went up, everyone first looked at which machine type was cheaper and which cloud provider offered bigger discounts. Only later did people realize that the real money-burners were usually not expensive individual machines, but instances nobody shut down, overly redundant clusters, unclaimed test environments, and services spun up on a whim.
AI token bills are no different. Model prices are indeed falling. Epoch AI’s trends page shows that at a fixed capability level, LLM inference prices have dropped roughly 40× per year, halving approximately every two months. OpenAI GPT-4o mini is already priced as low as $0.15 per million input tokens and $0.60 per million output tokens.
But falling unit prices don’t automatically save money. When tokens become cheap, teams wire more processes into models. Long documents they used to hesitate feeding in now get dumped wholesale. Workflows that used to need a single human click now have agents retrying repeatedly. AI that was once reserved for critical scenarios now gets plugged into every internal tool.
So when the AI bill goes up, step one is not swapping models. Step one is understanding what those tokens are actually buying.
Before cutting AI costs, split them into two categories.
One is internal efficiency — products like Copilot, Cursor, Claude Code, internal knowledge assistants, and office automation. This money buys employee productivity. It typically lands in R&D expenses, IT costs, employee tool budgets, or internal platform costs.
The other is customer delivery — products like customer service AI, AI search, Duolingo Max’s video calls, Adobe Firefly generation, and Intercom Fin’s customer service agent. This money buys product features or service outcomes. It flows into cost of revenue, support cost, or product gross margin.
These two types of costs cannot be cut with the same knife. The FinOps Foundation’s breakdown of AI procurement models illustrates this: direct model APIs charge per token, embedded SaaS AI often charges per seat or add-on, and self-hosted models shift costs into GPU hours, storage, networking, and platform teams.
They all fall under “AI cost,” but the control surfaces are entirely different.
China Merchants Bank offers a useful calibration for our intuition. According to Wallstreetcn’s coverage of CMB’s shareholder meeting remarks, by the end of May 2026, CMB’s daily token consumption reached 33 billion, with an LLM cost-benefit ratio of approximately 20%. The same report contains an even more telling figure: compute investment in LLM-assisted programming accounted for only about 5% of total compute.
This is not a regulatory filing — we can only treat it as media reporting of management remarks. But it serves as a reminder: AI coding, which commands the loudest voice in tech circles, is not necessarily the main battlefield for enterprise token consumption. In an organization like a bank, customer service, operations, risk control, office automation, and knowledge retrieval likely consume far more tokens.
If you only focus on saving money in R&D tools, you risk sweating the pennies while missing the pounds.
When AI costs are buying internal efficiency, the first cut should generally not go to model prices — it should go to idle resources and runaway processes.
For seat-based tools, start with activity rates. The company provisioned AI assistants for all R&D, operations, and sales staff, but how many actually use them daily? Which teams received licenses simply because the budget was loose? Which people haven’t triggered a single meaningful call in two weeks? This kind of waste requires no complex engineering optimization — simply revoking entitlements saves money.
For API-based and agent-based tools, look next at self-loops. The most expensive thing about internal agents isn’t a single model call being pricey — it’s the agent hitting an error and then repeatedly re-reading context, re-calling tools, and retrying. A task that should have failed and stopped will, without max steps, max retries, and max context, keep bleeding tokens in the background like an open tap.
That’s why internal-efficiency AI needs at least three hard constraints.
First, cap the maximum steps and maximum retries per agent task. Second, limit context length — don’t let employees stuff entire irrelevant documents in. Third, attribute costs by team, application, and workflow so the teams actually consuming tokens can see their bill.
China Merchants Bank’s management approach is instructive. It breaks down LLM costs into R&D personnel investment and token expenses on the cost side, then measures business impact across multiple dimensions on the returns side (Wallstreetcn). This method gets closer to real governance than “how many tokens did the entire company use.” Because only when costs are attributed to teams and business chains will internal restraint emerge.
The goal of internal-efficiency AI isn’t to drive token usage to the absolute minimum — it’s to ensure every call exchanges tokens for employee time, engineering quality, or process speed. Idle seats, unlimited context, agent self-loops, and untraceable platform costs are the first things to clean up.
When AI costs are buying customer delivery, the cost-cutting logic changes.
Here the most dangerous move is swapping to a smaller model and declaring you’ve saved on token costs. If a cheaper model gives inaccurate answers, customers will repeat their questions. A customer service issue that should close in one turn can balloon to five. Worse, the problem eventually escalates to a human anyway. The model money you saved may well get eaten up by human escalations, customer wait time, and duplicate tickets.
Customer-delivery AI shouldn’t be measured by cost per token — it should be measured by cost per outcome.
Duolingo is a rare clear example among public companies. In its 2025 10-K, it noted that overall gross margin declined from 72.8% to 72.2%, partly due to lower subscription gross margin, which in turn reflected increased AI costs from features like Video Call. Reuters also cited the CFO stating that Max-tier video calls carry marginal costs and contributed roughly 100 basis points of impact to overall gross margin.
This is the essence of customer-delivery AI: the more users love it, the more revenue might grow — but costs grow too. The real question is whether each video call, each generation, each search answer, and each customer service resolution covers its cost.
Intercom Fin’s pricing productizes this logic. It charges per outcome / resolution (Fin pricing). This isn’t a clever rewording of pricing copy — it’s about aligning the revenue unit with the cost unit. Behind each resolution there may be multiple conversation turns, knowledge base retrievals, model calls, and human handoffs. The pricing must cover these variable costs.
Klarna’s customer service AI also illustrates what to look at. Within its first month, its AI assistant handled 2.3 million conversations, two-thirds of all customer service chats, equivalent to 700 full-time agents, and was projected to deliver $40 million in profit improvement in 2024 (Klarna). The core metrics in this case aren’t total token volume — they are resolution rate, repeat inquiry rate, handling time, and human escalation rate.
Cost-cutting for customer-delivery AI should center on gross margin. For customer service, watch cost per resolved issue, containment, repeat inquiry, and handoff. For generative products, watch credits, quotas, and whether heavy users are turning a fixed subscription into negative margin. For AI search, check whether inference costs for free users and paid users are accounted separately. For agent platforms, check whether every action, tool call, and retry is under control.
Cheaper models are tools, not goals. The goal is stable margin per outcome.
Many companies can now see total token consumption but have no visibility into where those tokens go.
This is like looking only at a building’s master electricity meter. You know the machines are running, but you don’t know which production line is profitable, which is idling, and which piece of equipment was left on.
The numbers in China are already enormous. STAR Market Daily / Sina report that Doubao LLM’s daily token call volume has surpassed 180 trillion, Volcengine has reached a 49.5% share of China’s public cloud MaaS market, and the Trillion Token Club has over 200 members (Sina / STAR Market Daily).
These figures show that adoption scale is real. But the same report also quotes skepticism from a cloud industry analyst: among the Trillion Token Club, many may be free trial quotas vendors give to customers or pilot project traffic — how much is actual paid production usage remains an open question.
That skepticism applies just as much to enterprise cost-cutting.
If you lump together production calls, pilot calls, free quotas, internal testing, and benchmarks into one number, all you get is a bigger number. It won’t tell you where to cut, and it won’t tell you where to invest more.
A monthly AI cost review should break down into at least four views.
The first view splits by team, application, and workflow — to see who is spending. The second splits by model, context length, output length, and retry count — to see where the technical waste is. The third splits by business outcome — to see cost per outcome for customer service, generation, search, and agent tasks. The fourth splits by production, pilot, free quota, and internal testing — to see which tokens are actually running in production systems.
Without these four views, so-called AI cost reduction will mostly become guesswork: cut whichever team complains least, swap whichever model looks most expensive, pause whichever project can’t articulate short-term ROI.
That’s an easy way to cut the wrong things.
When a company begins cleaning up its AI bill, it can follow a simple sequence.
First, label what each AI cost is buying. Is it buying internal efficiency, or customer delivery? If this question goes unanswered, every subsequent action could be wrong.
If it’s internal efficiency, first cut idle seats, then constrain agent loops, retry counts, and context length, and finally implement team-level chargeback. Don’t start with model price negotiation. A huge portion of the waste may have nothing to do with model unit prices — it’s in things nobody uses, nobody manages, and nobody is accountable for.
If it’s customer delivery, first calculate cost per outcome. How much does resolving one customer service ticket cost? Can a subscription or credit cover the cost of one generation? Can the cost of one search answer align with paid-user revenue? Have human escalation rates, repeat inquiry rates, and failed retry rates gone up? Until these metrics are clear, blindly swapping to cheaper models risks destroying your gross margin.
Only then talk about model price negotiation, model substitution, small-model routing, and batch optimization. These levers are certainly useful — but they belong after attribution.
An AI bill becomes truly manageable not because you’ve bought the cheapest tokens, but because you know exactly which business outcome each category of tokens is supposed to pay for.