Over the past two months, a common refrain in the community has been that local LLMs are making a comeback. The reasons sound compelling: open-weight models like Qwen3.8-27B can now run reliably on consumer-grade GPUs, and the vast long tail of agent tasks is no longer the exclusive domain of frontier closed-source models. Looking at actual API call data, the growth momentum of open-weight models is indeed remarkably strong. On OpenRouter, weekly token volume for Chinese open-weight models reached 40.48T tokens compared to 9.66T for US models, with Chinese models claiming 4 of the top 5 spots by volume (data from August 17 to 23, 2026). The median blended API price for open-weight models is 82% lower than proprietary models at just $0.53 per million tokens, compared to $3.00 for proprietary models (BenchLM, August 21).
Yet behind these numbers lies an easily overlooked premise: OpenRouter is a remote API aggregation platform, and all calls on it run on the model providers’ cloud hardware. What these numbers demonstrate is that open-weight models have won on remote API calls, not that local deployment has won. Open-weight does not equal local—and untangling this conflation is the core distinction this article aims to clarify.
If local is the inevitable trend, why is the largest volume of traffic running entirely on someone else’s hardware? The answer is that what people casually call “local LLMs” actually bundles two completely different markets together. Let’s look at them separately: one market requires buying zero hardware, while the other buys hardware that cannot break even for 25 to 33 years. Their drivers are different, their evidence is different, and their falsification conditions are different.
To make sense of this, we can divide the inference procurement we usually discuss along two independent dimensions. The first dimension is whether model weights are accessible: if accessible, it’s open-weight; if not, it’s closed-source. The second dimension is billing structure: whether it is billed by time or billed by token. Crossing these two dimensions yields four distinct inference scenarios—four cells.
The first cell is open-weight plus time-based billing. This cell actually contains two things. One is the server running 24×7 in my home: two RTX 5090 graphics cards hosting the Qwen3.8-27B model, with weights stored entirely on local media, running a file directly from local disk whose SHA-256 hash can be verified at any time (we documented detailed benchmarking data in our August 24 2×5090 hands-on test), with ongoing costs limited to household electricity plus upfront hardware depreciation. The other consists of subscription services like Ollama Cloud and z.ai: Ollama Cloud operates on monthly subscriptions ($20/month for Pro, $100/month for Max), where the backend bills based on actual GPU time occupied rather than deducting per token. Both have access to open weights, and both charge by time rather than by token, so they fall into the same cell.
The second cell is open-weight plus token-based billing. OpenRouter is typical: you make a call, pay for the exact tokens consumed, and pay as you go. The weights are open, and billing follows tokens.
The third cell is closed-source plus time-based billing. AWS Bedrock’s provisioned throughput falls into this cell: instead of paying per token for each call, you purchase fixed capacity units called Model Units by the hour—billed even when idle—with no commitment, 1-month, and 6-month commitment tiers. Taking closed-source models as an example, AWS’s official pricing page lists a single Model Unit for Claude Instant at $39.60/hr. The weights are inaccessible, but billing runs on time, so it belongs in this cell. The fourth cell is the remaining closed-source plus token-based billing—the official APIs from OpenAI and Anthropic that everyone is most familiar with: you pay based on the number of tokens per call, the weights are inaccessible, and billing follows tokens.
Once these four cells are clarified, two conclusions emerge. First, time-based billing does not equal self-hosting. Bedrock provisioned throughput is billed by the hour, but the weights are closed-source and runtime behavior cannot be pinned; Ollama Cloud is also billed by time, but inference runs in the provider’s cloud. Time-based billing is merely a necessary condition for control, not a sufficient one—to achieve real control, you must combine open weights with hardware under your own roof. Second, the “local LLMs” everyone is enthusiastically discussing often refers only to the self-hosted corner of the first cell, but the open-weight capability axis actually spans both the first and second cells: Ollama Cloud, z.ai, and OpenRouter are all open-weight; they just use different billing models.
Open-weight and self-hosting are themselves two mutually independent dimensions. Open-weight represents the capability axis: the quality of open-source models has crossed the threshold of practical usability, yet their primary consumption channel remains remote API calls. Self-hosting represents the control axis: its core lies in opting out of the provider’s inference control layer. Where the two axes intersect is local self-hosting, but each axis can exist entirely on its own.
Once we separate these two dimensions, we can see two starkly different markets: the cost market falls on the remote side of open-weight (including Ollama Cloud, z.ai, and OpenRouter), while the control market sits in the self-hosted corner of the first cell. Let’s do the math on the cost market first. In our April 28 open-source model inference procurement guide, we calculated the actual workload for heavy agent scenarios in detail: roughly 835 million tokens per month (consisting of 750 million input tokens and 85 million output tokens, a ~9:1 input-to-output ratio). This volume was reverse-engineered from real-world tracking of Claude Code $200 Max tier power users using the ccusage tool: their monthly consumption, translated into official Sonnet 4.6 API pricing, came out to between $2,000 and $3,000 (assuming a 75% prompt cache hit rate on inputs). For that same workload, running on pure pay-as-you-go API billing would yield a monthly bill between $400 (DeepSeek V4 Pro discounted price) and $1,601 (DeepSeek official list price). However, choosing a cloud subscription for open models—such as z.ai Coding Plan Max at just $80/month or Ollama Cloud Max at just $100/month—was empirically proven to easily handle this volume. The price difference ranges from 5 to 20 times, and the user doesn’t need to buy a single piece of hardware.
This cost market is currently locked in an ongoing price war. According to BenchLM statistics, API prices for mid-tier models have fallen by 35.8% over the past 12 months. Every time cloud API unit prices drop, the argument that open-weight is cheap as a justification for self-hosted hardware grows weaker.
In contrast, real-world operational data for the control market is virtually nonexistent in public discussions. The closest hands-on sample is the 2×5090 server running permanently in my home. Powered on 24×7, this machine processes an average of about 224 million prefill tokens per day—of which 182 million hit the prefix cache—and generates around 2.25 million new decode tokens. The entire machine spends only about 3.8 cumulative hours per day actively producing tokens under heavy load. Based on the US Energy Information Administration (EIA) residential electricity rate of $0.1844 per kWh and a full-load system power draw of ~500W, the electricity cost for running a full month is approximately $11.
If we benchmark self-hosted hardware purely on cost against cloud subscriptions: compared to a $200/month subscription, a self-hosted node saves only around $20 per month. But amortizing the upfront hardware cost of $6,000 to $8,000 results in a static payback period of 25 to 33 years. As I wrote in the original article: you will never break even purely on cost savings; the rationale for self-hosting is not about saving money.
So what exactly are you buying with that money? The hands-on testing article summarized five core values: compliance and privacy with data remaining entirely on local physical media; precise pinnability of model runtime behavior (you know exactly which specific file on disk is running on your machine); dynamic hot-swapping of full fine-tunes and LoRA weights; certainty during offline disconnects; and an inference software stack under your complete control. Among these, pinnable behavior is particularly critical: on the cloud API side, what you pay for on paper is a version number, but what you actually receive is a hybrid of that version number and the provider’s server-side experiments.
In our July 29 payback analysis, we shared another finding: self-hosting has no fixed physical payback period; how fast your hardware breaks even depends entirely on which cloud billing model you are replacing. Benchmarked against a cold-cache API baseline, the payback period for running GLM-5.2 on an 8×H200 node is 1.15 years; benchmarked against a typical agent workload with a 92% cache hit rate, the payback period shrinks to 2.3 to 2.9 months; but benchmarked against an equivalent capacity of cloud subscription seats (29 seats), the payback period stretches out to 21.8 to 27.3 years. Three baselines for the same eight-GPU node produce payback periods differing by two orders of magnitude—the governing variable is the cloud billing model you are replacing.
Therefore, the four cells are complementary rather than mutually exclusive. Even the 2×5090 server in my home calls GLM 5.2 as an upstream model for complex tasks—routed through Ollama Cloud hosting, which is remote, not local. The actual workflow of self-hosting users is also a hybrid stack; the cells divide and coordinate labor rather than forcing a binary choice.
If you are doing the math on self-hosting, you must first clarify which billing model you are replacing. Replacing subscriptions yields a calculated payback period of decades; replacing high-cache-ratio agent API calls often takes just a few months.
Why did the four-cell framework only become relevant for serious discussion between the second half of 2025 and 2026? Because before then, the four cells had effectively collapsed into a single choice.
In 2024, agentic models were unreliable in complex long-context and multi-step tasks. During that stage, the open-weight side could not support the long tail of agent tasks; whether self-hosted or called remotely, model quality simply did not pass the reliability bar. For most actual builders, the near-sole option was closed-source models via token-billed official APIs—the fourth cell.
Over the past 12 to 18 months, however, three main threads in the technical ecosystem crossed critical tipping points simultaneously. First, the model thread: 27 to 30B agentic open-weight models successfully entered the 24 to 32GB VRAM footprint of consumer hardware—Muse Glimmer 30B under the Apache 2.0 license requires only 24GB VRAM to run, as Meta explicitly positioned, while Qwen3.8-27B benchmarked on our 2×5090 machine maintained a per-stream throughput of 141 tok/s even under 8-way concurrent load. Second, the hardware thread: the RTX 5090, with its 1792 GB/s bandwidth, 32GB VRAM, and ~$2000 MSRP, became the sweet spot for running 27B models; Apple’s M5 Ultra offers 512GB of unified memory and 1.2TB/s bandwidth, explicitly positioned by Apple as always-on agentic computing. Third, the workload thread: agents moved decisively from the demo stage into production environments, with Anthropic’s annualized revenue growing rapidly from ~$9 billion in late 2025 to over $65 billion by July 2026.
Once all three threads crossed these thresholds, the technological moats at the model layer began to dissolve. When the quality of open-weight models crossed the threshold of practical agent reliability, whose model you used was no longer a compelling reason for users to stay locked into any particular cloud provider. To build defensible moats, providers were forced to shift product differentiation to the inference layer—manifesting primarily in billing model design, rate limit allocation, runtime behavior governance, and user retention strategies. Understanding this is essential to interpreting what followed: the frequent limit adjustments, A/B testing, and output watermarking that emerged across major platforms in July and August 2026 were all, at their core, instruments of control exerted by providers at the inference layer.
During the concentrated window of July and August 2026, four representative events unfolded in rapid succession, all centered on the inference layer. The first event was the inability to precisely pin model behavior in the cloud. On August 22, developer argofowl discovered from API logs that starting with version 2.1.237, Claude Code silently remapped the user-configured “high” setting to “low” values, while the official changelog made no mention of it (community discussion thread). The underlying mechanism was a server-side A/B rollout experiment by the provider, which Anthropic engineer Thariq Shihipar subsequently confirmed in public discussions. The controversy sparked by this incident was not about performance tuning, but about the determinism of programmatic contracts: users paid and explicitly selected “high” on the client, but actually received parameters assigned to an experimental cohort. On the cloud API side, what you pay for on the surface is a version number, but what you actually get is often a hybrid of that version number and provider server-side experiments. On a local self-hosted node, this layer of risk simply does not exist: what you pin is a specific file on local disk, with a SHA-256 hash verifiable at any time.
The second event was instability in access allowances. Claude Code’s +50% weekly usage boost underwent at least four extensions since May (pushed back from July 18 to August 19, and then to August 31), each announced right on the eve of expiration (extension timeline). The official @ClaudeDevs account stated verbatim: “We hope to make this a permanent change to our plans, but strong demand for our models means that capacity may be tight over the coming weeks.” If baseline access were stable and predictable, a +50% boost would not be a promotional measure requiring repeated extensions. The trajectory of such instability is dictated entirely by the provider’s own capacity pressures, making it impossible to align with users’ actual planning needs.
The third event was the policing of output content. On August 2, Claude officially enabled SynthID-Text-based text watermarking (official announcement), covering the official API, Claude Code, and all cloud marketplace distribution channels. Yet just 4 hours after the official announcement, the open-source community released override bypass tools targeting the watermark (Wired report), and the GitHub project attracted over 100 contributors in short order. This control was neither necessary nor sustainable long-term, but it revealed a clear direction: providers are attempting to assert control over the provenance of output content.
The fourth event was the self-evident proof of incentive misalignment in the providers’ own product architectures. Anthropic introduced a public beta of Claude Code self-hosted environments for enterprise users (report), targeting Team and Enterprise tiers: working sessions can run on customers’ own infrastructure, but official guidelines explicitly mandate that “Model inference stays with Anthropic,” prohibiting requests from routing to AWS Bedrock, Google Agent Platform, Microsoft Foundry, or any third-party LLM gateway. If this misalignment could be resolved through standard product features, providers would have opened it up long ago. They did not, and commercially they cannot concede: the task harness runs on your network to provide a sense of control, while core model inference must stay on their side because inference is the recurring revenue-generating engine they cannot surrender. Anthropic effectively redefined self-hosting as your harness plus their inference.
Of course, while observing these tightening moves, we must also objectively draw honest boundaries around reality. Within the exact same timeframe, providers were also making massive price concessions (pricing comparison): on August 11, Anthropic made Sonnet 5’s price cut permanent and canceled its planned September price hike outright; on July 30, OpenAI slashed GPT-5.6 Luna by 80%, dropping it to $0.20 per million input tokens and $1.20 per million output tokens. Therefore, we cannot claim that the misalignment is widening. The only claim we can substantiate today is that the misalignment objectively exists, but its direction remains undecided. This is a dynamic multi-party race, not a settled fait accompli.
If you rely heavily in production workloads on the specific behavioral characteristics of cloud APIs, the recommendation is to lock model versions and monitor behavioral fingerprints. Because the price of relying on the cloud is that one day you won’t know which version you are actually getting.
Returning to the practical question of selection: as developers diverge from the traditional closed-source × token-billed cell, they naturally flow in different directions based on their own sensitivities.
Cost-sensitive users head toward the remote side of open-weight: choosing subscriptions (Ollama Cloud, z.ai) at $80 to $100 per month—5 to 20 times cheaper than pure token-billed APIs—or token-billed OpenRouter to pay for actual usage. Control-sensitive users move to the self-hosted corner of the first cell: open-weight plus self-hosting, taking on $11/month in electricity, accepting a 25 to 33 year hardware payback period, and buying deterministic control. Their drivers are different, their evidence is different, and their falsification conditions are different.
In my friend’s class of 100 people, only 4 could run Qwen3.8-27B locally. People often use figures like this to argue that professional developers cannot afford hardware, but this is a completely circular argument. The sample consisted entirely of non-technical students who would never buy GPUs for work in the first place—much like polling people who don’t eat rice to prove rice is too expensive, where the outcome is entirely an artifact of sample selection. The true value of this number is threshold benchmarking: the entry point for running a 27B model locally has dropped to $1,300 to $1,800 for a used complete system, or $6,000 to $8,000 for a dual-5090 setup like the one I use. The control market does not need to cover all 100 people on day one; much like Kubernetes was a toy for a few geeks in 2015, infrastructure always becomes the default choice among high-incentive cohorts first (this is an analogy, not data).
Current 27 to 30B open-weight models are more than capable of handling the human-in-the-loop work layer—such as pair programming, which covers roughly 80% of daily agent workloads—but for fully unattended frontier complex tasks, the pinnacle capabilities remain with cloud APIs. The root of this divide lies in the fact that smaller models’ primary limitation is knowledge structure. Methodological capabilities like reasoning and tool use can be compressed into fewer parameters through architecture and distillation, but long-tail obscure knowledge behaves more like a storage problem, remaining heavily dependent on parameter scale. Bojie Li’s IKP paper used 1400 obscure factual probes to find that factual capacity scales log-linearly with parameter count; smaller models catching up on reasoning benchmarks cannot be extrapolated to long-tail facts, as we unpacked thoroughly in our April 29 knowledge capacity paper guide. With human supervision, a human can catch and bridge knowledge gaps at any moment, but in unattended scenarios the parameter pool deficit is laid bare. Consequently, the claim scope of the control market is the long tail of agent tasks, not frontier reasoning.
The real open question is a race: the speed at which providers tighten the inference layer versus the speed at which the cloud resolves the control dimension. If providers continue to tighten the inference layer, the control market will expand further; conversely, if remote providers effectively solve the control dimension (for example, by offering contract-level behavior pinning, guaranteeing zero background A/B testing, and providing auditable zero data retention), the unique value of self-hosting will diminish accordingly.
Today, the inference layer has become a tier that developers can procure independently. The selection question is no longer a generic local vs. cloud, but rather which cell you buy in and why. Your workload utilization, data sensitivity, and requirements for behavioral certainty determine which cell you ultimately land in. If you cannot answer these three questions right now, start by testing your actual usage: run your existing setup steadily for 7 to 14 days, examine your telemetry metrics clearly, and only then decide which cell to buy in.