When we search for restaurants on our phones or look up travel tips on our computers, we increasingly skip clicking those blue web links. We are more accustomed to letting the AI assistant on the other side of the screen gather information for us and deliver curated results directly to us. In this process, the banner ads that once captured our attention have completely lost their chance to catch the human eye.
It’s clear that the traditional advertising-based business model is breaking down. AI agents are now reading web pages on behalf of humans—they have become the most active cohort on the internet. But in the face of this shift, has anyone started taking action? As website creators, how do we survive in this new environment?
Network infrastructure provider Cloudflare has taken several consecutive steps forward. They have embedded machine-readable rules into the new website publishing flow, attempting to reconstruct the entire edge network. They are transforming the edge network from a distribution layer for human web pages into a negotiation layer for agent requests. Their latest tool, Cloudflare Drop, directly integrates the Markdown for Agents feature into the standard new-website publishing flow. From now on, the default intended reader when a site goes live is no longer just the human browser—AI agents have become a parallel first-class reader. Combined with the x402 billing layer and the Access identity layer, Cloudflare’s edge network now instantly answers four questions at once: who is visiting, can they access the content, should they pay, and what format should be returned. A close analysis of these moves offers valuable insights for today’s practitioners.
In mid-2026, the structure of global web traffic experienced a major shift. Statistics show that bot traffic now accounts for 57.5% of all internet traffic. This figure arrived a full 18 months ahead of all analysts’ projections. Today, thousands of websites face an entirely new reality. The primary visitors to their pages are no longer humans staring at screens—various tireless AI agents have become the overwhelming majority.
However, when agents attempt to read and understand a web page, the process is extremely cumbersome. First, they must issue an ordinary HTTP request to fetch bulky HTML code. Next, they need to clean up the complex DOM tree. They must strip away the top navigation bar, remove the sidebar ads, and delete the footer information. They also need to identify and filter out third-party scripts and analytics code. Only after cleaning up this useless noise can the agent extract the clean body text. Finally, they must feed the body text into the large model’s context window.
This parsing process not only wastes time but also imposes a heavy financial burden on developers. Take a typical Wikipedia page as an example. If the raw HTML is fed directly to a large model, it consumes 68,000 tokens. The large model itself is intelligent enough—the root cause of this overhead lies in the format design. Today’s web page formats are inherently designed to serve browsers. They were not created with machine reading habits in mind.
Faced with this increasingly severe pain point, network service providers have begun transforming the edge network. They want to turn the edge network from a web page distribution layer into an agent request negotiation layer. On February 12, 2026, Cloudflare released a new feature called Markdown for Agents. This feature is built on HTTP protocol and implements flexible content negotiation mechanisms. Technologists can refer to the Markdown for Agents official documentation and the Markdown for Agents blog post to deploy it.
When an AI agent attempts to read a certain page, it sends an
ordinary HTTP request. The request header includes an
Accept: text/markdown declaration. When Cloudflare’s edge
node receives this request, it does not return the raw HTML to the
client. The node performs the conversion directly on the request path,
transforming HTML into Markdown format in real time.
The data format produced by this real-time conversion is designed to
be extremely clean, typically consisting of three parts. The first part
is YAML-formatted metadata, containing the page title and description.
The second part is the noise-stripped body text in Markdown, with the
edge node automatically removing navigation bars, sidebars, and ads. The
third part is structured data appended at the end, preserving related
information from the page. When returning the response, the edge node
also includes token counts in the header, with two key fields:
x-markdown-tokens and x-original-tokens.
Through these two numbers, the agent can immediately know how much
budget was saved on this request.
According to Cloudflare’s test data, a page that originally required 12,345 tokens only needs 725 tokens after conversion. This means the token consumption for large models reading web pages is reduced by 94%. The specific compression effects can be seen in the visual comparison below:
Of course, the technology of converting HTML to Markdown is not Cloudflare’s original invention. In the market, Jina Reader already provides a similar service. Users simply need to add a specific prefix to a web page URL to get cleaned Markdown data. Meanwhile, Firecrawl also does similar work, providing interfaces specifically designed for web crawling and cleaning.
However, Cloudflare’s true advantage lies in its network position and massive service scale. Cloudflare’s edge network spans over 330 cities worldwide. This means nodes perform the conversion directly on the request path. Website owners don’t need to configure proxy servers or introduce complex middleware. For any agent capable of making HTTP requests, they don’t need to install any third-party client libraries. They simply need to send one extra format declaration, just like accessing any ordinary web page.
Features only truly change users’ default habits when they enter the workflow. On July 8, 2026, Cloudflare launched the drag-and-drop deployment tool Cloudflare Drop. For details on this release, see the Cloudflare Drop changelog. On the surface, it looks like a hosting tool modeled after Vercel Drop. Users simply drag a website folder into the browser, and the site goes live within seconds. Developers can compare the interaction differences through the Vercel Drop release notes.
In the Hacker News community, this project sparked extensive discussion. It garnered 523 upvotes and 285 comments—readers can find the details in the Hacker News discussion thread. However, the discussion at the time was almost entirely focused on terms of service and hosting competition. Almost no one noticed that it contained a strategy targeting agents.
The real change lies in the claim configuration menu for newly published websites. After users drag, upload, and claim a website, a setup guide panel appears. Here, Cloudflare offers four basic options: bind a custom domain, enable website monitoring, enable machine format, and set up access control.
The core significance of this product does not lie in saving a few steps. What it truly changes is the default setting when publishing a website. In the past, when developers launched a website, the only intended reader was the browser. People would spend time optimizing layout, compressing images, and designing interactive animations. Now, this guide option reminds you that machines are also your readers. As soon as a site goes live, you can enable machines to read it smoothly with a single click.
At the technical level, this hosting system relies on a temporary deployment mechanism. The relevant technical details can be found in the Claim Deployments technical documentation. In this official documentation, the development team lists AI automatic deployment and agent sessions as primary use cases. The command-line tool Wrangler provides a temporary sandbox, allowing running AI agents to deploy code directly within it. And Drop’s web interface merely wraps this machine-oriented command-line capability into a human-oriented graphical operation.
We need to place Drop and Markdown for Agents back into Cloudflare’s overall landscape. Over the past few months, the company has completed a series of tightly coupled moves in the AI space.
The first piece is the acquisition of the VoidZero team. This provides a foundational toolchain for agents to write and deploy code, and also makes Vite the default tool of the AI era. For the depth of its toolchain integration, see Cloudflare and VoidZero’s AI toolchain strategy. The second piece is Markdown for Agents and Drop, which have opened up a low-cost content reading channel. The third piece is the Monetization Gateway billing gateway, which allows machines to negotiate payments via the HTTP 402 protocol. For how the billing network operates, see the Monetization Gateway billing gateway. The fourth piece is the Access identity and bot management system, responsible for identifying machine identities and performing security verification.
These four pieces come together to reveal an exceptionally clear vision. Cloudflare doesn’t need to develop an agent runtime environment. Instead, it focuses its energy on the outside of the runtime, building a global network negotiation layer. When an AI agent’s request reaches a Cloudflare edge node, the network completes multiple interactions in an instant, as illustrated below:
The edge network now answers four questions simultaneously: who is visiting? Can they enter? Should they pay? What format should be returned? Achieving such multi-dimensional negotiation requires massive infrastructure and deep vertical integration across multiple capabilities. In the current cloud computing ecosystem, no other competitor can yet offer such a complete negotiation experience.
Take Vercel, for example. It has deployment and AI generation capabilities but lacks edge format negotiation and edge machine billing—though they have open-sourced Vercel agent-browser for local browser experimentation. AWS, while launching Bedrock AgentCore, primarily focuses on agent orchestration but lacks a managed browser and ultra-low-latency memory sharing.
Cloudflare has chosen a clear path. Through edge distribution and vertical integration, they have gained a unique competitive advantage in the agent era. At present, Cloudflare’s Workers AI inference layer already supports over 70 models from 12 providers. Details can be found on the Cloudflare Agents technical page and the Cloudflare Agents Week roundup. Through close collaboration with Anthropic, they have achieved a 5-millisecond ultra-low cold-start sandbox environment. For this technology, see the Anthropic and Cloudflare collaboration blog post and the Agent Readiness blog post. These accumulated capabilities make the edge negotiation layer operate with exceptional fluidity.
Format negotiation lowers the barrier for machine reading, but that doesn’t make web content inherently safer. When AI agents read external web pages, they are extremely vulnerable to prompt injection. The security field calls this technique Agentjacking. For the specific attack principles, see Agentjacking and prompt injection attacks.
The edge node transforms complex HTML into clean Markdown. Large models can thus read and understand web content more smoothly. This also introduces potential risks. Lower reading friction also lowers the barrier for injection attacks. Hackers can more easily hide attack instructions in web pages. As large models read smoothly, they may inadvertently execute them.
To regulate machine use of content, Cloudflare proposed the Content Signals mechanism. It
allows website owners to declare their content usage preferences. For
example, setting ai-train=yes or search=no.
However, this is merely a declaration, not a rigorous security
verification. It cannot prevent malicious websites from engaging in
deception. When the reading friction at the network transport layer
drops to zero, the demand for security verification and trust
certification becomes even more urgent.
Viewing the edge network as the negotiation layer for the machine era is a logical inference. However, objective reality may overturn it at any time. To truly make this network model an industry standard, Cloudflare must pass the following five difficult tests.
The first test is the adoption rate among website owners. Currently, Cloudflare bundles this feature with paid plans. Website owners must purchase at least the $25/month Pro plan to enable this service—free-tier users can only watch from the sidelines. The high price barrier may severely slow the adoption of this technology. We can look at the trajectory of the llms.txt specification. Developers can refer to the llms.txt official specification to customize text files. And according to the Ahrefs study on llms.txt, among sampled websites, although 28% of websites published the file, machines have never accessed 97% of those files. If the majority of website owners don’t enable this option, the format negotiation layer cannot coalesce into an industry default ecosystem.
The second test is the coordination attitude of client frameworks. To
trigger real-time format conversion at the edge node, the agent client
must proactively include the Accept: text/markdown request
header when making requests. However, currently, mainstream agent
frameworks have not yet adopted this logic by default.
The third test is the conversion quality of complex dynamic pages. The officially advertised 94% traffic savings comes from demonstration pages with relatively simple structures. The real web is filled with dynamic single-page applications and dense data tables. The real-time conversion quality and stability of the edge node still need to be validated.
The fourth test is the evolution of large models’ ability to parse native HTML. If future large models can effortlessly understand raw HTML code, then the marginal value provided by the intermediate network conversion layer will inevitably shrink rapidly.
The fifth test is the practical usability of proxy patterns in real-world scenarios. External conversion services like Jina Reader have an average latency of about 7.9 seconds. Humans cannot tolerate delays of several seconds, but for agents operating asynchronously, this latency is perfectly usable. Therefore, the edge network’s ultra-low-latency advantage may not form a decisive differentiator.
AI agents are replacing humans in browsing the web, and the advertising model is becoming unsustainable. Cloudflare has already built the edge network negotiation layer for this interaction. Publishing a web page now requires default support for machine reading and edge billing. We must allow machines to consume our content smoothly, and also charge them at the edge.