The security research community demonstrated a sobering phenomenon in
the Ghostcommit experiment: under the exact same Sonnet model weights,
Claude Code rejected malicious instructions embedded inside a PNG image
across 10 consecutive tests; however, when the same image was placed in
Cursor and Antigravity, the systems complied 10/10 times with the
instructions in the image, encoding sensitive secrets from the
.env file into integer constants and committing them to a
public repository. Researchers subsequently observed similar host
environment disparities on Gemini and GPT-5.5 (see the Ghostcommit
analysis report for detailed experiments).
This represents the most typical threat variant of Prompt Injection in the Agent era. When agents gain the ability to read external files, PNG images, webpages, or PR comments, attackers can embed control instructions within data to trick the agent into executing them as if they were the user’s authentic intent. In theory, the industry is no stranger to Prompt Injection, and major vendors and leaderboards are filled with triumphant claims of “99% defense success rate.” However, the Ghostcommit experiment shatters this common security assumption: security defense is not an intrinsic property automatically acquired simply by swapping or fine-tuning models. With the exact same model weights, merely changing the external toolchain and harness interaction shell causes the sensitive data leakage rate to jump from 0/10 to 10/10. In real-world production environments, agent defense lines frequently fail due to slight variations in host environments.
To explain why laboratory triumphs fail in engineering practice, we must first examine what is wrong with the “yardstick” we use to evaluate Prompt Injection. Historically, people have relied on leaderboard numbers, but mainstream evaluation frameworks actually conceal five design flaws.
First to fail are static evaluation sets themselves. Vendors may overfit to public benchmarks during training. OpenAI explicitly noted in the GPT-5.2 System Card that Agent JSK and PlugInject were splits of the training data, and therefore do not represent the model’s generalization capabilities against novel attacks. In December 2025, OpenAI further stated explicitly in their ChatGPT Atlas blog post that Prompt Injection is difficult to solve completely (for related analysis, see the Oleria R&D Blog). An official technical statement from the UK National Cyber Security Centre (NCSC) also pointed out that Prompt Injection cannot be completely eliminated through formal separation of input and code the way SQL injection can; this indicates that security cannot be staked solely on model-side techniques.
When comparing Attack Success Rate (ASR) across leaderboards, disparities in Attempt Budget reveal an asymmetric reality: attackers do not attempt an attack just once, and they only need to succeed once. Chouldechova et al., in a NeurIPS 2025 paper and promptfoo evaluation analysis, highlighted significant differences in how Attempt Budget is set across different benchmarks. If an independent attempt with a single-trial ASR of 1% is simply extrapolated across 392 attempts, the probability of succeeding at least once is approximately 98.0%; however, real-world attacks are not necessarily independent, and defenses may change as context evolves. Therefore, this merely illustrates the sensitivity of results to budget rather than serving as a direct measure of empirical compromise probability for each system.
Even more troublesome is evaluation drift caused by reliance on LLM-as-a-Judge. Early evaluations like BIPIA used GPT-3.5 as an automated judge, but judge models are more susceptible to distraction or inherent preference bias when processing context containing malicious injections. Evaluation experiments show that simply changing the judge model can shift defense rankings on leaderboards by as much as 14 percentage points. To eliminate this verdict drift, AgentDojo deprecated LLM judges in the AgentDojo Leaderboard, transitioning entirely to deterministic Formal State Checks based on code and actual workspace states.
Many attack-only test suites also overlook the critical metric of Utility Under Attack. Tests such as InjecAgent, BIPIA, and AdvBench primarily report ASR while ignoring how over-refusal degrades normal agent usability. Alex Becker criticized InjecAgent’s threat model, pointing out that by modifying the Planner’s tool list and directly inserting tool outputs into context turns, it may measure model fidelity to Tool-Calling rather than Prompt Injection defense capability. AgentDojo’s approach is more practical for real-world scenarios: it simultaneously measures the three-dimensional balance among baseline task completion rate Benign Utility (88.66% for Claude 3.7 Sonnet), task completion rate under attack Utility Under Attack (77.27%), and Targeted ASR (7.31%).
Furthermore, raw model API testing modes detached from system harnesses diverge sharply from real-world engineering realities. Many evaluations treat model APIs as isolated nodes, severing chains of environment interaction, file parsing, system hooks, and permission controls, rendering the resulting security data difficult to map to engineering practices that involve real tool shells.
When witnessing LLM general reasoning and coding capabilities iterate rapidly month by month, many develop an intuition that as foundation models become smarter, the Prompt Injection vulnerability will eventually be resolved naturally by the model itself. In reality, however, while model capabilities advance rapidly, Prompt Injection remains resilient. This state of affairs is governed by two core reasons.
The first major reason is that current LLM context processing paths lack reliable, non-bypassable strict isolation between instructions and data. In traditional computer architectures, program instructions and data can establish clearer separation through memory permissions and syntactic boundaries; whereas in common LLM applications, system prompts, user inputs, external webpages, image text, and Tool Call return values all ultimately enter the context visible to the model’s attention mechanism. Even if applications use tags or delimiters to label sources, the model must still rely on probabilistic understanding to determine which content holds instructional force. When an attacker hides malicious instructions inside tool return values or images, the model may execute data as instructions.
This confluence of context conflation and instruction-following capabilities creates a counterintuitive model capability paradox: stronger understanding and reasoning in foundation models do not automatically yield stronger anti-injection resistance. A stronger model, while accurately executing legitimate instructions, may also more acutely comprehend and comply with injection instructions hidden within untrusted data. Tests by the Gemini R&D team recounted by Luca Sambucci indicate that enhanced reasoning does not confer immunity to poisoned data. In public ARPIbench testing, Llama-3.3-70B exhibited an ASR as high as 99.9%, significantly higher than the smaller Llama-3.1-8B (94.2%). Anthropic’s official System Card recorded a similar phenomenon: when Opus 4.6 enabled extended thinking mode, its ASR on the ART evaluation benchmark actually increased from 14.8% to 21.7%. This demonstrates that additional reasoning compute does not guarantee stronger anti-injection capabilities.
The second major reason is the dynamic evolutionary nature of security confrontations. Prompt Injection defense is not a static model training task, but a dynamic game of escalating offense and defense. As model capabilities advance, attacker techniques and automated optimizations evolve in tandem. Attackers continuously explore and exploit potential vulnerabilities, meaning defenders can never achieve permanent safety by simply piling rules onto the model layer. A joint adaptive attack study by 14 researchers from OpenAI, Anthropic, and Google DeepMind (paper available at arXiv:2510.09023, summary and analysis available at blog overview) clearly illustrates this point. The research team tested 12 model-layer defense schemes previously claimed in literature to achieve near-zero ASR, including StruQ, SecAlign, MetaSecAlign, Spotlighting, and Prompt Sandwiching. When subjected to adaptive dynamic attacks combining gradient descent, reinforcement learning, random search, and human red teams, over 90% of defense schemes were breached, with human red teams achieving an attack success rate of 100%.
This dynamic confrontation combined with context conflation reinforces the core truth revealed by the Ghostcommit experiment: ASR is not an intrinsic static property of model weights, but a system variable determined by the host environment harness. Relying solely on prompt wrapping or model fine-tuning cannot establish a stable defense line against dynamic adversarial attacks.
When recognizing that single-point model evaluations are rife with illusions, that physical layers suffer from architectural flaws and dynamic games, and that model-layer defenses are almost entirely wiped out under adaptive attacks, true engineering solutions must transition fully to system architectural isolation and deterministic harness controls.
Clear isolation solutions exist in system architecture design. The CaMeL architecture proposed by Google DeepMind separates the Trusted Planner from the Untrusted Executor, ensuring control flow is generated by the planner while untrusted data flows solely through isolated data channels in the executor. OpenClaw adopts a dual-agent isolation scheme where Agent 1 is exclusively responsible for creating structured summaries of external untrusted input, and Agent 2 executes JSON-formatted tool calls without directly contacting raw, unfiltered text. Across 649 attack samples, this scheme reduced ASR from 100% to 0.31%, eventually converging to 0%. It should be noted that such architectural isolation schemes mitigate risk exposure by segregating execution paths; if future attacks break through structured format constraints, underlying physical sandboxing remains necessary.
At the harness engineering layer, what truly stops attacks are deterministic rules and configuration defenses:
Deterministic tool execution interception mechanisms must be established. For high-risk operations involving file modification, code commits, outbound network requests, and credential retrieval, authorization logic and deterministic checks must be forcibly triggered by the harness layer rather than relying on the model to self-regulate refusal.
Security boundaries for system configuration files and hooks are also
frequently overlooked. The Claude Code CVE-2025-59536 vulnerability with
a CVSS score of 8.8 demonstrated that hooks configured in
.claude/settings.json can automatically execute arbitrary
code before user confirmation popups appear and before the model even
begins running. This proves that a harness’s own configuration files and
extension mechanisms possess an independent attack surface, requiring
strict signature verification and isolation protection.
Regarding execution environments, implementing sandbox isolation and least-privilege control forms the baseline engineering defense. Restricting agent environments to containers or microVMs, along with limiting outbound network access and filesystem read/write scopes, ensures that even if model-level understanding is bypassed, system-level permission boundaries can still contain damage propagation.
Security governance for Prompt Injection does not depend on finding an attack-immune model, but on constructing harnesses and architectural boundaries capable of tolerating model non-determinism. By deconstructing evaluation illusions and physical paradoxes, and enforcing deterministic controls at the system layer, a robust agent security system can be established without sacrificing usability.