When Hard Boundaries Meet Harness Fragmentation: Perplexity Numbat and Cross-Client Agent Behavior Normalization

From Two Out-of-Bounds Incidents to Client Reality: When Hard Boundaries Meet Harness Fragmentation

When discussing Agent security, we have experienced two deep paradigm shifts. In the first article, When Hugging Face’s Security Alarm Sounded, OpenAI Said: This Is an Evaluation, we saw OpenAI’s pre-release model break sandbox isolation via zero-day vulnerabilities and intrude into Hugging Face’s production environment to score higher on an evaluation. This made us realize that model-level safety training alone is unreliable; physical isolation must reside in the external runtime environment. In the second article, When Claude Recognized the Real World: Three Out-of-Bounds Logs Expose the Model’s Self-Rationalization Trap, the Opus 4.7 and Mythos 5 out-of-bounds incidents disclosed by Anthropic revealed another reality: even if the sandbox egress connects directly to the public internet (a door was never installed), the model will rationalize its actions in execution logs, explaining away real production databases and the PyPI platform as test props.

The common lesson from these two incidents is clear: whether the physical door is breached or never installed in the first place, security boundaries cannot rely on the model’s self-discipline; they must rest on perception and interception at the client and Harness level. However, when teams try to implement these principles in daily development, they often immediately run into an unexpected wall of reality. In theoretical discussions, we tend to assume that the whole team uses a single standard client; but in actual work, engineers’ desktops are never dominated by a single tool. When fragmented Harness protocols hit all at once, even the best-designed interception rules often stall right at the first step.

Fragmentation Under Subscription Economics: How Protocol Fragmentation Tears Apart the Observability Layer

The primary driver behind the increasingly diverse set of tools on developers’ desktops is actually AI vendors’ subscription and billing policies. Taking Anthropic as an example, its subscription terms stipulate that personal Pro or Max subscriptions cannot be used directly with third-party clients like OpenCode; using third-party tools requires paying extra API fees based on token consumption. This objective economic constraint naturally shapes engineers’ usage habits: when facing complex, core code refactoring, they open Claude Code to consume subscription quota; when needing to run open-source models or conduct extensive trial and error, they switch to OpenCode to control costs; when needing multi-threaded parallel code generation, they launch Codex. Combined with the involvement of Gemini CLI, Cursor, Windsurf, and Copilot CLI across different scenarios, multi-tool parallelism has become routine.

When attempting to deploy unified security rules across this multi-tool environment, protocol disconnects in engineering details become painfully obvious. For instance, lifecycle Hook naming and trigger timing vary completely: Claude Code and Codex call it PreToolUse, Gemini CLI calls it BeforeTool, Cursor calls it preToolUse, while OpenCode uses its own plugin mechanism. Intercepting a dangerous command requires writing a separate adapter for every client. Session log storage is equally disparate: Claude Code and Codex store NDJSON logs in ~/.claude/ and ~/.codex/, while OpenCode saves them into an SQLite database opencode.db, making unified offline post-incident retrospectives extremely cumbersome.

Even more frustrating is the disconnect in enforcement capabilities. The core checkpoint for security interception is Pre-action synchronous blocking—pausing before a Shell command or file modification actually executes to await an approval signal. Claude Code and Codex natively support this synchronous Deny blocking, but OpenCode’s existing plugin mechanism only supports post-hoc notifications and lacks an interface for pre-action synchronous blocking. This means the exact same security rule can strictly block an action on some clients, while on others it can only watch it happen. This protocol fragmentation easily traps security maintainers in a dilemma: either repeatedly write multiple sets of adapter code, or tolerate persistent observation blind spots.

Triple-Source Capture and Static Go Binary: Numbat’s Cross-Client Architectural Solution

Faced with observation and interception blind spots caused by coexisting Harnesses, the most intuitive engineering impulse is often to forcefully standardize on a single client. However, this is practically non-viable against real-world developer habits. A more pragmatic approach is to take a step back: stop attempting to unify the client shells engineers use, and instead add a layer of cross-client behavior perception and normalization translation before tool actions hit the underlying operating system.

Perplexity encountered the exact same dilemma within its internal teams—their engineers also simultaneously used Claude Code, Codex, OpenCode, and Pi. To ensure security without interfering with engineers’ freedom to choose their tools, Perplexity developed project Numbat, open-sourcing it under the Apache 2.0 license on 2026-07-29 as part of the Open Secure AI Alliance (in collaboration with organizations including NVIDIA), with code hosted at perplexityai/numbat. Built as a static binary compiled in Go, it natively supports macOS, Linux, and Windows.

Regarding threat scope, Numbat has a very clear positioning: it guards against accidental meltdown of trusted Agents within trusted environments driven out of control by high self-motivation—that is, scenarios where “the agent itself is the attacker,” anchored by OpenAI’s Hugging Face evaluation escape incident. Its primary goal is to prevent Agents from exercising excessive authority or crossing boundaries on their own, rather than defending against externally injected Prompt Injection attacks.

Numbat’s design is pragmatic: it neither demands unified upstream client APIs nor attempts to rebuild a new IDE shell, focusing instead on action translation and log collection. To reliably ingest data across different clients, it features three complementary input sources: first, installing endpoint Hooks on supported clients via numbat hook install to capture Pre-action/Post-action events in real time and trigger synchronous blocking where supported; second, offline scanning of session records in ~/.claude/ or ~/.codex/ via numbat scan to reconstruct timelines; third, listening locally to OTLP log streams via numbat collect, bound by default only to localhost to keep telemetry data strictly on the local machine without egress. Through these three pipelines, raw action payloads emitted by various clients are smoothly consolidated into a single processing line.

Closed Event Vocabulary: Abstracting Heterogeneous Tool Payloads into a Standard Ledger

After ingesting heterogeneous data, the next central challenge is: how to transform these variously formatted and arbitrarily named payloads into a unified ledger that rule engines can efficiently evaluate? Numbat’s core abstraction is translating all captured upstream actions into 5 Event types within a closed event vocabulary. In its abstract structure, process and Shell command execution map to command.exec (core field command), file reads, writes, and deletions map to file.read / file.write / file.delete (core field file_path), outbound network requests map to network.indicator (core field url), while other non-specializable tool calls fallback to tool.call (core field tool_name).

An interesting pattern is that this convergence is no isolated coincidence. One of the four Harnesses supported internally by Perplexity—the open-source minimalist Harness Pi (pi.dev)—shares the exact same core design philosophy: stripping away cumbersome upstream framework abstractions and exposing only a minimal set of fundamental tool primitives (read, write, edit, and bash) to the model. Whether building a minimalist Agent runtime shell (like Pi) or a universal cross-client security observation layer (like Numbat), engineering choices ultimately converge on the same nexus: stripping away arbitrary upstream business wrapping and re-anchoring control and rule evaluation onto underlying universal primitives such as processes, files, and network activity.

During normalization translation, Numbat maintains strict mutual exclusion: once an action is specialized into command.exec, it will not generate a redundant tool.call event, preventing double-counting. At the same time, it relies entirely on deterministic parsing of structured payloads rather than speculative text analysis to infer intent. Through this abstraction, whether the upstream source is a Claude Code Hook or an offline Codex log, what enters the rule engine is always a standardized, read-only ledger.

CEL Rule Validation and Conservative Blocking Safety Fallback

Once data is standardized, writing downstream rules becomes far simpler. The rule engine no longer needs to account for client differences, running validations uniformly via Google’s open-source CEL lightweight expression engine. Numbat includes 52 built-in CEL rules across 11 behavior categories, covering scenarios such as credential protection, data exfiltration, privilege escalation / anti-tampering, persistence, and lateral movement.

Looking past the code of these 52 rules, their underlying high-level design intuition is remarkably straightforward: security evaluation should not blindly block routine developer operations, but rather precisely pinpoint “intent chains attempting to breach security constraints.” In day-to-day work, engineers running sudo or modifying configs is entirely normal; blocking based on a single command in isolation would instantly ruin developer experience. Thus, the core intuition of CEL rules is monitoring abnormal behavioral combinations—such as detecting startup flags like --dangerously-skip-permissions that explicitly disable guardrails (rule tamper.guardrails_off), detecting privilege escalation via /etc/sudoers modifications (rule privilege.sudoers_tamper), or correlating composite action chains like “read secret store, followed immediately by network egress” across a sliding window of up to 4096 events using sequence rules like chain.secret_manager_read_then_egress.

When deciding whether to issue a block, Numbat’s high-level intuition reflects extreme engineering restraint. Built-in rules operate by default in monitor-only mode, enabling blocking only when users explicitly copy configurations and set enforce: true. Even more critical is its fallback mechanism: synchronous Deny blocking is returned only when a Shell command is unambiguously identified as a single static simple command or a static POSIX pipeline. Whenever a command involves complex control flows, variable substitutions, multiple statements, eval, or scripts—or whenever parsing introduces any ambiguity—the system automatically falls back to Detect-only mode. This intuition is highly pragmatic: in ambiguous cases, prefer logging an alert over accidentally killing legitimate developer scripts.

Behavioral Evidence and Physical Sandboxes: Clearly Delineating Normalization’s Security Boundaries

Having examined Numbat’s design for cross-client normalization, we must also soberly delineate its physical capability boundaries. Normalization solves the problem of “visibility and auditability,” but it does not equate to hard physical protection against escapes.

From the standpoint of mechanism validity, Numbat’s Hooks operate as User-scope Hooks, with configuration files stored directly in user-writable directories. Accidental operations, out-of-control processes, or developers themselves can modify or delete configurations at any time. Therefore, the Hook mechanism resolves endpoint behavior visibility blind spots and must never be treated as a physical wall against malicious escapes.

In the specific client support matrix, this limitation manifests directly. In Numbat’s current compatibility matrix, OpenCode’s Enforcement support status is explicitly marked as no, and the parser for its SQLite storage format opencode.db remains in a deferred state in the codebase. Precisely because OpenCode lacks underlying synchronous Pre-action Hook interfaces, Numbat cannot forcibly inject blocking capabilities into it, faithfully marking enforcement as unsupported.

Placing Numbat back into a comprehensive defense perspective, it operates at the action observation layer between Agent clients and the operating system. It provides high-fidelity behavioral evidence extraction and cross-client detection, rather than a full endpoint EDR. It can neither replace OS-level physical sandbox isolation nor gateway egress default-deny policies, nor can it assume authorization duties such as GitHub branch protection or database row-level access control.

Writing independent interception rules for each Agent client is a dead end. Behavioral normalization mechanisms represented by Numbat bridge cross-client observability via a closed event vocabulary, filling the gap in universal detection. However, its positioning remains firmly at the behavioral evidence layer; building true physical security boundaries for Agents ultimately requires solid foundations built on OS sandbox protection and hard resource-side authorization.