This week, TypeSafe launched its cloud API for Jev: send it a snippet of text, and your program gets back a row of discrete probabilities. Slack Code, announced by Slack in August, moves the intermediate process of coding agents writing software into dedicated group chat channels. OpenAI is testing Sponsored Agents in ChatGPT, allowing users who click promotional cards to converse for several turns with a sponsoring brand’s custom bot. Meanwhile, DeepSeek rolled out a new version of its execution framework, DeepSeek Harness, offloading code temporarily generated by the model into clean, isolated background processes while adding session resumption and remote machine scheduling along the way.
These four tools touch entirely different layers of the stack and share no technical overlap. For engineers immersed in systems day in and day out, the real questions are: what does each tool hand off, and to whom? What does it return in between? How do downstream systems pick up execution? And behind the promises written in official documentation, just how much public evidence is actually on the table? Among these shifts, the one that first lays bare the gap in data interaction is Jev, which treats LLMs as discrete classifiers.
For a concrete scenario, look directly at the customer support example listed in TypeSafe’s quickstart documentation: the system receives an incoming customer message: “Hi, I’ve been trying to connect my Stripe account for 3 days and it keeps failing. I’m losing sales. Please help ASAP.”
Conventional approaches either prompt an LLM to generate an
empathetic reply or instruct it to output a JSON classification; when
chasing ultra-low latency, teams train small local classifiers on
labeled data. Jev takes a different route: the program passes this
message in the state parameter and concurrently attaches
three pre-written questions within the same request.
All three questions are scored in parallel in a single request:
department is a single-choice question with options
restricted to billing, technical, and sales; frustration is
an ordinal rating question; and is_urgent is a boolean
question. The result returned by Jev contains no explanations—only
discrete probabilities composed of numbers. The most informative result
is the department question: technical has the
highest probability, yet confidence remains relatively low because the
billing option captures nearly one-sixth (0.159) of the
weight. The second-choice option contains real signal, not noise.
For this branching logic to hold, the numbers themselves must be trustworthy—precisely what large language models natively fail to provide. The underlying principle of LLMs is next-token prediction; their output probabilities represent the likelihood of sequence continuations, and their training objectives lack calibration against the correctness of judgment: a model might report 0.3 when its actual certainty is closer to 0.9. Jev’s core claim is that its numbers undergo calibration training and can be used directly against thresholds, allowing callers to perform multi-tiered routing in business logic (pedagogical example):
// Pedagogical example: Business routing consuming probability distributions
if (result.department.confidence < 0.3) {
// Confidence too low; model cannot determine intent clearly. Route to human triage.
return routeToHumanAgent(ticket);
}
// Confidence threshold met; route primary intent to the highest-scoring team
assignTicketToTeam(ticket, result.department.choice); // Assign to technical
if (result.department.probabilities.billing > 0.15) {
// Second-choice probability is significant, reflecting user's statements about payment failures; CC billing team as watchers
ccTicketToTeam(ticket, "billing");
}The third-tier check in this snippet depends on the full distribution, but the calibration issue discussed above applies equally here: whether the 0.159 weight can be taken at face value hinges on whether Jev’s calibration claims actually hold. If the numbers are trustworthy, the full distribution genuinely reveals the real weight of the second candidate, adding the billing team as watchers alongside the primary assignment to the technical team; if the numbers are untrustworthy, this multi-tiered logic is built on quicksand. Official marketing claims of “zero hallucinations” cover only the schema layer, offering no independent verification for calibration—which is precisely the key focus to watch in the empirical testing discussed below.
According to TypeSafe’s official blog post and question primitives specification, Jev’s entire system includes only three question primitives: boolean, single-choice, and ordinal rating questions, with answers uniformly represented as probability distributions. Single-choice and rating questions include a confidence score, while boolean questions lack an independent confidence field. Each question is evaluated independently, sharing no context with others. Because there are no contextual dependencies, the server can push questions to the underlying model for scoring simultaneously, resulting in very short network round-trip times; but precisely because context is severed, questions cannot cross-reference each other for joint reasoning—the severity of an earlier outage will not bias it toward classifying an issue as a technical problem.
The speed and cost advantages are backed by two third-party benchmark reports released on the same day. Every.to’s benchmark report recorded a set of figures: across the same batch of text evaluation tasks, Jev was roughly 25 times faster than standard baseline models; however, out of 7 text defects intentionally planted by the tester, Jev missed 1, catching only 6, whereas the baseline LLM caught all of them. The tester specifically cautioned: until its accuracy is validated in production, do not simply use it to replace complex workflows.
A validation report from Good Start Labs shows that Jev achieved a 91.5% agreement rate with baseline models, while consuming two orders of magnitude less monetary cost. However, the evaluators explicitly highlighted in bold: Agreement is the share of checks on which two judges gave the same verdict; it is not accuracy—agreement does not equal correctness. The evaluation team also caught a counterexample: DeepSeek’s model, spending slightly more money, yielded a slightly higher agreement rate. How much cost is saved on paper depends entirely on which baseline an engineer selects.
The vendor itself is not aligned on unit pricing. The official blog states that input tokens cost $0.042 per million while output tokens are free; in code comments within the developer documentation, however, an older set of prices written in July remains, with figures noticeably higher. A public pricing page has yet to launch, leaving outsiders unable to verify whether the current low price stems from a durable technological advantage or temporary losses subsidized out of pocket by the vendor—a point the vendor’s representative on Hacker News admitted they cannot prove at present.
The vendor’s promoted “zero hallucinations” claim breaks down into three engineering layers. The first layer is syntactic validity, which is a constructive guarantee: output strictly falls within the provided options without overflowing, which the vendor considers an enforceable type-system guarantee rather than empirical statistics (“Our number is not empirical. Schema matching is guaranteed”). The second layer is option correctness, where no system guarantee exists whatsoever: TypeSafe’s CEO admitted on Hacker News that the model can be “confidently wrong,” and the ground-truth references are merely the averaged outputs of two frontier models rather than ground-truth labels (evals.typesafe.ai). The third layer is confidence scoring, which reflects only aggregate statistical properties across the entire sample set, offering no guarantee for the specific decision of any single invocation.
For backend pipelines that only require discrete classifications, collapsing long-form text into probabilities eliminates tedious schema parsing. But when AI steps out from behind APIs calculating probabilities and into everyday communication tools to write code directly in project repositories on behalf of engineers, the interaction model shifts from one-way data requests into a collaborative chat room observed by multiple people.
For software engineering teams, fixing a bug often starts with a message in a channel: QA or product posts an error screenshot and casually @-mentions the development agent. But once the agent picks up the task, everyone’s burden often increases. If it works directly in the group, it floods the channel, pushing human discussions out of view with line after line of execution logs. If a human engineer takes over, they switch to their local terminal and work in isolation, eventually dropping a PR link back into the chat—forcing teammates to click out and scroll through web pages to piece together what changed. Leaving it to a CI pipeline is simpler, but leaves everyone passively waiting for results.
The Slack Code announcement on August 20 targeted this exact friction. Now, invoking a supported agent in any conversation prompts the system to automatically spin up a new channel named after the task—officially dubbed a “code channel,” dedicated one channel per task. The feature is rolling out in staged canary phases; Dreamforce in mid-September served merely as a second showcase.
When team members join the channel, they are no longer greeted by standard chat messages, but by a collaborative coding workbench: the center of the interface displays line-by-line diffs where reviewers can leave comments on specific lines of code, bundling these revision suggestions into a unified feedback payload to submit back to the agent; frontend changes feature embedded HTML prototype previews that render results directly; and the view integrates Canvas documents, Block Kit cards, and outbound links to code-hosting platforms. A persistent status indicator at the top of the channel displays one of six states—Working, Idle, Needs attention, Done, Inactive, and Archived—flanked by a red Stop button that permits any authorized member to halt the task at any time.
Under the hood, the Slack platform runs zero code itself: it neither
spins up containers locally nor executes git clone to pull
repositories, acting solely to orchestrate collaborative messages and
render rich-text interfaces in the chat view. All substantive code
writing and environment testing take place entirely within third-party
partner clouds: according to GitHub’s
official integration docs, GitHub Copilot completes modifications
asynchronously in secure cloud sandboxes and streams diffs back;
Anthropic’s Claude, via the Claude
Tag enterprise service, clones the user’s GitHub repository into an
ephemeral sandbox, commits edits back to a branch, and destroys the
sandbox once the session goes idle; Cognition’s Devin operates in
dedicated virtual machines, making the Slack channel essentially a
real-time mirror of its web workbench; and Vercel executes within its
own cloud infrastructure, pushing live preview URLs back into the
channel once builds complete. This pattern shifts the point of team
intervention earlier: rather than passively conducting delayed reviews
after an agent opens a PR, teams can now watch the coding process as it
unfolds—enabling non-technical stakeholders to catch requirement drift
early against live prototypes, and senior engineers to pull the plug
before flawed architectures take shape.
However, introducing multi-person channel observation directly into code authoring inevitably collides with permission boundaries. Among the four integrated partners, only GitHub clarifies a two-tier permission mechanism in public technical documentation: within the Slack channel, any participating member can suggest ideas or input natural-language instructions to the agent; however, only users with write permissions on the underlying target repository are authorized to trigger Copilot to actually modify the codebase. Furthermore, Copilot automatically generates pull requests using a GitHub App bot identity; if the repository enforces branch rulesets, the bot’s lack of attribution to a specific human developer means default policies require an independent human review on top of the repository’s standard approval requirements. As for claims by Slack executives that “Everything is done on behalf of the user, using the user’s ACLs,” no public technical documentation currently substantiates this statement.
The concrete implementation details around sign-off—the human approval mechanism for critical actions emphasized by the vendor—likewise remain blank. Slack claims that when encountering high-risk operations such as pushing code to production, agents will pause modifications within the channel to await expert approval. In existing public documentation, however, this flow remains confined to abstract behavioral descriptions: who defines the high-risk manifest, how specific approvers are designated in the system, and whether the agent pushes autonomously to production upon approval or requires a human to manually click a confirmation button are questions for which the vendor has yet to publish configuration specifications or control-plane documentation.
In terms of commercial strategy, Slack announced that code channel capabilities are available free across all plan tiers. The platform chooses to provide the messaging conduit and interactive UI at no charge, offloading token consumption and compute billing models entirely to third-party agent vendors. In the 27 days since the product was announced, no verified engineering team in the broader tech community has publicly published an in-depth retrospective of its use; discussions on Hacker News offer no reports of successful runs in real-world production environments, and tech outlet The Next Web published an open critique noting that “a thumbs up in a busy channel is not a code review.” Meanwhile, the ChatGPT integration—touted on partner slides early at the announcement—remains listed as “coming soon” across official documentation and developer portals.
While ChatGPT has yet to formally step into Slack’s enterprise collaboration channels, inside OpenAI’s own primary interface, the team has already begun testing how to push third-party proprietary agents directly in front of everyday users. Only this time, the role has shifted: from a coding tool helping engineers fix bugs to sales representatives dispatched by commercial brands.
On September 10, 2026, e-commerce analyst Juozas Kaziukėnas documented a live interaction while using ChatGPT. He entered a routine shopping query into the prompt box: “i want to buy the best picture frame with gold leaves”. Below the standard text recommendations, the interface surfaced a sponsored promotional card from US home goods retailer Wayfair, featuring a call-to-action button labeled “Chat with us” in the lower-right corner. When the tester clicked the button, rather than launching an external webpage in a browser, a brand-new conversation pre-populated with an initial query opened directly within the active ChatGPT window—backed by a custom chatbot pre-configured by the brand.
Six days later, OpenAI published an announcement on its official website titled “Reimagining advertising with AI”, formally christening this format as Sponsored Agents. Official help center documentation notes that this capability is currently in limited testing with a select group of US-based advertisers and is not open to unsolicited applications. In the announcement, the vendor outlined a representative scenario: after consulting ChatGPT for general interior decorating advice, a user can click a brand card to drill down into specific dining table dimensions, seating capacity, and tabletop finish care instructions.
This flow spans five discrete stages, each supported by varying degrees of verified evidence. The first stage is the ad card beneath the organic answer; OpenAI explicitly promises in official documentation that ad modules remain independent of objective responses and maintain clear visual separation. The second stage is clicking the “Chat with us” CTA button, which verified media recordings and first-hand analyst observations confirm launches a chat view directly on-site. The third stage is entering the on-site branded conversation, where the counterpart shifts from a neutral assistant to the sponsor’s brand representative. Examining official demo videos frame by frame, Hacker News user sigmar noted that once inside the branded conversation, prominent “ad” and “sponsored” labels vanished from view, leaving only a faint “learn more about business chats” prompt at the end of the line that scrolls out of the viewport as the user navigates downward (single-source demo observation, not a live product test).
The fourth stage involves multi-turn follow-up inquiries, such as asking the brand bot about dining table durability and dimensions (pedagogical illustrative example; the vendor has yet to publish unedited transcripts of live interactions). The final stage is the handoff: once purchase intent is established, the agent posts an external link to the merchant’s official website in the chat, and the user clicks through to leave ChatGPT, completing final checkout and entering shipping details on the merchant’s e-commerce platform (confirmed by official documentation).
Nowhere in existing public materials is there any mechanism for on-site payment or autonomous order placement. Settling transactions directly within the chat interface belongs to Instant Checkout, a separate product line developed in partnership between OpenAI and Stripe; across current official announcements and technical documentation, these two tracks remain entirely distinct and make no reference to one another.
On legal compliance and liability attribution, OpenAI establishes a sharp division of responsibility in §4 of its Ad Tools Terms: even if an agent’s prompts are drafted or configured for deployment by OpenAI on the advertiser’s behalf, the terms stipulate that “you are deemed the builder of the Sponsored Agent.” The advertiser assumes full legal liability for all statements, interaction logic, and external actions, with related claims treated directly as actions against the advertisement itself. Brands experimenting with this marketing format must shoulder the full compliance fallout if a bot hallucinates or misrepresents facts.
The technical openness of the platform is similarly restricted.
According to official Ads API documentation, while parameters define a
business_agent delivery mode, the specification provides
zero open API endpoints for creating, modifying, or managing agents. To
be frank, in the current closed test, all brand agents are manually
provisioned and deployed one by one by OpenAI personnel through internal
workflows. Based on public documentation, advertisers likely hand over
static product catalogs and Q&A collateral (an architectural
inference); there is no evidence indicating that these agents connect in
real time to merchant backend inventory databases or pull customer
profiles from CRMs.
Regarding user privacy and data boundaries, OpenAI currently promises publicly that advertisers receive only the messages users send directly within the sponsored conversation, along with de-identified, aggregated statistical reports; advertisers have no access to any prior conversation history between the user and the primary ChatGPT assistant. Yet an unresolved gap separates policy commitments from genuine engineering isolation: whether sponsored conversations are permanently retained in a user’s personal chat history, who ultimately owns copyright over conversation corpora, and how strictly the two systems are physically isolated at the infrastructure level remain unaddressed in official documentation.
As for billing models, standard ads running in ChatGPT are post-billed primarily on a CPM (cost per thousand impressions) or CPC (cost per click) basis; official documentation explicitly notes that even when advertisers opt for conversion optimization strategies, billing triggers remain tied to impressions or clicks rather than completed orders. For Sponsored Agents specifically, the vendor has not released detailed pricing terms; industry speculation regarding per-turn pricing or revenue-sharing arrangements is currently unsupported by public evidence. Furthermore, the only publicly verifiable pilot advertisers are Angi and Wayfair, both of which used forward-looking experimental phrasing in public communications; previously reported brands such as Best Buy, Lowe’s, and Adobe are standard static display advertising clients and were not included in this agent pilot.
Whether custom sales bots for merchants or intelligent assistants running in vendor clouds, their reasoning brains reside in vendors’ proprietary data centers, with all data transmitted across API protocols. But when developers return to local machines and want models to write scripts and execute commands directly on local file systems, text interaction alone is no longer enough—they need a sturdy cage around model-written code inside the host operating system. That is precisely the core issue tackled by the latest release of DeepSeek Harness.
A developer runs a command in their local terminal to launch DeepSeek Harness, abbreviated as DSH. The system’s underlying division of labor is straightforward: DeepSeek’s large language model runs entirely within cloud data centers, solely reading incoming context and performing language inference without ever touching local disks or network ports directly; what actually creates files, reads code, parses tool parameters, and executes concrete commands in the local terminal is a Node.js coordinator running on the developer’s local machine—what the industry calls a harness (the execution scaffolding responsible for receiving model instructions, assembling parameters, and calling system APIs locally). All logs and conversation history generated during execution are persisted as JSONL files in the local working directory.
On September 15, the DeepSeek team shipped a new preview release for this open-source execution framework: v0.1.6-alpha.1. The update performs major surgery on underlying execution mechanisms, delivering both new automation capabilities and breaking changes to existing scripts into developers’ hands simultaneously.
The first capability introduced in this release is session resumption
in headless mode. Headless mode refers to running purely as a background
process driven by external scripts without spawning an interactive GUI
or terminal interface. Previously, every time an external automation
script invoked DSH, it received a fresh, randomized session ID, and the
process exited immediately once the task finished—preventing scheduled
cron jobs or CI/CD pipelines from feeding follow-up commands back into
the preceding execution context, forcing teams to rely on makeshift
third-party community patches (such as
dsh-resume-headless).
The new release formally adds session adoption flags: external scripts can specify a prior session ID to resume, feed subsequent tasks directly via standard input (stdin), and receive runtime events streamed in real time as newline-delimited JSON (NDJSON) back to external monitoring scripts. According to the design specification, resumption enforces a strict adopt-only policy (adopting only existing sessions without creating new ones on the fly): if the system cannot locate the specified session ID on disk, if the working directory mismatches, or if the session is locked by another process, the framework fails fast with an error rather than silently opening a blank new session on typos—preventing pipelines from accumulating untracked dirty state.
The second capability is the introduction of SSH-based remote workspaces. According to the official SSH subsystem documentation, the system’s security and execution boundaries are partitioned with precision: DSH’s primary coordinator, the API keys required to call the model, and all local session records remain on the developer’s local machine; however, all task-driven file reads and writes, code inspections, and command executions are routed entirely to a remote POSIX server connected via SSH.
A helper program validated via SHA-256 checksums is pre-installed on the remote machine to execute concrete file operations and script commands on behalf of the local coordinator. This design avoids moving the model to the remote host and foregoes fragile Network File System (NFS) mounts; official design specifications dictate that if the underlying SSH connection drops unexpectedly, the system reports failure immediately rather than attempting automatic reconnection to replay partially executed commands.
The release’s most critical runtime behavioral change lands in the architecture of Programmatic Tool Calling (PTC, the mechanism allowing models to write executable code directly to batch-orchestrate local tools). PTC was designed to eliminate turn-by-turn round trips of structured tool calls back to the client for complex tasks, allowing the model instead to draft a small TypeScript script and execute batch operations in one pass inside the local runtime. The illustrative code below demonstrates this approach:
// Pedagogical example: Model-written PTC script completing batch queries in a single pass
const keywords = ['auth', 'session', 'token', 'crypto', 'permission'];
const summary: Record<string, number> = {};
for (const keyword of keywords) {
const matches = await tools.search({ query: keyword, limit: 10 });
summary[keyword] = matches.length;
}
return summary;In this code, five searches run locally in a single pass, avoiding the need to inject lengthy intermediate text generated during search back into conversation history on every turn; the model receives only the processed summary results at the end. Prior to 0.1.6, this script ran directly within a worker thread embedded in DSH’s primary process; the new release decouples its execution completely, spawning an isolated Node.js child process in the system background specifically to run the code.
A change far more aggressive than spawning a child process occurs the
instant this child process starts: the runtime proactively wipes all of
its environment variables, preserving only six minimal system keys such
as PATH and TEMP, and resetting process.env inside the
child process to a virtually blank object (see the implementation in the
process.ts
source code). In the execution instructions delivered to the model,
the vendor lays down a single rule: “process.env starts empty.”
In the Network Proxy Guide, the vendor explains the root motivation behind this change: preventing model-written code from iterating over environment variables to exfiltrate host network proxy configurations containing credentials, access tokens, and other sensitive secrets. Tightening this security boundary, however, immediately introduces backward-compatibility breakage. If automation scripts previously relied on host-injected environment variables to retrieve application configuration or auth tokens, upgraded child processes will read an empty object, breaking any logic dependent on those variables. It is worth emphasizing that this environment wiping is strictly confined to the Node.js PTC child process; the DSH primary host process and standard commands invoked via Bash tools remain unaffected.
The new features and operational breakage share a single origin. To
enforce process-level sandboxing locally, the vendor dropped support for
the E2B cloud sandbox backend, consolidating code execution around two
paths: isolated local child processes and remote SSH environments.
Dependency packages were also renamed from the code-runtime
series to the ptc-runtime series (changing invocation
references from ctx.codeRuntime to
ctx.ptcRuntime), with the initial release offering no
backward-compatible aliases for legacy names. On npm, the
latest dist-tag remains pinned to the older release
candidate, pulling the new release only when explicitly requesting the
alpha tag. Even so, dozens of migration tickets have
already been opened across the community ecosystem to adapt to these
changes; meanwhile, within experimental packages, new modules for
browser and desktop control suffered day-one bug reports from developers
noting failures when reusing multiple sessions (see GitHub
Discussion #6789).
Among these four tools, Slack Code debuted in August and had its second showcase this week, while the other three launched this week—yet empirical benchmark data from independent third parties remains virtually non-existent. Jev has only brief day-one verifications from two organizations, Slack Code lacks any publicly documented operational post-mortems from production engineering teams, Sponsored Agents has only a single analyst documenting the entry UI, and fewer than two days have elapsed since DSH pushed its new release.
For engineers working in front-line systems, vendor claims can only be treated as unverified technical hypotheses for now. Determining whether these tools can actually be deployed in production depends on when several key verification signals surface: Jev must await an official public pricing page and third-party evaluations measuring accuracy on real-world business workloads; Slack Code hinges on whether early adopters among engineering teams can produce rigorous production retrospectives; Sponsored Agents still lacks unedited, end-to-end conversation logs and transparent commercial billing terms; and DSH requires monitoring the migration progress of community ecosystem plugins along with the long-term stability of its environment-wiped child processes under sustained heavy load. Only as these concrete operational logs and system feedbacks become public will the true capability boundaries of these tools in production engineering systems become clear.