Figure 1: Comparison of the three surviving bets in the legal AI
market: core difficulty assumptions and architectural forms across the
content/validation, weight, and integration layersOn September 17, 2026, OpenAI announced Astra for Law. In a 200-question test on a private validation set from independent benchmark provider Vals AI’s legal research benchmark, with maximum reasoning effort enabled on both sides, the product achieved a 54.0% pass rate—15.3 percentage points higher than the 38.7% recorded by the base GPT-6 Astra model using standard web search alone. While official messaging touted a 40% relative performance improvement, calculating from the baseline figures yields an actual increase of 39.5%.
This comparison is self-reported by the vendor, restricted to a 200-question private validation set, and has not been independently audited by a third party. Legal practice offers virtually zero tolerance for error. When drafting a legal memorandum, even if counsel gets most points right, misidentifying the good-law status of a single pivotal precedent destroys the argument’s validity and exposes counsel to malpractice liability. Vals AI established granular scoring criteria requiring a system to satisfy every individual rubric item on a question to earn credit; missing even a single item yields zero—a rule known as all-pass. The 54.0% figure merely indicates the proportion of questions where the system satisfied every criterion; it does not imply that legal AI is ready to replace practicing attorneys. Nor can the remaining 46% of failed questions—which include points lost for partially incomplete answers—be conflated with the model’s hallucination rate.
As product lead Boehmig explained in an interview (LawNext interview): this release does not introduce a new model, but rather a model configuration tuned specifically for legal practice. Far more revealing than benchmark scores on paper is this concrete engineering approach—understanding what the product actually looks like under the hood.
In building Astra for Law, OpenAI neither pre-trained a vertical model from scratch nor performed full-parameter fine-tuning. The system consists primarily of three parts: the general-purpose foundation model GPT-6 Astra, an external legal retrieval index, and system instructions tuned for legal analysis and legal writing. The accompanying index spans 230 million URLs—a figure that strictly reflects the total count of indexed web pages and legal document links, rather than 230 million distinct judicial opinions. Dynamically updated daily, this index covers U.S. statutory codes, administrative regulations, court procedural rules, and administrative adjudications. The underlying data is sourced from Free Law Project, the non-profit that maintains the open-source legal database CourtListener. According to the Free Law Project wiki page, the project claims to cover 99.9% of published U.S. precedent, though that claim dates to a 2023 cutoff and is likewise a self-reported metric by the provider.
Armed with abundant compute and top-tier research talent, a general model company entering a legal market characterized by high barriers to entry and strong willingness to pay opted neither to retrain a vertical foundation model nor to conduct domain fine-tuning. Instead, it delivered the lightest, least labor-intensive play on the field. That technical choice warrants closer examination.
The explanation lies in how legal tech has evolved over the past four years. To make sense of vertical AI products of this kind, one must first look at what the product actually looks like. A product’s form is not merely a reflection of code architecture; it represents a high-stakes strategic bet backed by real capital: practitioners must wager on whether the core bottleneck of a vertical domain is trapped inside neural network weights or located in the supporting environment outside the model’s boundary.
A persistent assumption has long gripped the industry: that when foundation model giants enter professional domains, they will tear down existing systems and start from scratch—that vertical industry deployment requires pre-training a dedicated base model on domain corpora, or at least performing full-parameter fine-tuning. This expectation came with a tacit engineering pecking order: pre-training from scratch and fine-tuning were hailed as hardcore, heavy-duty engineering worthy of the word “professional,” while organizing retrieval logic, refining system instructions, and building evaluation harnesses were dismissed as glue code and superficial wrappers operating on the periphery. The form OpenAI chose for its product shatters this prejudice: a team flush with compute deliberately chose foundational work at the bottom of that pecking order and polished it into a vertical product that commands serious industry scrutiny. Over the past four years, market participants spent real capital conducting an industry-wide ablation study—even before the term had become common parlance.
The first approach to be tested by reality and retired was pre-training standalone vertical foundation models from scratch. Early teams were convinced that specialized verticals possessed insurmountable corpus moats, believing legal and financial documents were steeped in arcane syntax, archaic jargon, and intricate reasoning that general corpora could never absorb, making domain-specific training data an absolute prerequisite. In March 2023, Bloomberg spent heavily to unveil BloombergGPT, curating a training corpus that included 363 billion tokens of proprietary financial data to pre-train a 50-billion-parameter model from the ground up. Yet as general frontier foundation models raced ahead with larger parameter counts and more diverse corpora, the generational gap in general complex reasoning widened dramatically, and specialized small-to-midsize models hit a ceiling. Wayne Barlow, Bloomberg’s Global Head of Terminal Products, made this plain in an interview (a-teaminsight interview): BloombergGPT was an exploratory research model that never made it into any official Bloomberg commercial product; Bloomberg’s current product suite runs on a hybrid multi-model architecture spanning commercial proprietary models, open-weight models, and internally developed small custom language models. From that point on, industry enthusiasm for pre-training proprietary vertical foundation models cooled.
The next strategy to hit a wall was full-parameter fine-tuning of general foundation models on domain-specific corpora. Harvey, a legal workflow automation startup, gathered vast troves of U.S. judicial precedent for its first-generation product, running intensive fine-tuning on top-tier closed-source general base models. Practical hurdles mounted quickly: statutes and judicial precedents are subject to continuous revision, and encoding fluid legal facts into static neural network weights made generated output prone to obsolescence. Meanwhile, as frontier general reasoning models leaped across generations, their inherent general cognitive capabilities erased the marginal advantages fine-tuning had fought to sustain. As research firm Sacra detailed in a Sacra research report: frontier reasoning models rapidly turned legal logical deduction into a commoditized capability, leaving Harvey no choice but to abandon the proprietary models it had spent heavily to fine-tune.
Two successive generational shifts left behind a firm underlying consensus: disconnected from the reasoning engine of general foundation models, closed vertical models cannot stay competitive over a technological marathon; and attempts to bake dynamic industry facts into model weights through fine-tuning can neither keep pace with the evolution of base models nor satisfy professional standards for factual and temporal precision.
Following two rounds of shakeouts, the contenders remaining in the arena reached a consensus: the primary bottlenecks in legal work lie largely outside general cognition, residing instead in the runtime environment and professional support network external to the model. Yet as they pushed toward mature commercial software, teams diverged on exactly where the critical difficulty lies, spawning three distinct product bets.
Inside a litigation practice group at a major law firm, a junior associate files a motion to dismiss, citing what appears to be controlling precedent. When oral arguments begin, opposing counsel pulls out a subsequent decision showing that the core legal rule established in that precedent was reversed by an appellate court two years ago. The judge denies the motion on the spot; the junior associate not only loses the motion, but also faces an internal malpractice inquiry. This cautionary scenario highlights the unique nature of legal authority: within a judicial opinion, only the core rule of law necessary to decide the actual issue in dispute carries binding authority, a concept known as the holding; judicial commentary or peripheral discussion is non-binding, collectively referred to as dicta. A holding valid at trial may be reversed or vacated years later by a higher court. Every day, newly issued decisions cite, endorse, distinguish, or overrule past precedents, weaving an ever-shifting web of authority. If an attorney drafting a brief fails to trace how subsequent decisions have treated a key proposition, there is no way to verify whether it remains good law. In the legal industry, the indexing network that dynamically tracks the continuing validity of judicial authority is none other than the citator graph, perfected over more than a century.
Legacy legal publishers built their moats of authoritative content and cross-validation atop this very citator apparatus. Thomson Reuters, with its century-old case law repository and team of professional legal editors, alongside LexisNexis, which commands the century-old citator brand, placed their bets squarely on data foundations outside the model. Thomson Reuters noted in a blog post (TR blog): the defining variable in legal AI is anchored deep within the content foundation the model relies upon; the base model itself does not constitute the primary differentiator. Underpinning that stance are more than 1,500 attorney-editors manually curating case law and maintaining the KeyCite citation research service. LexisNexis adopted a similar technical playbook. When launching its Protégé platform in May 2026, CEO Sean Fitzpatrick set the tone (LexisNexis announcement): legal AI must deliver trusted work product that attorneys can verify with confidence and defend in court. The platform relies on the Shepard’s citation service for cross-validation, dynamically routing tasks between custom toolchains and Anthropic-powered skills. According to vendor disclosures, Thomson Reuters did experiment with modifying weights: investing roughly $40 million (with vendor-disclosed compute expenses of approximately $100,000 to $200,000, consuming 35,207 B200 GPU hours) to perform continued pre-training and domain fine-tuning on an open-source Qwen base, resulting in the Thomson 1.0 model. Official reports showed that its AIME mathematics score dropped from 93.3 to 90.0, while Terminal-Bench fell from 45.2 to 40.5, forcing the engineering team to introduce a multi-model hybrid routing architecture to offset the regression. The cost of altering model weights was illustrated here in stark detail.
After domain fine-tuning fell out of favor, why did attempts to manipulate model weights experience a resurgence? The key lies in the fact that they address entirely different technical problems. The earlier fine-tuning approach focused on static knowledge injection, attempting to etch shifting statutes and case holdings into parameters—an effort doomed to lag behind fast-moving judicial reality. By contrast, post-training initiatives such as Tenet focus on behavioral conditioning: using reinforcement learning to guide the model through multi-step workflows repeatedly, honing behavioral paradigms around retrieval, judgment, and long-horizon reasoning. Patterns of analytical behavior are far more stable than specific statutory provisions, and once internalized, they do not suffer from factual obsolescence. Harvey, the legal workflow startup, committed to this path: launching its Tenet post-training initiative, selecting the long-context Kimi K3 base, partnering with Fireworks for inference and training cloud infrastructure, and applying the asynchronous reinforcement learning algorithm GSPO across roughly 150 B300 GPUs over two months of training (self-reported vendor figures). Ultimately, on the LAB held-out evaluation suite, the trained model achieved a near-doubling in overall completion rate compared to the initial base model (likewise self-reported). The Harvey team summarized in their technical post-mortem (training retrospective): post-training, evaluation pipeline optimization, and grader architecture design must lock together seamlessly; what ceiling a model can demonstrate depends on how much room for exploration the external training environment affords. They remain convinced that intricate legal deduction and long-horizon chains of thought must be etched deep into the parameters—a depth that external glue code simply cannot replicate.
The third path focuses its energy on the workflow integration layer surrounding the model, embedding the model into office software as configured parameters and external plugins. Tech giants have displayed striking strategic alignment here: Microsoft integrated Legal Agent into Word in April; Anthropic launched Claude for Legal in May with external plugins and Model Context Protocol (MCP) connectors; Google released Gemini Enterprise for Legal in August; and OpenAI debuted Astra for Law in September. All four tech titans opted for peripheral configurations and plugin integrations; not a single one pre-trained a dedicated vertical model. Legora, a startup building a multi-model legal workspace, took the same stance, orienting its business toward grid-based contract review and collaboration portals while treating underlying foundation models as swappable, decoupled general components—a pragmatic engineering philosophy that resonated widely in a Hacker News discussion.
Figure 1: Comparison of the three surviving bets in the legal AI
market: core difficulty assumptions and architectural forms across the
content/validation, weight, and integration layers
To see how this integration-layer path plays out, Astra for Law offers a ready-made engineering case study. The system directly leverages the flagship general-purpose base model GPT-6 Astra without modifying internal parameters; all incremental work centers entirely on peripheral components. On the data retrieval side, the system attaches a legal retrieval index encompassing 230 million URLs. As noted earlier, this metric measures the total number of web page and document links rather than distinct judicial decisions; updated dynamically daily with data feeds from the non-profit Free Law Project, it allows the model to query active, good-law precedents, federal and state statutes, administrative regulations, court practice guides, and administrative rulings at any time. On the reasoning guidance side, the team crafted an exhaustive set of system instructions tailored to factual lead retrieval, issue framing, and memorandum formatting, conditioning the model to structure its analysis along the lines of legal reasoning. On the security and compliance front, the platform erected an enterprise-grade governance framework tailored to legal corporate environments.
Under data governance, OpenAI opened a dedicated Trusted Access pipeline directly into ChatGPT and Codex for selected law firms, offering zero data retention (ZDR) on APIs for eligible firms and by default excluding ChatGPT Enterprise usage logs from human review. Designed in collaboration with prominent law firm Latham & Watkins, the compliance framework accounts for permission isolation, ethical walls, client mandates, and firm oversight—with a LawNext report revealing that the zero data retention agreements span roughly 30 pages. Alongside the launch, OpenAI unveiled its partner ecosystem: 26 partner-developed plugins, 9 community-maintained plugins, and 47 custom skills, along with a ChatGPT for Word add-in bridging editorial workflows. Plugin integrations include preview connectors for Thomson Reuters’ HighQ and CoCounsel Legal, alongside specialized providers such as iManage, Intapp, and DeepJudge. Opening 26 initial partner plugin slots demonstrates that the platform giant has reined in ambitions of monopolizing end-to-end vertical workflows, acknowledging on an engineering level that control over specialized workflows remains firmly in the hands of dedicated vertical software vendors.
The officially promoted 54.0% benchmark victory demands an objective examination of its evidentiary boundaries. The Vals AI benchmark page notes that this evaluation was run on a private validation set, meaning the findings are vendor self-reported. Assuming an equal-weight, all-or-nothing scoring scheme across 200 questions in a single run, the base model’s 38.7% score would translate to 77.4 questions. That non-integer figure points to probable multi-trial averaging, hierarchical aggregation, or weighted scoring (the newly configured version’s 54.0% arithmetically matches 108/200, but given the asymmetrical numerical precision, one cannot reverse-engineer the true number of correct questions), and the official methodology has yet to be clarified. A Vals AI official X post confirmed these figures stem from vendor internal testing while disclosing another set of self-reported results: a weighted score of 90.0% vs. 84.1% and an all-pass rate of 53.7% vs. 39.6%, currently undergoing closed-door re-verification against an undisclosed held-out test set. On the public leaderboard based on 208 undisclosed test questions, only the stock, unconfigured GPT-6 Astra appears, hovering around 49% (ranking 20th among 61 models, with the top two tied for first at 55.29%); the version equipped with the legal configuration has yet to establish an official public record.
The two comparative case studies OpenAI highlighted in its presentation exhibit similar limitations common to cherry-picked demos. In the litigation research example, the general model cited a 2025 Delaware trial court decision, Paragon Metal Holdings v. Smith. The Delaware Supreme Court had already reversed in part that decision on July 1, 2026 (reversing and remanding on reasonable reliance while affirming the finding of misrepresentation); the general model failed to recognize the change in legal validity, whereas Astra for Law retrieved the subsequent reversing opinion. In the transactional review case, Astra for Law successfully located Oliver Wyman v. Eielson, 282 F. Supp. 3d 684 (S.D.N.Y. 2017)—a Southern District of New York decision applying Massachusetts substantive law to counterclaims—which the baseline control model missed entirely. As noted in a Technology.org report, these two vendor-selected case studies have not been independently tested; a one-off demonstration cannot establish the true probability distribution of encountering similar pitfalls during continuous, high-volume production use.
With this legal configuration baked directly into the native platform, different tiers of the ecosystem are absorbing disparate impacts. Taking the most direct hit are thin wrappers with negligible technical moats. In the past, startups could slap together a crude vector database query interface, plug into a general LLM API, and sell per-seat subscriptions to law firms. Now that OpenAI has opened native access to GPT-6 Astra Law for selected firms—bundling an external index spanning 230 million URLs, specialized legal instructions, and enterprise data governance—rudimentary tools that rely solely on prompt engineering tricks and generic retrieval shells have lost any justification for charging recurring fees.
Mature vertical software in the second ecosystem tier has displayed
greater defensive resilience. Harvey, entrenched in law firm operational
workflows, has posted substantial commercial growth. According to a Sacra research
report, ARR.club, and valueaddvc, Harvey’s annual recurring revenue
(ARR) jumped from $100 million in August 2025 to $190 million in March
2026 and $300 million in May, reaching $400 million by September 2026,
while its valuation climbed from $3 billion to $15.5 billion (figures
self-reported by the vendor and estimated by market trackers, not
independently audited). In terms of market penetration, Harvey has
reached 80% of the top 100 U.S. law firms and 20% of the Fortune 500,
predominantly on a per-seat subscription model (estimated by third
parties at roughly $100 to $200 per user per month for major law firms,
not official public list pricing). OpenAI has explicitly named Harvey
and Legora as launch customers for the forthcoming dedicated
gpt-6-astra-law API. While platform enhancements have
upended the rationale for startups to train their own base models, law
firm organizational division of labor, approval workflows, and
permission fabrics remain firmly controlled by vertical vendors.
Incumbent data publishing giants occupying the third tier avoided any direct hit. Thomson Reuters and LexisNexis rely on century-old citator graphs, proprietary annotated codes, and deeply embedded practice management ecosystems to form a formidable content moat that open-web indexes cannot rattle. The real tension manifests elsewhere. As a comment circulating in a Hacker News discussion noted: don’t worry, they can simply keep building their business on top of the platform. Yet vertical vendors that anchor their core deliverables entirely to foundation model endpoints, while reaping the benefits of lightweight configurations, must constantly stay vigilant against the looming threat of platform providers shifting their functional boundaries in future releases.
Figure 2: Stratified blast radius of the Astra for Law release: thin
wrappers directly absorbed, the application layer pivoting from models
to workflows, and incumbent data giants remaining insulated
Examining Astra for Law’s technical architecture yields two transferable engineering takeaways for AI engineers deploying systems in other specialized verticals.
The first engineering lesson centers on benchmark evaluation design: when assessing post-training in a vertical domain, control baselines must mandatorily include a professionally configured general foundation model. When presenting technical breakthroughs, industry R&D teams frequently make a blunt comparison between their proprietary vertical model and a raw, stock base model, attributing every ounce of performance gain to internal weight modifications. Astra for Law upends this paradigm: without touching underlying model architecture—simply by bolting on a vertical retrieval index, integrating meticulously crafted analytical instructions, and running closed-loop evaluations—the system achieved a vendor-reported 15.3 percentage point gain under an all-pass standard on a private validation set, which remains unverified by independent audits. Extrapolating from this evidence: anyone advocating for custom proprietary models or internal weight modifications who fails to benchmark against a general foundation model equipped with equally high-quality retrieval indexes and prompt engineering is almost certainly measuring inflated gains. Establishing the empirical performance ceiling of external configurations is an indispensable baseline before deciding whether to modify weights.
Engineers would do well to study the legal tech sector, as it serves as a high-pressure proving ground for testing the outer bounds of LLM capabilities. The domain offers publicly accessible corpora, steep technical barriers, and near-zero tolerance for factual inaccuracies—where citing bad law risks legal malpractice claims. At the same time, the industry boasts high margins and generous software budgets capable of absorbing costly experimentation. Over four years, the legal market completed a full cycle from pre-training vertical base models from scratch to full fine-tuning, reinforcement learning post-training, and ultimately returning to lightweight configurations, leaving behind an industry-scale ablation log. This leads directly to the second takeaway: when evaluating any vertical AI architecture, the effective order of evaluation is to first examine what the product actually looks like under the hood, and then scrutinize the comparison baselines. This logic dismantles the elitist engineering hierarchy that devalues peripheral systems: the true value of an engineering technique lies in what fraction of the difficulty distribution it actually shifts; modifying weights is merely a trade-off in resource allocation, not an inherently superior pursuit just because the code lives inside rather than outside the model.
At this juncture, two pivotal questions remain unanswered, and they will determine the true significance of this lightweight configuration approach.
The first is the independent re-evaluation currently underway by Vals AI on a held-out test set. The 54.0% figure is derived solely from vendor self-reported results on a private validation set, and the benchmark organization is currently conducting closed-door verification against an undisclosed 208-question held-out dataset. Whether a lightweight architecture assembled from a general-purpose base model, external indexes, and system instructions can replicate its private validation set performance in a blinded evaluation sealed off from potential data contamination will serve as decisive evidence of whether this specialized configuration holds up to scrutiny.
The second is OpenAI’s commercial pricing strategy for the dedicated
gpt-6-astra-law API endpoint. As this API opens to
specialized developers such as Harvey and Legora, how the platform
prices access to an interface bundling a 230-million-URL legal index and
proprietary instructions will directly shape the distribution of margin
across the vertical value chain. If the specialized API is priced close
to the base general model, the application layer will be able to
leverage ready-made infrastructure at low marginal cost, translating
that into expanded margins and richer delivery systems; if the platform
giant leverages its index scale and compliance moat to demand an
exorbitant premium, vertical application teams will inevitably
recalculate the economics of building in-house retrieval or turning to
open-source alternatives. That ultimate price tag will be the true acid
test of how much confidence the platform giant actually places in the
defensibility of peripheral configurations.