On October 8, 2026, Anthropic introduced preview versions of Claude Dashboards and Claude Motion in an official announcement. According to a Reuters report, documents, slides, and design tools were also officially launched on the same day. After users type everyday prompts into the chat box, the interface generates an interactive chart dashboard. Seeing such a demo for the first time, technical practitioners often feel that conversational data analytics has finally become practical.
Viewed over a longer horizon, this marks the second wave of conversational data analytics over the past decade. Chart rendering has long matured as a general-purpose capability; what truly makes a dashboard viable is whether the underlying aggregated numbers are accurate and trustworthy.
Whether an AI dashboard is trustworthy depends on who it entrusts that trust to. One approach locks down business definitions upfront and has the model compute strictly against them; the other lays out the query process and leaves verification to the user. The former delegates responsibility to those who define metrics, while the latter pushes the burden of verification back onto whoever reads the dashboard.
Natural language to SQL is not unique to large language models. The WikiSQL and Seq2SQL paper was published back in 2017, and the Spider benchmark paper in 2018 showed that the strongest model at the time achieved an exact-match accuracy of just 12.4% across unseen databases, illustrating how difficult it is for machines to understand unfamiliar tables.
With the rise of large language models, this problem only became more pronounced. The 2023 BIRD benchmark paper showed that ChatGPT achieved an execution accuracy of only 40.08% on this benchmark, compared to 92.96% for humans. By November 2024, the Spider 2.0 benchmark paper replaced test cases with complex, real-world enterprise databases (a significantly harder upgrade from the 2018 Spider). The results revealed that even the cutting-edge o1-preview agent at the time achieved an execution accuracy of just 21.3%, whereas that same model scored 91.2% on the original Spider. As soon as the problems switched to real enterprises, the illusion vanished.
Even the yardsticks used to evaluate them proved unreliable. An evaluation analysis in VLDB published in 2026 pointed out that annotation error rates in BIRD and Spider 2.0-Snow reached 52.8% and 66.1%, respectively. With the benchmarks themselves riddled with errors, chasing high scores on leaderboards is completely detached from whether these models actually hold up in enterprise production.
Commercial software faced the same dilemma. Tableau launched Ask Data in 2019 to support natural language queries (Tableau product retrospective), and ThoughtSpot announced the integration of GPT-3 to launch Sage in a 2023 press release. Yet everyone quickly ran into the same wall: tools merely translated prompts into SQL, while the calculated numbers clashed with business definitions and contradicted one another.
AtScale, a semantic layer vendor, later reviewed this period (with obvious commercial interest): first-generation natural language queries only translated user prompts into SQL without understanding business rules, leading to conflicting definitions that nobody trusted and causing the entire space to visibly recede between 2021 and 2023 (AtScale retrospective). The bottleneck was never writing code; it was metric definitions and trust.
Since older approaches kept stumbling over the past decade, why did conversational analytics erupt once again in the second half of 2026? Ultimately, four foundational technologies only recently closed their engineering loops.
First, models acquired the closed loop of autonomously writing and executing code. On July 6, 2023, OpenAI introduced Code Interpreter in the ChatGPT release notes, equipping the model with a sandboxed environment to run code and verify values, finally granting it the ability to retrieve data and compute numbers independently.
Second, outputs transformed into independently interactive artifacts. Anthropic introduced its dedicated side-panel rendering mechanism in the Artifacts announcement on June 21, 2024, followed quickly by direct tabular data processing in the analysis tool overview on October 24.
Third, enterprise data connectivity protocols converged toward standardization. To allow models unified access to data warehouses, CRMs, and SaaS tools, Anthropic released the MCP open source announcement on November 25, 2024, introducing the Model Context Protocol (MCP) to establish a standardized, secure pipeline.
Finally, front-end product surfaces and collaboration mechanics merged. Anthropic opened the Cowork preview in January 2026, published an overview of custom visuals on March 12, launched Claude Design on April 17, and fully integrated these collaboration modalities via a blog post on September 16, with users creating over 45 million documents, slides, and designs to date.
Competitors maintained an equally aggressive cadence: OpenAI introduced Data agent in its data agent announcement on September 10, 2026—four weeks ahead of Anthropic—and unveiled streaming component compilers in the GPT-6 release notes on October 7. Anthropic’s official rollout of Dashboards and Motion on October 8 reflects both technological maturity and competitive pressure. (Official social post)
Once real-time conversational dashboard generation crossed the usability threshold, different companies adopted divergent approaches to ensure data authenticity.
One path sets the rules upfront. The same term, such as “sales revenue,” can be calculated differently by different people (Do refunds count against it? Are unpaid orders included? Is foreign exchange calculated on the order date or the shipping date?). Therefore, a layer is added on top of the database to strictly standardize business rules and definitions, leaving the model solely responsible for querying against those definitions. This centralized repository for managing definitions is known as the semantic layer.
This design philosophy places trust in the selection of definitions and metrics. Accordingly, OpenAI’s Data agent adopted a similar strategy by leveraging an enterprise’s existing definitions and metrics, guaranteeing only that it can explain which pieces of evidence substantiate each number rather than promising to display SQL statements. Other data platforms and BI vendors widely follow this path: Microsoft integrated governed semantic models into Copilot, while Databricks explicitly noted that without providing business context to the model, its “data agents” perform extremely poorly—to the point of being unacceptable.
The other direction relies on laying out the process, exemplified by Anthropic’s Claude Dashboards. It attaches an individual SQL query to each visual; clicking any number on the chart expands that query, and a corner timestamp displays when the data was last refreshed (getting started guide). According to the official documentation, dashboards are private by default, and when shared, rerun under each recipient’s permissions so they only see data they are already authorized to access—though this permission model has yet to undergo third-party penetration testing (The New Stack citing official statements). The accompanying animation tool Motion similarly uses code to drive visuals rather than generating video files, making modifications straightforward (Motion getting started guide).
Google sits in the middle. Before returning an answer, its conversational analytics displays both the reasoning steps and the exact generated SQL, cites the sources for each conclusion, routes queries through its own governed modeling layer, and logs every query into an audit trail (BigQuery announcement). It establishes rules while preserving the paper trail.
Anthropic has taken a more understated stance. It explicitly positions Dashboards for rapid exploration and ad hoc queries rather than replacing enterprise BI, offering one-click handoffs to analytics platforms such as Grafana, Hex, and Sigma (with Looker and Tableau planned for the future) (official announcement). Sigma announced its corresponding integration on the same day (Sigma partnership statement), putting this division of labor into practice.
Laying out the query so users can click to see how a number was computed is undoubtedly an improvement over the black boxes of several years ago. Yet it only solves whether you can inspect it, not whether you should trust it. Shoving raw SQL directly in front of business users pushes the burden of finding bugs right back onto them: they turned to the tool precisely to avoid reading code, only to be asked whether the query logic holds up (Omni’s critique). If every number still requires manual verification to confirm its accuracy, what time has the tool actually saved them?
Even more problematic is that because large language models generate SQL unconstrained, the same question can yield different answers each time. dbt conducted a controlled experiment: when the same model wrote SQL directly, accuracy hovered between 84% and 90%; but once queries were standardized against a shared definition, accuracy reached 98% to 100%, with virtually zero errors (dbt’s evaluation). Since dbt sells semantic layer solutions, the figures warrant some discount, but they demonstrate a fundamental point: what drifts is not the model, but the definitions.
The closer you get to real-world enterprises, the starker the contrast becomes. On academic benchmarks, models achieve 80% to 90% accuracy; once dropped into uncleaned enterprise databases, raw SQL generation drops to just 10% to 30%, with complex multi-table joins sometimes failing entirely (colrows’ cliff analysis, promethium’s benchmark guide). These numbers also come from companies selling semantic layers and deserve scrutiny, yet they match the lived experience of front-line engineers: the more confident the model sounds, the harder it is for anyone to catch the mistake.
Yet the industry has already voted with its feet. Gartner listed the semantic layer as critical infrastructure alongside data platforms and cybersecurity for 2030, warning bluntly that ungoverned models directly informing business decisions will cause financial or reputational damage in the near term (Gartner trend predictions); Forrester went so far as to define the semantic layer as the control plane for the entire data application ecosystem (Forrester perspectives). What analyst firms value is not the chat box.
In retrospect, while this race superficially appears to be about who can generate charts faster with AI, it is fundamentally a competition over who stands behind the correctness of the numbers. The rule-setting camp passes responsibility to those defining metrics; the process-transparency camp pushes responsibility back to whoever reads the dashboard. Neither path is cheap: the former requires someone to build out metric definitions upfront, while the latter demands someone capable of reading SQL. Which side you choose depends on the weight of the numbers in your hands. If you are merely looking for inspiration or exploring initial directions, either works; but if you need to present to leadership or allocate budgets, you must first ask: if this number is wrong, who takes the fall?
Whether Claude Dashboards has addressed semantic layers, row- and column-level permissions, auditing, or scheduled refreshes remains unmentioned in official public materials and is still an open question. As for accuracy, the two frontrunners are on equal footing: neither Claude Dashboards nor OpenAI’s Data agent has released comparable benchmark figures. As for who can retain the flexibility of spontaneous queries while tackling the hard problem of metric definitions, the verdict is not yet in. But one thing is already clear: generating charts has long ceased to be valuable; what is truly scarce is the link willing to take responsibility for the numbers.