📊 Data & Analytics
AI for turning raw data into queryable, analysable, actionable insight. Streaming analytics, MLOps, and feature engineering are good practice with proven deployments at scale. The bulk sits at leading-edge, held back not by tooling but by data quality and governance gaps — 60% of AI projects stall on data readiness. Nearly all practices are stalled in trajectory.
The Headline
AI agents now repair broken data pipelines in production, but only with a person approving each fix. The bigger risk is quieter: AI-built analysis that looks right and isn't.
The Picture
The foundations of data work are mature and increasingly come built into the big platforms. That covers streaming, quality monitoring, lineage and experiment tracking, and most large companies already have them or are getting them as standard features from Databricks and Snowflake. The layer executives hear most about is different: AI agents (software that acts on its own without being prompted) that answer questions, build dashboards and write commentary. These are widely available but thinly used. 451 Research finds 36 percent of organizations using assistive analytics agents, and describes most "agentic" products as rebranded features that still run with a person in charge. The small group getting real results, including Fanatics, Atlassian and S&P Global Energy, first agreed shared, governed definitions of their business metrics. If you haven't done that, you are in the pack, and buying more agent tooling will not move you ahead.
This Fortnight
Snowflake reports five months of running an AI agent that repairs its own broken data pipelines, with a person approving every fix. The agent investigates failures and proposes a code fix in under 16 minutes on average, and hundreds of its fixes have been merged. Snowflake gives no baseline. Its own summary is that "a person still decides what ships". That is the credible model today: much faster diagnosis, with sign-off kept in human hands.
Monte Carlo launched Bayes, an agent that makes and verifies pipeline fixes on its own, with the customer setting how much autonomy it gets. The vendor says an unnamed South American financial institution caught 200 percent more incidents with it, but gives no methodology. Analyst firm ISG notes that a rival's autonomous incident-manager agent has not shipped yet. Treat claims of unattended repair as unproven until a vendor shows audited results on data like yours.
An independent audit found that most AI-built machine-learning pipelines contain at least one hidden methodology error. Researchers at CWI Amsterdam reviewed 220 tasks and found that data leakage, where test answers seep into training, went hand in hand with inflated scores. When an agent hands you an unusually good model, the strong score itself is a reason to have someone check how it was built.
A practitioner testing Databricks' tool for migrating Power BI reports found it silently swapped a year-over-year calculation for a flat number. Simple measures transferred correctly, but time-based logic needed fixing by hand, and nothing flagged the error. If AI is carrying your reporting migration, have someone verify every period-comparison metric before the old system is switched off.
Databricks made its real-time mode for Spark generally available and claims it runs up to 92 percent faster than Flink, the incumbent streaming engine. Databricks ran the benchmark itself and published no method. Zeta Global's AWS case study is closer to how most companies actually work: data refreshed about every 15 minutes, with a separate fast path only where a decision needs it. Ask vendors to prove speed claims on your own workload, and ask whether you need sub-second data at all.
Coming Up
Meta is buying Sundial and will end the service for external customers once the deal closes, with no closing date or export path published. Sundial let business users question a data warehouse in plain English, and the definitions customers built inside it may not be portable. Data-quality tools are consolidating the same way: Metaplane went to Datadog, Synq to Coalesce and Elementary to Boomi. Before signing with a standalone tool, write into the contract that you can export your metric definitions and history.
Under the EU AI Act, post-market monitoring duties for standalone high-risk systems such as credit scoring have moved from August 2026 to December 2027. The delay buys time but does not remove the duty. AWS has also closed SageMaker Model Monitor to new customers. Use the window to decide who owns production model performance and which tool will monitor it, so that a deadline does not make the choice for you.
Vendors are racing to sell agents that fix data problems without asking first. The deployments with the most credible numbers, at Snowflake and Lennar, keep a person approving changes. Within the next year, set an explicit policy on what any agent may change unreviewed in finance or customer data, before a vendor's default settings decide for you.
What's Hard About This
Errors in this domain are quiet: a wrong number looks exactly like a right one. One agency's automated reporting agent overstated return on ad spend by nearly 30 percent for about a month after a data-feed change, and a client caught it before the agency did. A single test run does not catch this kind of failure. Only staged validation, with figures computed outside the AI, does.
Agents are only as reliable as the business definitions underneath them, and someone has to write those by hand. Before Fanatics could trust its agents, it had to reconcile three competing definitions of Gross Gaming Revenue spread across about 200 Tableau dashboards. That is organizational work about ownership, and no tool does it for you.
The sophisticated method often fails to beat the simple one. A controlled test found that elaborate agentic forecasting setups beat the best single forecasting model by less than 6 percent. Separate studies found simple models matching or beating complex ones in transit demand and model tuning. Require every AI analytics proposal to show its gain over a plain baseline, or you will pay for complexity you cannot show you need.
Practices in this Domain (16)
| PRACTICE | TIER | TREND |
|---|---|---|
| Anomaly & outlier detection | BLEEDING EDGE | — Steady |
| Automated exploratory data analysis | LEADING EDGE | — Steady |
| Causal inference & uplift modelling | LEADING EDGE | — Steady |
| Data catalogue, metadata & lineage management | GOOD PRACTICE | — Steady |
| Data pipeline, dashboard & report generation | LEADING EDGE | — Steady |
| Data privacy & anonymisation automation | LEADING EDGE | — Steady |
| Data quality, cleaning & transformation automation | GOOD PRACTICE | — Steady |
| Feature engineering, AutoML & predictive modelling | GOOD PRACTICE | — Steady |
| Geospatial data analysis & visualisation | LEADING EDGE | — Steady |
| Graph analytics & relationship discovery | LEADING EDGE | — Steady |
| MLOps — experiment tracking & model monitoring | GOOD PRACTICE | — Steady |
| Narrative generation from data | LEADING EDGE | — Steady |
| Natural language data querying & semantic search | LEADING EDGE | — Steady |
| Real-time streaming analytics | GOOD PRACTICE | — Steady |
| Synthetic data generation | LEADING EDGE | — Steady |
| Time series forecasting | LEADING EDGE | — Steady |
Read the full technical briefing (1,678 words) →
Where AI Stands in Data & Analytics
Data and analytics is the domain where AI's progress is easiest to mistake for completion. The foundations are mature, running in production and increasingly dull. Kafka and Flink carry fraud scoring at Adyen and analytics at Robinhood. MLflow reports more than 60 million monthly downloads, and Kubeflow graduated from the CNCF in August. Uber's UDQ platform generates standard quality tests automatically across more than 2,000 critical datasets, and banks such as HSBC model attribute-level lineage because regulators require it. The platforms are absorbing all of this. Databricks and Snowflake now ship quality monitoring, lineage, streaming and natural-language querying as native features, and Databricks this cycle made Real-Time Mode for Spark generally available as a direct challenge to Flink. For this layer, the team that does without it is the one that has to justify the choice.
The layer most people mean by "AI for analytics" is a different story: agents that query, explore, build dashboards and write the commentary. The tooling is everywhere, including OpenAI's Data agent in ChatGPT Work, Claude Dashboards in beta, Fabric IQ, Databricks Genie and Snowflake Cortex Agents. Use is much thinner than availability. 451 Research finds 36% of organisations using assistive BI and analytics agents, and calls current agentic implementations "largely rebranded features in vendor platforms used with human agency". The deployments that work, at Fanatics, Webmotors, Atlassian and S&P Global Energy among others, run on curated semantic layers and governed metric definitions that the customer built first. Before agents could be trusted at Fanatics, the company had to reconcile three competing definitions of Gross Gaming Revenue spread across about 200 Tableau dashboards. The wider readiness picture is bleak. Gartner predicts organisations will abandon 60% of AI projects through 2026 that lack AI-ready data. Teradata reports that 77% of enterprises have data unready for agentic AI, and BARC finds that only 29% of 225 enterprises can fully locate the data relevant to AI. The binding constraint is data and ownership, not models.
Two things set this domain apart. First, its errors are quiet. A wrong revenue figure looks like any other revenue figure, a pipeline that passes its test looks healthy, and a drift alert looks urgent whether or not anything has drifted. ArrivalBench put numbers on this: grading each pipeline on a single run certified 86–100% of agent-generated pipelines as correct, yet replaying them with late, duplicated and retried records found 7.0–79.2% of the certified ones silently wrong. Second, simple baselines keep winning. In forecasting, demand modelling and hyperparameter tuning, the elaborate method regularly fails to beat the plain one. So the frontier practices (time-series foundation models, text-to-SQL, synthetic data, narrative generation) are all available off the shelf, but none has a default configuration a competent team can adopt and trust. Each deployment is still a bespoke engineering commitment, and the adopter has to build its validation.
What's New, 2026-10-21 to 2026-11-04
Autonomous pipeline repair reached real production this fortnight, but with a person still at the gate. Snowflake's internal data team reports five months of an agent that investigates monitored pipeline failures and opens fix PRs in under 16 minutes on average, with hundreds merged, and states that "a person still decides what ships". Lennar runs more than 5,000 daily rules and anomaly checks over 1,500+ tables on Ataccama, with a median under 15 minutes to detect and acknowledge an issue. dbt Labs says design partners using dbt Wizard saw 15–20% fewer production incidents, though it gives no baseline. Monte Carlo went further with Bayes, which implements and verifies fixes on its own, and claims that an unnamed South American financial institution caught 200% more incidents with 80% fewer false positives, with no methodology. ISG reports that Anomalo's autonomous incident-manager agent is still among the four of its nine agents not yet shipped. In streaming, Databricks backed Real-Time Mode with a self-run benchmark claiming up to 92% faster than Flink, published without method. Zeta Global's AWS case study is more typical of real practice: roughly 15-minute lakehouse freshness, a separate path for anything needing sub-minute latency, and compute costs cut by 50%.
Most of the other new evidence cut against agentic claims. An independent CWI audit of 220 tasks found that most pipelines built by ReAct-, AIDE- and MLEvolve-based agents contain at least one silent methodological failure, with leakage correlating with inflated scores. A controlled protocol found the best agentic forecasting orchestrators beat the best single foundation model by less than 6% on MASE. A preregistered audit found covariate-aware forecasters under-respond to real interventions by 2–5×, and a leakage-free test found Chronos-2 and TimesFM-2.5 mostly failing to beat a random walk on carbon prices. The counterpoint came from air quality: across 29 cities, zero-shot models matched a tuned LightGBM on PM2.5 using roughly a tenth of the energy. In natural-language querying, learning schema from query logs reached 95.7% execution validity against 60.8% for embedding retrieval, in the authors' own evaluation. A clinical study found that physicians' freely composed questions needed clarification 24.8% of the time, against 1.8% for predefined ones. Meta's purchase of Sundial will end the service for external customers, and no export path has been published. A practitioner found Genie Code's Power BI import silently replacing a time-intelligence measure with a flat number. One monitoring platform retracted all 446 drift incidents its canary rule had opened. Protegrity's PII model scored 0.906 F1 on its own test set and 0.467 on SPY. ANYbotics raised anomaly recall from 0.50 to 0.62 with synthetic clips but got more false alarms. The overall standing of the practices did not move.
Key Tensions
Certified correct, silently wrong. The domain's characteristic failure throws no error. An agency's reporting agent overstated ROAS by nearly 30% for about a month after an API schema change before a client noticed. A Checkr LLM wired straight to Snowflake gave different incorrect answers to the same question. Agent-built ML pipelines in the CWI audit leaked data in ways that flattered their scores. Single-run testing, the check most teams use, is the check that misses these errors.
The semantic layer is the real product. Every reliable natural-language, dashboard and narrative deployment depends on definitions someone curated by hand. Uber's QueryGPT lost accuracy as tables were added and needed curated workspaces and confirmation agents. Solid tells customers to scope each model to 5–10 tables and treat 85% accuracy as the floor. Alation warns that indiscriminate cataloguing creates a 'context swamp', and cites Forrester's forecast that only one in four context-layer efforts will reach production scale in 2027.
Sophistication versus the baseline. The extra machinery often buys little. Random Forest beat graph convolutional models on transit demand across 251 Kalamazoo County zones (R2 = 0.915). Plain Optuna beat LLM-advised tuning in all four of one small test's runs. No model significantly beat a random walk on EU ETS carbon prices. Teams that skip the baseline comparison pay for complexity they cannot show they need.
Autonomy claimed faster than it is measured. The boldest claims this cycle came from vendors grading their own work. Examples include Databricks' 92% figure without a method, Monte Carlo's customer numbers with no methodology, and PII models that score far higher on their makers' test sets than on public ones. The deployments with the most credible numbers, Snowflake's repair agent and Lennar's checks, keep a person approving each change, and Databricks' own guidance says auto-approve is not a security boundary.
Platforms absorb, point tools vanish. Standalone tools are being folded into platforms or closed down, and their users bear the cost. Meta's Sundial deal ends external service. Neptune's hosted tracker shut in March with its data deleted. Metaplane, Synq and Elementary have been acquired. AWS says SageMaker Model Monitor is no longer open to new customers. Choosing a tool now means weighing who will own it in two years as well as what it does.
Top 10 Evidence Items
- The Silent Failures in Agentic Data Science (research-paper) — The CWI audit is the sharpest independent evidence for the briefing's 'certified correct, silently wrong' failure mode in agentic AutoML. https://arxiv.org/pdf/2610.14115
- Do Time Series Foundation Models Forecast Carbon Prices? Leakage-Free Evidence from the EU ETS and China’s Carbon Markets (research-paper) — Directly backs the 'sophistication versus the baseline' tension cited in the briefing: no TSFM beats a random walk on carbon prices. https://www.mdpi.com/1996-1073/19/20/4999
- Advanced Demand Prediction for Demand-Responsive Transit (DRT) Systems: Integrating Graph Convolutional Networks and Deep Learning Techniques (research-paper) — The Kalamazoo result is the briefing's lead example of elaborate models losing to a plain baseline on real geographic data. https://onlinelibrary.wiley.com/doi/10.1002/eng2.71118
- Secloudis – AI, Data & Cloud – Engineered in Harmony (opinion) — A small but clean negative result showing LLM-guided tuning adds nothing over plain Optuna, reinforcing the baseline tension with a different technique. https://secloudis.com/agentic-automl-with-qwen-on-okd/
- How Fanatics turned data governance into a foundation for AI at scale | dbt Labs (case-study) — Shows the unglamorous precondition for Fanatics' agent success: reconciling conflicting metric definitions before agents could be trusted. https://courses.getdbt.com/case-studies/fanatics
- QueryGPT - Natural Language to SQL using Generative AI (case-study) — Uber's own experience that retrieval degrades with scale is the clearest illustration that the semantic layer, not the agent, is the real product. https://www.uber.com/tr/en/blog/query-gpt/
- How Snowflake Built an Agent to Fix Broken Data Pipelines (case-study) — The most credible autonomy claim this cycle, and notably still gated by a human who decides what ships. https://www.snowflake.com/en/blog/ai-agent-fixes-data-pipeline-failures/
- Monte Carlo Data launches Bayes, an AI agent for autonomously monitoring and fixing autonomous agents (news-coverage) — Contrasts with Snowflake's measured rollout: Bayes' numbers come from the vendor itself with no stated methodology. https://siliconangle.com/2026/10/28/monte-carlo-data-launches-bayes-an-ai-agent-for-autonomously-monitoring-and-fixing-autonomous-agents/
- Meta Acquires Sundial to Build Analytics Agents for Its Own Use (news-coverage) — Concrete case of the 'platforms absorb, point tools vanish' risk, with no transition path for Sundial's existing customers. https://www.beri.net/article/meta-sundial-acquisition-ai-analytics-agents-external-customers-shutdown-warehouse-credentials-migration
- Can Claude or ChatGPT Replace Your BI Tool? - Omni Analytics (opinion) — A real production failure mode for ungoverned natural-language querying, echoing the domain's theme of quiet, inconsistent errors. https://omni.co/articles/can-claude-or-chatgpt-replace-your-bi-tool