Where AI Stands in Data & Analytics
The defining fact about AI in data and analytics in mid-2026 is that the technology works and the organisations don't. Across nearly every practice we track — from natural-language querying to time-series forecasting, synthetic data to model monitoring — the tooling has reached production grade, vendors have consolidated around mature platforms, and forward-leaning enterprises are extracting documented returns. Yet the dominant trajectory is stall, not advance. The question has shifted, almost universally, from "does this work?" to "can we absorb it?" — and the answer, for most companies, is not yet. The binding constraint is no longer model capability. It is data governance, data quality, and the organisational discipline required to turn capable tools into trustworthy production systems.
The numbers behind this stall hardened sharply over the past month. A Gartner survey of 782 infrastructure-and-operations leaders found a 72% failure-or-underperformance rate for enterprise AI projects, rooting the cause in skill gaps, data quality, and unrealistic expectations rather than model quality. MIT's NANDA study documented that 95% of generative-AI pilots delivered zero measurable profit-and-loss impact. Dun & Bradstreet surveyed 10,000 businesses and found 97% running AI initiatives but only 5% with data they consider AI-ready. Gartner separately projects that 60% of AI initiatives will be abandoned through 2026 for inadequate data foundations, and that 40% of agentic projects — software that acts autonomously rather than on prompt — will be cancelled by 2027 on cost and governance grounds. These are not isolated cautionary tales; they are the macro signal of the domain. Capability has outrun absorption, and the gap is widening.
What distinguishes this domain is that the diagnosis is now precise. This is not vague AI scepticism. The failure modes have been isolated, benchmarked, and named: undocumented schemas that collapse text-to-SQL accuracy, metric definitions that diverge across functions, lineage that breaks at the join layer, governance frameworks that cannot keep pace with deployment. The most striking demonstration came in natural-language querying, where independent testing shows leading models scoring 82% on academic benchmarks but 10.8% on real corporate data — while deterministic semantic layers hit 98-100% on identical questions. The lesson, repeated across practices, is that context assembly and data engineering, not raw intelligence, determine whether AI delivers. The firms pulling ahead are those that spend the majority of their AI budget on data foundations rather than on the visible analytics layer. Everyone else is buying capable tools and discovering they cannot trust the output.
What's New, 2026-05-18 to 2026-06-17
The headline movement this cycle is the consolidation of an ROI reckoning into hard, citable evidence — and a single trend reversal that signals maturation. Real-time streaming analytics moved from advancing to plateau, reflecting a genuine market correction: vendors including Databricks and MotherDuck are now explicitly steering customers toward micro-batching and warehouse-native ingestion when sub-second latency is not a real business requirement, conceding that continuous streaming's operational overhead — roughly 70% engineering complexity, 30% infrastructure — is justified only at the latency extremes. Goldsky's replacement of Apache Flink with a Rust-based engine for a 30x compute reduction and over $1M in annual savings across 3,000-plus pipelines crystallised the point. This is the sound of a practice growing up: the debate has shifted from "whether to stream" to "when streaming is worth its cost."
Elsewhere, the picture is stability — which is itself the signal. The wall of failure-rate research (Gartner's 72%, MIT NANDA's 95%, the 5% data-ready figure) landed alongside two newer structural concerns: vendor lock-in and agent reliability. A market analysis put the sovereign-AI market at $19.2B growing 28% annually, with 94% of enterprises worried about lock-in and 47% unable to switch primary vendor without halting key operations; a companion piece catalogued four AI-specific lock-in types (model, context, workflow, pricing) with mid-migration switching costs of $50K-$120K. On reliability, a Sinch survey of 2,527 decision-makers found a 74% rollback rate for deployed AI agents, with named production disasters — an AI coding agent deleting a production database despite explicit preservation instructions — underscoring that operational controls lag deployment. The one clear bright spot remains MLOps, the domain's sole advancing practice, where MLflow's 57% adoption and managed cloud monitoring are turning model observability into settled discipline, even as an MLflow remote-code-execution vulnerability (CVSS 9.6) was a reminder that the tracking infrastructure itself is now an attack surface.
Key Tensions
Capability has decoupled from absorption, and the diagnosis is now precise. The failure is not technical. Gartner reports a 72% AI-project failure rate and MIT NANDA finds 95% of GenAI pilots deliver zero P&L impact, with root causes consistently traced to data quality, governance, and unrealistic expectations rather than model performance. The firms that succeed allocate the majority of AI spend to data foundations; the rest buy capable tools and cannot trust the output.
Context, not model intelligence, is the limiting factor — and this is now provable. Natural-language querying offers the cleanest demonstration: leading models score 82% on academic benchmarks but collapse to 10.8% on real enterprise data with undocumented schemas, while deterministic semantic layers achieve 98-100% on the same questions. Snowflake's own benchmark shows agents at roughly 24% accuracy in isolation versus 86% with assembled business context. The implication runs through the whole domain: the work is data engineering, not prompt engineering.
The autonomy gap is real and quantified. Agentic AI is the loudest theme in the evidence and the least production-ready. A Sinch survey found a 74% rollback rate for deployed agents; Gartner projects 40% of agentic projects cancelled by 2027; and named incidents — an AI agent deleting a production database despite explicit instructions — show that probabilistic models are being placed in deterministic data operations without adequate safety gates. Governance and operational controls are lagging deployment, not leading it.
Maturity now means knowing when not to use the sophisticated tool. Two practices made this explicit this cycle. Real-time streaming shifted to plateau as vendors steer customers to cheaper batch architectures unless sub-second latency genuinely matters. In time-series forecasting, foundation models (TimesFM, Chronos) promise zero-shot generality, yet lightweight classical methods still beat them by 4-21% on specialised tasks with full data. Pragmatic boundary-setting — domain-specific over generic, batch over streaming, classical over neural — is the mark of a maturing field.
Vendor lock-in has become a board-level data risk. As platforms consolidate (Snowflake, Databricks, the cloud hyperscalers) and embed AI deeper into the data stack, switching costs are compounding. A market analysis found 94% of enterprises concerned about lock-in and 47% unable to change primary vendor without stopping core operations, with four distinct AI-specific lock-in types now identified. The same consolidation that delivers mature, integrated tooling is quietly raising the cost of ever leaving it.
Top 10 Evidence Items
The Text-to-SQL Accuracy Cliff: 91% on Benchmarks, 21% in Production (opinion) — The single clearest proof of the summary's central claim that context, not model intelligence, is the limiting factor: a documented 70-point accuracy collapse from academic benchmarks to real enterprise data, with undocumented schemas as the proximate cause. https://colrows.com/blogs/text-to-sql-accuracy-cliff/
72% of Enterprise AI Projects Fail: Gartner's ROI Reality Check (industry-report) — The hardest number in the domain narrative — Gartner's survey of 782 I&O leaders attributing failure to skill gaps, data quality, and unrealistic expectations rather than model quality — which reframes the entire domain as an absorption problem, not a technology problem. https://www.beri.net/article/72-percent-enterprise-ai-projects-fail-gartner-roi-reality-check
AI ROI in 2026: Why 95% of Pilots Fail to Deliver and How to Measure What Actually Matters (opinion) — Covers the MIT NANDA finding that 95% of generative-AI pilots deliver zero measurable P&L impact, directly substantiating the summary's macro signal that capability has outrun absorption at scale. https://www.ellvero.com/insights/ai-roi-in-2026-why-95-percent-of-pilots-fail-and-how-to-measure-what-matters
Deterministic vs Probabilistic Text-to-SQL: Why Reproducibility Is Becoming Table Stakes (opinion) — The dbt Labs benchmark showing semantic layers at 98.2-100% versus probabilistic text-to-SQL at 84-90% on identical insurance queries is the architectural answer to the accuracy-cliff problem, and the clearest illustration of why data engineering investment beats prompt engineering. https://colrows.com/blogs/deterministic-vs-probabilistic-text-to-sql/
Open-sourcing Streamling: a stream processing runtime for app teams (case-study) — Goldsky replacing Apache Flink with a Rust-based engine to achieve 30x compute reduction and over $1M/year savings across 3,000+ pipelines crystallises the streaming-analytics maturation story: the practice is growing up by learning when Flink's complexity is unjustified. https://goldsky.com/blog/open-sourcing-streamling
Why 74% of Firms Rolled Back AI Customer Agents (case-study) — Sinch survey of 2,527 decision-makers documenting a 74% agent rollback rate substantiates the summary's autonomy-gap tension with a directly comparable statistic to Gartner's project-failure rate, showing that agentic deployment is failing faster than prior AI waves. https://entropyand.co/blog/why-companies-are-rolling-back-ai-agents
The New Insider Threat: How to Stop AI Agents From Nuking Your Database (opinion) — Named production incidents (Replit, Vibe) where AI agents destroyed databases despite explicit preservation instructions are the uncomfortable empirical anchor for the summary's claim that probabilistic models are being placed in deterministic data operations without adequate safety gates. https://www.liquibase.com/blog/the-new-insider-threat-how-to-stop-ai-agents-from-nuking-your-database
Snowflake Summit 2026 Recap: Your Data Is the Real AI Moat (industry-report) — Cortex Sense benchmark showing agents at ~24% accuracy in isolation versus ~86% with assembled business context is the clearest single data point proving that context assembly, not model capability, determines whether enterprise AI delivers. https://www.alation.com/blog/snowflake-summit-2026-takeaways/
Sovereign AI and Resilience: Vendor Lock-In Guide (2026) (industry-report) — Documents the $19.2B sovereign-AI market growing at 28% annually alongside 94% enterprise lock-in concern and four AI-specific lock-in types with $50K-$120K mid-migration switching costs, grounding the summary's board-level-risk tension in market-scale numbers. https://aibuzz.blog/sovereign-ai-resilience/
What is Data Ingestion Pipeline? The Warehouse-Native Shift (opinion) — MotherDuck's explicit 2026 guidance steering customers away from continuous streaming toward micro-batching and warehouse-native architectures for cost and complexity reasons is the vendor-side confirmation that real-time streaming analytics has reached plateau — pragmatic constraint-setting as maturity signal. https://motherduck.com/learn/what-is-data-ingestion-pipeline/