The AI landscape doesn't move in one direction — it lurches. Some techniques leap from experiment to table stakes in a single quarter; others stall against regulatory walls, technical ceilings, or organisational inertia that no amount of hype can dislodge. Knowing which is which is the hard part. The State of Play cuts through the noise with a rigorously maintained index of AI techniques across every major business domain — classified by maturity, evidenced by real-world adoption, and updated daily so you always know where you stand relative to the field. Stop guessing. Start knowing.
A daily newsletter distilling the past two weeks of movement in a domain or two — delivered to your inbox while the index updates in the background.
Each dot marks the weighted maturity of practices within a domain — hover for a brief summary, click for more detail
AI applied to streaming data for real-time pattern detection, alerting, and decision-making on live data flows. Includes stream processing with ML models and real-time anomaly detection; distinct from batch analytics which processes historical rather than live data.
Real-time streaming analytics has matured into established good practice. The discipline — applying ML models, statistical aggregations, and pattern detection to continuous data flows rather than batch windows — now rests on a stable ecosystem of GA tooling, managed cloud services, and battle-tested deployment patterns. Apache Flink and Kafka have become de facto standards; all three major cloud providers offer managed streaming services; and the industry recognizes data streaming as a formal software category. Production deployments span banking (fraud detection, real-time risk assessment), payments (sub-10ms decision latency), and operational analytics (billions of events daily). Enterprises document strong financial returns: surveys show 89% of IT leaders consider streaming platforms critical, and organisations report average ROI above 200%. The practice has transitioned past the "whether" question to operational "how"—but that transition remains incomplete. The binding constraint is not technical but organisational: the multi-disciplinary expertise in distributed systems, state management, and streaming semantics that production deployments demand. Large enterprises with dedicated data engineering teams absorb the operational complexity that managed services have reduced but not eliminated. Mid-market adoption lags, constrained not by capability but by the specialized expertise and operational maturity required to productionise these systems reliably.
Vendor consolidation has solidified around Apache Flink as the stateful processing engine and Apache Kafka as the transport layer. AWS has shifted entirely to Managed Service for Apache Flink (replacing Kinesis Data Analytics), Microsoft Fabric earned Forrester Wave leader recognition, and Flink 2.2.0 introduced ML_PREDICT and VECTOR_SEARCH — embedding LLM inference and vector similarity directly into streaming pipelines. Financial institutions (Rabobank, ING Bank, Capital One, Nationwide Building Society) now run real-time fraud detection and risk management on event-driven Flink pipelines; Riskified processes $60B in annual transaction volume with sub-10ms fraud detection decisions. Beyond finance, PostNL migrates IoT asset tracking to managed Flink, Intuit operates 200+ Kubernetes clusters handling 5B daily messages, and teams at Alibaba, Netflix, Uber, and LinkedIn maintain large-scale pipelines for recommendations and operational analytics. The ecosystem around Flink continues advancing: Kubernetes Operator 1.14.0 (March 2026) adds blue-green deployment capabilities for zero-downtime updates; research (ICDE 2026) targets latency reduction through prefetching and cache optimization; and practitioner tooling has matured with comprehensive production guides documenting exactly-once semantics, RocksDB state backends, and sub-millisecond latency architecture patterns.
Market growth reflects sustained enterprise adoption. Analyst projections cluster around 18-20% CAGR, with estimates ranging from $6.11B (Stratistics MRC, streaming segment) to $89.3B (broader streaming analytics market) by the early 2030s. Nearly half of enterprises now deploy streaming in live operational processes, up from analytics-only use cases a year prior.
Operational friction persists at the seams, however. IBM documentation from early 2026 details Kubernetes operator failures — JobManager cleanup deleting HA metadata, Java cipher suite restrictions breaking SSL handshakes — that illustrate the configuration complexity lurking beneath managed-service abstractions. Practitioners report checkpoint overhead, schema evolution failures, and write amplification in lakehouse architectures. Regulated sectors face additional headwinds: healthcare and pharmaceutical organisations find that platform speed outpaces validation frameworks like GAMP 5, creating compliance gaps. These barriers keep mid-market adoption tethered to technology-forward organisations with dedicated streaming expertise, even as large enterprises consolidate streaming as standard infrastructure.
— Comprehensive guide covering stream processing maturity (event time vs. processing time, watermarks, exactly-once for billing) with named deployments showing Flink dominance across platforms.
— Ververica technical analysis showing streaming-first architecture eliminates hours of stale-data blindspots in fraud/compliance, unifying batch-and-real-time.
— Stripe Database GA providing real-time access to payment data without webhooks/sync logic, reducing infrastructure complexity for real-time analytics.
— Flink CEP library functions as critical pre-processing layer for AI systems, detecting patterns across e-commerce, telco, and IIoT at production scale.
— Gaming architecture handles 1M concurrent players (28K–139K events/sec peak), demonstrating sub-100ms latency for anti-cheat and real-time leaderboards at production scale.
— Uber's AthenaX processes >1 trillion daily Kafka messages, accelerating production deployments from weeks to hours via SQL-compiled Flink jobs.
— Tacnode critical analysis identifying three structural failure seams in canonical Kafka→Flink→feature store stack under concurrent load and latency trade-offs.
— Systems architecture analysis documenting why production requires three specialized components; details failure modes (backpressure, schema evolution, checkpoint failures) that teams encounter when integrating real-time pipelines.