The AI landscape doesn't move in one direction — it lurches. Some techniques leap from experiment to table stakes in a single quarter; others stall against regulatory walls, technical ceilings, or organisational inertia that no amount of hype can dislodge. Knowing which is which is the hard part. The State of Play cuts through the noise with a rigorously maintained index of AI techniques across every major business domain — classified by maturity, evidenced by real-world adoption, and updated daily so you always know where you stand relative to the field. Stop guessing. Start knowing.
A daily newsletter distilling the past two weeks of movement in a domain or two — delivered to your inbox while the index updates in the background.
Each dot marks the weighted maturity of practices within a domain — hover for a brief summary, click for more detail
AI-assisted tracking of ML experiments and monitoring of deployed models for drift and degradation. Includes experiment comparison and automated drift detection; distinct from AI Governance model evaluation which assesses safety and fairness rather than operational performance.
Experiment tracking and model monitoring have crossed into proven, accessible territory. MLflow commands 57% adoption and 30 million monthly downloads; all three major cloud vendors offer fully managed deployments; and enterprises report measurable gains in deployment speed and model reliability. The tooling question is settled — the rollout question is not. Tracking experiments is now straightforward, but monitoring deployed models for drift and degradation remains the harder discipline. An estimated 87% of models still fail to reach production, and that gap points squarely at monitoring rather than tracking. Teams scaling from dozens to hundreds of production models face a strategic choice: managed platforms from Databricks, Azure, or SageMaker trade operational simplicity for lock-in, while self-hosted MLflow and Kubeflow preserve flexibility at the cost of integration overhead. The practice is mature; the challenge is organizational execution at scale.
MLflow consolidates dominance as the production standard, with 30+ million monthly downloads, 20K+ GitHub stars, and 900+ contributors. Databricks continues platform expansion: MLflow 3.0 GA introduced deployment orchestration workflows (job triggers on model registration), and April 2026 brought critical GA feature for storing MLflow traces in Unity Catalog as native SQL-queryable tables—enabling unlimited trace retention and cost-controlled analysis at enterprise scale. Market confidence is quantified: the MLOps market reached $1.115 billion in 2025 with 41.3% projected CAGR through 2031. AWS SageMaker, Azure ML, and Databricks each offer fully managed MLflow hosting; Kubeflow retains deployment presence at organizations (Samsung SDS, IBM, Coupang) requiring Kubernetes-native orchestration but faces adoption friction from learning curves and documentation drift.
Monitoring remains the constraining bottleneck despite mature tracking tooling. Production evidence shows the gap starkly: 91% of ML models degrade over time without active monitoring; 75% of deployments experience performance decline without retraining; 87% of models fail to reach production altogether. Only one-third of organizations have risk mitigation controls in production workflows, leaving two-thirds operating without detection of data drift, concept drift, or silent model failures. Security maturity remains incomplete: MLflow 3.8.0 carries CVE-2025-15379 (CVSS 10.0 RCE via poisoned artifacts), signaling that production deployments require rigorous validation. Platform integration gaps persist—Microsoft Fabric exposed MLflow API limitations, Azure ML remains incompatible with MLflow 2.8+ features. The core tension sharpens: experiment tracking is commodified and consolidating around MLflow, while monitoring methodology fragments across statistical approaches (KS test, KL divergence), proprietary platforms (Arize, Fiddler), and open-source tools (Evidently, WhyLabs). Increasingly, teams unbundle MLOps stacks—pairing MLflow with specialized drift detection rather than seeking a single all-in-one platform.
— AWS SageMaker GA support for MLflow v3.10 with pre-built performance dashboards (latency, throughput, quality scores), mlflow.genai.evaluation() API for LLM quality, trivial provisioning via Studio console.
— Uber Michelangelo platform deploys 400 active ML use cases, 20K training jobs/month, 15M predictions/sec. Shadow testing on 75% of critical models with auto-rollback on performance breach.
— Microsoft Fabric MLflow native integration enables cross-workspace experiment tracking with synapseml-mlflow plugin, consolidating ML assets across Databricks, Azure ML, and on-prem environments into unified MLOps platform.
— End-to-end production MLOps on Kubernetes: MLflow experiment tracking, PostgreSQL metadata store, MinIO artifact storage, Argo Workflows orchestration, Prometheus/Grafana monitoring with automated quality gates and retraining.
— LLM monitoring market reached $482.6M in 2026, quantifying enterprise investment in drift detection and model degradation mitigation as critical MLOps capability across GenAI deployments.
— Uber D3 drift detection system quantifies monitoring ROI: 45-day detection delay cost millions; partial data incidents have 5X longer TTD than complete outages. Column-level monitors check null%, FK consistency, percentiles, distribution drift.
— Databricks MLflow 3 GA with 30+ million monthly downloads, Deployment Jobs for lifecycle automation, Unity Catalog integration for governance and queryable experiment tracking.
— Databricks MLflow system tables GA enable SQL-queryable experiment data with experiment lifecycle, run parameters/metrics history, and Unity Catalog access control for production monitoring and governance.
2019: Experiment tracking emerged as critical MLOps infrastructure with major vendor investments (AWS SageMaker Experiments GA, MLflow Model Registry). Open-source MLflow showed rapid adoption (800k monthly downloads) but significant scalability limitations. Model monitoring recognized as essential but tooling immature.
2020: Vendor platforms matured with AWS expanding SageMaker Model Monitor for production observability; MLflow consolidated as open-source standard across cloud platforms. Adoption broadened among organizations running 3-5 production models, but usability gaps and scalability constraints persisted; most teams relied on manual retraining rather than automated monitoring.
2021: Production adoption accelerated with Kubeflow survey showing 48% of users in production (3x growth YoY); AWS advanced model monitoring with quality metrics and CloudWatch alerting. However, enterprise integration fragility emerged (SQL Server backend failures), and critical assessments highlighted gaps between laboratory metrics and production robustness in real-world deployments.
2022-H1: MLflow reached 11M monthly downloads and introduced Model Registry Webhooks (Feb) and Pipelines framework (Jun) for end-to-end automation. Cloud platforms deepened integration with serverless deployment patterns and SaaS alternatives (WandB) gaining mindshare. Kubeflow user survey confirmed steady adoption. Deployment patterns matured but automated retraining and multi-model monitoring orchestration remained unsolved.
2022-H2: MLflow 2.0 shipped in November with 13M downloads, 500+ contributors, and major API refinements (MLflow Recipes, stable evaluation APIs), signaling category-level platform maturity. Cloud vendors scaled serverless MLflow deployment. However, Kubeflow survey revealed 59% of users identify monitoring as biggest gap; clinical research exposed monitoring gaps in high-stakes healthcare; MLflow 2.0 introduced artifact upload scalability issues. Monitoring remained the bottleneck.
2023-H1: Peer-reviewed research validated experiment tracking tool maturity; Kubeflow 1.7 advanced Katib UI and pipelines-as-components; Microsoft Azure and AWS deepened vendor investment in experiment tracking with new dashboard features. Enterprise case studies (DataRobot/SageMaker) demonstrated production monitoring architectures. However, MLflow artifact downloading failures and lack of unified drift detection standards continued to constrain broader adoption across model fleets.
2023-H2: Kubeflow's acceptance into CNCF incubator (July) signaled ecosystem maturity with 150+ adopting companies and 10 commercial distributions. Experiment tracking tools ecosystem solidified with comparative analyses of MLflow, DVC, and alternatives showing production-grade adoption. Model monitoring tools proliferated (Arize, Evidently, Datadog), with market projections of $5.9B by 2027 (41% CAGR). AWS and cloud vendors deepened monitoring capabilities with automated retraining patterns. However, critical security vulnerability (CVE-2023-43472) in MLflow 2.x allowing model/data exfiltration highlighted production readiness gaps; tools remained fragmented and lacked unified drift detection standards.
2024-Q1: Cloud vendors deepened MLflow integration (Databricks GA on AWS January 2024, Azure MLflow dual-tracking March 2024) signaling production maturity. Kubeflow demonstrated successful deployments on GKE/Vertex AI (QAware case study, March 2024) but critical fragility emerged—teams documented migration away from Kubeflow to Flyte (aiXplain, January 2024) due to operational unreliability and complexity, with MLflow operator failures on cluster restarts (February 2024) exposing artifact store brittleness. Vendor-specific approaches competed (SageMaker, Databricks native tools) creating trade-off complexity for teams. Ecosystem remained fragmented with no dominant production solution for distributed model fleet governance.
2024-Q2: AWS released fully managed MLflow on SageMaker (June), removing operational burden and signaling vendor consolidation around open-source standards. Production monitoring adoption accelerated with Mayo Clinic publishing peer-reviewed monitoring platform research (June), documenting real-world challenges. Research community advanced monitoring science with Helmholtz AI 2024 drift monitoring systems and ICT4S 2024 empirical trade-off analysis across 7 algorithms. Integration challenges persisted between MLflow and Kubeflow (documentation gaps noted June). Educational implementations proliferated with capstone projects demonstrating accessibility of production patterns. Vendor competition intensified with trade-offs between managed simplicity (SageMaker, Databricks) and open-source flexibility (MLflow, Kubeflow) becoming sharper. Monitoring remained bottleneck: cost, standardization, and drift detection complexity constrained broader adoption.
2024-Q3: Vendor consolidation accelerated with Azure GA tooling for MLflow tracking and production monitoring (August); peer-reviewed research (August 2024) revealed persistent adoption barriers and low practitioner awareness despite industry rhetoric. GenAI monitoring emerged as extension to traditional MLOps. Operational fragility continued—production Kubernetes deployments encountered database migration failures (July issue) affecting reliability. Practitioner analysis highlighted silent failure risks and drift detection methodology fragmentation. Monitoring remained critical bottleneck despite mature tooling landscape.
2024-Q4: Vendor consolidation completed with Azure Databricks and Azure ML releasing GA MLflow integration (Nov); Databricks published production MLOps workflows with Model Registry governance (Dec). Despite three major cloud vendors offering fully managed MLflow, open-source adoption showed mixed results: named Kubeflow deployments (Samsung SDS, IBM, Coupang) operated successfully with Datadog monitoring, but multi-user deployments encountered MLMD connectivity failures after node restart (Nov) and Kubeflow 1.9 installation via Juju remained incomplete (Dec). Experiment tracking commodified but monitoring methodology remained fragmented; GenAI monitoring emerging as new frontier. Core tension: vendor consolidation traded flexibility for reliability; organizational adoption barriers persisted despite mature tooling.
2025-Q1: Vendor consolidation matured with Azure ML GA model monitoring capabilities (March) supporting drift, prediction, and data quality signals with automated alerting. Enterprise adoption accelerated: 78% of enterprises now have dedicated MLOps teams (up from 32% in 2023), managing average 250+ production models; 94% implementing drift and concept drift monitoring. Research advanced monitoring automation with MLMA framework validated at scale in real deployments (Feb). However, operational brittleness persisted: MLflow scalability ceiling at ~3500 experiment runs; integration failures with Azure Government environments; implementation complexity in ground truth mapping for monitoring. Monitoring methodology remained fragmented despite vendor tooling maturity—real-world deployments revealed silent failure risks and trade-offs in detection approaches.
2025-Q2: Vendor tooling stabilized with financial and enterprise case studies demonstrating production MLflow deployments (Aalto SaaS case study, June). Market confidence in monitoring tools reached $3.8B invested ecosystem with significant vendor expansion (April). However, critical integration challenges persisted: MLflow-Azure SDK version mismatches continued (June); production scope boundaries clarified with documentation confirming MLflow tracking is unsuitable for model serving (June). Practitioner guides (May) with 10+ years production experience highlighted endemic silent failure risks, monitoring cost barriers, and methodology fragmentation. Real-world monitoring data revealed stark degradation: 91% of models degrade within 1-2 years without proactive retraining; B2B contact data decays 22-70% annually. The core tension sharpened: vendor platforms (SageMaker, Azure ML) offered simplicity but lock-in; open-source tools (MLflow, Kubeflow) required significant operational investment. Despite maturity, monitoring remained the critical blocker to enterprise adoption at scale, with ongoing friction in platform integration and scope clarity.
2025-Q3: Vendor consolidation achieved maturity with Azure Databricks GA MLflow system tables (Sept) enabling SQL-based experiment analysis and Azure ML documenting production model deployment (Aug). Open-source ecosystem advanced: Kubeflow Model Registry formalized experiment tracking APIs (Aug-Sept), showing convergence with model registry. Real-world adoption case study: Graylight Imaging documented MLflow implementation for FDA-regulated medical workflows (July), demonstrating deployment in high-stakes compliance contexts. Industry monitoring frameworks (Evidently AI) published standardized pyramid with DoorDash/Booking.com references, but methodology fragmentation and cost barriers persisted as adoption constraints.
2025-Q4: MLflow consolidated dominance with 57% adoption in experiment tracking, up from 42% YoY, establishing tracking as most consolidated MLOps domain. Databricks released MLflow 3.0 GA with generative AI support (LLM evaluation, prompt versioning), extending platform beyond traditional ML. Market growth projections accelerated to $23.4B by 2030 (38.9% CAGR) with enterprises reporting 3-5x faster deployment cycles and 50-70% reduction in model failures. However, monitoring remained critical bottleneck: 87% of models still failed to reach production despite mature tracking tools, revealing structural limits in adoption despite platform maturity. MLOps unbundling trend accelerated with teams mixing specialized monitoring tools rather than relying on single platforms; vendor consolidation on managed services increased but exposed trade-off between simplicity and operational flexibility.
2026-Jan: Experiment tracking ecosystem stabilized with empirical validation of MLflow (8.30/10) as highest-scoring platform across 6 technical criteria; MLflow 3.1.0+ advanced GenAI monitoring capabilities with tracing APIs. Market analysis confirmed $5.64B ModelOps market (41.3% CAGR through 2030) positioning experiment tracking as foundational enterprise capability. However, platform integration gaps persisted: Kubeflow adoption hampered by learning curve, documentation obsolescence, and AWS authentication complexity; Azure ML's tracking server incompatible with MLflow 2.8+ Logged Models API, forcing separate training/inference environments. Monitoring remained strategic bottleneck despite mature tooling landscape.
2026-Feb: MLflow's dominance strengthened with 30M+ monthly downloads and 20K+ GitHub stars across 900+ contributors, consolidating as de facto standard. Azure Databricks shipped GA feature for MLflow traces in Unity Catalog with OpenTelemetry support (Feb 1), enabling SQL-queryable experiment records. Splunk and major observability vendors deepened Kubeflow/MLflow integrations. Practitioner narratives documented production-grade patterns (Docker Compose, PostgreSQL, MinIO) with LLM fine-tuning extensions. However, platform fragmentation persisted: Microsoft Fabric exposed MLflow API gaps (aliases, metrics access limitations); Kubeflow continued to show adoption friction despite CNCF maturity. Monitoring remained constraining factor despite advanced tracking tooling.
2026-Mar: MLflow 3.10.0 GA shipped with multi-workspace support and trace cost tracking, addressing enterprise-scale adoption and generative AI observability. Enterprise case studies demonstrated production deployment: financial services automated MLOps with governance (Persistent Systems), edge ML drift monitoring with automated remediation (OpenClaw Rating API). Practitioner validation: LLM drift detection framework quantified real production drift (0.0-0.575 scores) with documented silent failure risks. Security assessment documented 9+ CVEs across MLflow versions, indicating production deployment validation requirements. Critical finding from ETR research: AI model monitoring remains biggest unmet need despite mature tracking tooling, with observability platforms failing at drift detection and auditability. Regulatory context emerged: AML systems require continuous monitoring and governance (FATF, Federal Reserve SR 11-7), positioning drift detection as compliance mandate. Core tension persisted: experiment tracking commodified and consolidated around MLflow; monitoring methodology fragmented between statistical approaches (KS test, KL divergence), proprietary platforms (Arize, Fiddler), and open-source tools (Evidently, WhyLabs).
2026-Apr: MLflow 3 platform maturity demonstrated through expanded feature set: Deployment Jobs (Public Preview) automate full model lifecycle with registration triggers, and Azure Databricks GA feature stores MLflow traces in Unity Catalog as queryable SQL tables—addressing scale limitations and operational requirements. Market analysis confirmed $1.115B MLOps market (2025) with 41.3% CAGR through 2031. Production deployment evidence from practitioners (PulseFlow, 28+ years experience) documented end-to-end MLOps patterns with ETL, Airflow orchestration, and Docker composition. Named org adoption: Cisco CX deployed 100+ agents across 20K-person team with advanced drift monitoring (4 independent drift variables, statistical thresholds via KS test). Kubeflow's CNCF maturity confirmed with health score 86/100, 6,892 contributors, 1,146 adopting organizations, $492.8M software value. AWS managed MLflow (SageMaker) GA with Wildlife Conservation Society case study demonstrating serverless scaling. Monitoring discipline advancing with technical drift detection frameworks: behavioral fingerprinting achieves 86% detection power for LLM provider drift; regression canaries and statistical monitoring (PSI, KL divergence) documented as production methods. Gartner 2025 quantified drift impact: undetected drift costs $3.1M annually per enterprise. However, security maturity gaps: 11+ critical CVEs in MLflow including 10.0 CVSS RCE via command injection and hardcoded credentials, signaling governance and security validation requirements despite broad adoption. Enterprise monitoring gaps persisted: 91% of models degrade over time; 75% of deployments decline without monitoring; 87% never reach production. Monitoring methodology remained fragmented; only one-third of organizations have risk mitigation controls; drift detection taxonomy lacked standardization despite mature tooling landscape.
2026-May: Vendor consolidation on managed MLflow services continued with AWS SageMaker GA MLflow v3.10 (May 5) and Microsoft Fabric MLflow cross-workspace integration (May 1). Uber published production deployment case study demonstrating ML safety mechanisms at hyperscale: Michelangelo platform executing 400 active use cases, 20K training jobs/month, 15M predictions/sec with shadow testing on 75% of critical models and automated rollback on performance breach. Uber also published D3 drift detection system quantifying monitoring ROI: partial data incidents experience 45-day detection delays costing millions; column-level monitors (null%, FK consistency, percentiles, distribution drift) eliminate manual threshold tuning at petabyte scale. Enterprise adoption metrics: LLM monitoring market reached $482.6M (April), signaling substantial investment in drift detection across GenAI deployments. Independent practitioner deployment (Eric Nguyen, April) documented end-to-end MLOps on Kubernetes with MLflow, PostgreSQL, MinIO, Argo Workflows, and Prometheus/Grafana with automated quality gates—exemplifying mature, production-grade engineering patterns now accessible to individual teams. Monitoring science advancing: research on concept drift detection (ICLR 2026) validated multi-signal approaches with real malware classification datasets, confirming detection requires complementary metrics across feature importance, prediction agreement, activation stability, and coverage. Platform consolidation matured: MLflow as de facto experiment tracking standard (30M+ monthly downloads), all three major cloud vendors GA MLflow support with unified managed offerings, open-source ecosystem stabilizing around standardized APIs. However, monitoring methodology remained fragmented with no sector-wide consensus on drift detection standards; organizational adoption barriers persisted despite mature, accessible tooling.