Perly Consulting │ Beck Eco

The State of Play

A living index of AI adoption across industries — where established practice meets the bleeding edge
UPDATED DAILY

The AI landscape doesn't move in one direction — it lurches. Some techniques leap from experiment to table stakes in a single quarter; others stall against regulatory walls, technical ceilings, or organisational inertia that no amount of hype can dislodge. Knowing which is which is the hard part. The State of Play cuts through the noise with a rigorously maintained index of AI techniques across every major business domain — classified by maturity, evidenced by real-world adoption, and updated daily so you always know where you stand relative to the field. Stop guessing. Start knowing.

The Daily Dispatch

A daily newsletter distilling the past two weeks of movement in a domain or two — delivered to your inbox while the index updates in the background.

AI Maturity by Domain

Each dot marks the weighted maturity of practices within a domain — hover for a brief summary, click for more detail

DOMAIN
BLEEDING EDGEESTABLISHED

Data governance & rights management for AI

BLEEDING EDGE

TRAJECTORY

Stalled

Governance frameworks for managing data used in AI training and fine-tuning, including provenance, consent, data rights, and opt-out management. Includes training data documentation and deletion-from-model workflows; distinct from general data privacy which manages operational rather than AI-specific data.

OVERVIEW

Data governance for AI sits in a precarious split: the infrastructure half has matured while the hardest technical problem remains unsolved. Governance platforms now provide production-grade lineage, access control, and documentation capabilities, and regulatory mandates like the EU AI Act and U.S. federal procurement standards have made these table-stakes for regulated deployment. That side of the practice works. The other side -- verifiable deletion of training data from models -- does not. Peer-reviewed research continues to show that machine unlearning methods suppress rather than truly remove learned information, and no scalable proof-of-deletion mechanism exists. This bifurcation defines the bleeding-edge status: organisations can govern what goes into training pipelines, but they cannot yet prove data has been removed once a model has learned from it. The gap between regulatory expectation and technical capability is the defining tension, and it is not closing.

CURRENT LANDSCAPE

Governance infrastructure has reached production maturity and market saturation, now focusing on agentic AI as the operational frontier. Databricks, Microsoft Azure, AWS, and specialist vendors (Collibra, Immuta, Informatica, OpenMetadata) offer GA platforms for lineage tracking, access governance, and automated compliance workflows. Collibra's March 2026 AI Governance product unifies use-case, model, and agent registries; Immuta's April 2026 "Agentic Data Access" and June 2, 2026 Snowflake integration treat AI agents as first-class governed data users with ephemeral, scoped access and dual-identity audit trails—addressing governance emergence as agentic deployment accelerates. Governance has shifted from compliance checkbox to scaling enabler: enterprises with governance frameworks achieve 12x higher production project success and 6x greater autonomous system scale (Databricks May 2026 study, 20,000+ enterprises). Yet deployment significantly lags governance readiness: a June 2026 Gartner analysis found 57% of IT leaders pushed to adopt AI before organizationally ready, with only 14% confident data is secured/governed. Rights exercise has moved from theoretical to operational: DataGrail reports deletion requests surged 567% since 2021 (now 87% of all data subject requests), with manual handling costing enterprises $1.5M/year on average, creating market pressure for automated governance platforms. Hong Kong Privacy Commissioner audit (May 2026) of 60 organizations shows 95% use AI but governance gaps—only 29% retained personal data post-processing, only 29% disclosed AI in privacy notices—revealing enforcement-driven maturity indicators across jurisdictions.

Real production failures continue to expose governance gaps. May 2026 evidence reveals persistent disconnect between governance platforms (mature) and deployment readiness (immature): Gartner data shows 57% of IT leaders pushed to adopt AI before organizationally ready, with only 14% confident data is secured/governed. Case studies document failures in practice—ungoverned AI agents encountering sensitive data (SSNs, billing records) in tickets, healthcare models perpetuating bias through uncontrolled training data, governance tools treating accessibility compliance differently across vendors. Agentics analysis confirms governance (not cost, talent, or technology) is the #1 blocker to scaling AI across regulated industries. Governance multiplier effects are measurable: enterprises with governance frameworks see 12x more projects reach production and 6x more scale autonomous systems. Yet 40% still lack adequate governance despite deployment, and enterprises deploying agentic AI report 90%+ struggle with audit trail opacity and data lineage uncertainty—governance governance architectural debt compounding at scale.

Regulatory enforcement and compliance barriers intensified through June 2026, with DPA guidance operationalizing rights management architecture. GDPR enforcement reached EUR 5 billion in cumulative fines; the EU AI Act's August 2, 2026 compliance deadline for high-risk systems (EUR 35M or 7% revenue penalties) drives urgent governance adoption. CNIL (French DPA) published January 2026 guidance operationalizing GDPR principles (purpose, roles, rights facilitation, retention) for AI, acknowledging "particular and unprecedented difficulties" in exercising rights on model weights while recommending proportionate implementation patterns. EDPS (European Data Protection Supervisor) issued June 8, 2026 formal orientations to EU institutions on generative AI governance, signaling movement from advisory to enforcement posture. Analysis of 19 regulatory guidelines across jurisdictions reveals enforcement divergence masked by surface consensus: Italy fined OpenAI EUR 15 million for inadequate legal basis and transparency; Brazil's ANPD suspended Meta's AI training; Hong Kong Privacy Commissioner documented governance gaps in 60 audited organizations. The core deletion problem shows early technical advances offset by persistent verification gaps. May 2026 SoK paper documents that both unlearnability and unlearning suffer "shallow dememorization" with falsely claimed forgetting and lack formal guarantees. However, June 2026 peer-reviewed research (UMD, Model State Arithmetic/MSA) demonstrated selective unlearning without full retraining using training checkpoints, enabling Article 17 erasure rights compliance at feasible operational cost. May-June 2026 research convergence: ALU framework enables mass deletion via public data augmentation; D² paradigm addresses latent knowledge re-emergence; yet June 2026 papers on reconstruction attacks and reversibility show unlearning brittleness under adversarial queries—information can be rapidly restored via fine-tuning. Organisations now face asymmetric information: governance platforms control data input maturely, deletion-from-model mechanisms show early-stage feasibility but lack formal verification guarantees, and regulatory authorities acknowledge both paths while demanding proportionate compliance solutions by August 2, 2026.

TIER HISTORY

ResearchJan-2023 → Apr-2024
Bleeding EdgeApr-2024 → present

EVIDENCE (94)

— Fresh (June 8) European Data Protection Supervisor formal guidance to EU institutions on gen AI data governance (DPIAs, data minimization, fairness, rights exercise); signals supervisory-authority enforcement posture moving from advisory to mandatory compliance.

— Systematization of 14 reconstruction attacks against synthetic data generation; NIST-validated finding that differential privacy protection plateaus at high epsilon and synthesizer choice dominates risk—essential for evaluating data governance tool effectiveness.

— Snowflake Summit announcement (June 2, 2026) of Collibra AI Command Center integration enabling production agentic AI governance; signals ecosystem maturity for governed data access at enterprise scale.

— Immuta-Snowflake agentic data access implementation: agents receive ephemeral, provisioned access scoped to user permissions with dual-identity audit trails; demonstrates production architecture for governing AI agent data access at scale.

— ICLR 2026 research (Model State Arithmetic/MSA) enabling selective unlearning via training checkpoints without full retraining; demonstrates technical feasibility of Article 17 erasure rights compliance at scale without model rebuilding.

— Regulatory authority audit of 60 organizations: 95% use AI but governance gaps evident—only 29% retained personal data post-processing for rights exercise, only 29% disclosed AI in privacy notices, revealing enforcement-driven governance maturity indicators.

— Quantified adoption signals: deletion requests surged 567% since 2021; 87% of data subject requests are now deletions; manual DSR handling costs $1.5M/year—demonstrating scaling of rights exercise operationalization and governance market maturity.

— IAPP legal analysis identifies governance flaw: consent validity becomes questionable when processing design makes withdrawal structurally impossible. Signals regulatory gap in data rights management.

HISTORY

  • 2023-H1: Data governance for AI emerged as urgent industry priority post-ChatGPT. Databricks acquired Okera to add AI-specific governance; TDWI published governance frameworks for ML assets. Unlearning research validated feasibility of data deletion from models.
  • 2023-H2: Regulatory enforcement accelerated; Italy suspended ChatGPT, Canada, France, and Spain opened investigations. Unlearning research advanced (EMNLP, NeurIPS competitions) but critical limitations emerged: methods may not achieve true data removal, utility trade-offs remain unsolved. Copyright opt-out mechanisms proved ineffective without platform transparency. Gap widened between regulatory expectations (right to be forgotten) and technical reality.
  • 2024-Q1: Unlearning research advanced on efficiency and multimodal models, with partial amnesiac approaches reducing fine-tuning overhead. Data Provenance Initiative documented 1,800 curated datasets. Databricks Unity Catalog expanded into financial services for EU AI Act compliance. Enterprise surveys showed 36% identified AI governance as GenAI adoption barrier. Analyst predictions: 80% of governance initiatives will fail by 2027. Regulatory gap widened: EU AI Act exempted open-source models from dataset transparency requirements.
  • 2024-Q2: EU AI Act finalized with explicit copyright opt-out and data governance mandates (€35M/7% penalties, 24-month compliance window). Vendors (Databricks) accelerated platform adoption for production GenAI deployments. IDC research showed governance maturity as key driver of AI initiative success (20% fail without infrastructure). U.S. state-level regulations emerged (Colorado CAIA, Utah AI Policy Act). Critical gap remains: opt-out implementation infrastructure and practical deletion-from-model workflows still lacking at scale. Governance becoming table-stakes for regulated deployment but organizations struggle with training pipeline integration.
  • 2024-Q3: Vendor governance platforms matured (Microsoft/Azure Databricks best practices published). Gartner forecast 30% GenAI project abandonment by 2025 due to poor data quality and governance gaps. Critical limitations in unlearning emerged: Google/Princeton research exposed adversarial vulnerabilities (model accuracy degraded to 3.6%); MUSE benchmark found most algorithms fail privacy/utility simultaneously; Oxford/MIT survey concluded unlearning cannot reliably enable deletion-from-model workflows. Opt-out infrastructure and verification mechanisms remained absent. Governance platforms adopted for lineage and access control; deletion-from-model compliance mechanisms still immature.
  • 2024-Q4: AWS launched SageMaker Data and AI Governance GA, signaling broad vendor platform maturity for governance infrastructure. Research revealed severe unlearning vulnerabilities: reconstruction attacks recovered deleted data despite unlearning, emphasizing differential privacy as mitigation necessity. Industry surveys documented widespread governance adoption barriers—80% of AI projects fail (RAND/Gartner), with 62% citing lack of governance and only 12% of organizations reporting sufficient data quality for AI. Governance became recognized adoption blocker and competitive differentiator.
  • 2025-Q1: Unlearning research advanced with new evaluation metrics and parameter-efficient frameworks (ICLR 2025 papers), but critical vulnerability assessments revealed state-of-the-art methods fail at scale—they degrade model quality or merely modify classifiers without truly removing training data influence. Governance platform maturity continued (Databricks DAGF v1.0 framework released), but enterprise adoption surveys showed 21% of organizations still lack governance frameworks, 33% cite leadership misalignment, and 60%+ cite data quality barriers. EU AI Act compliance deadline (April 2025) approached with deletion-from-model mechanisms still unproven, widening gap between regulatory mandate and technical feasibility.
  • 2025-Q2: April 2025 EU AI Act compliance deadline arrived without reliable unlearning solutions. New research exposed verification gaps: arXiv survey on unlearning verification (June 2025) found behavioral and parametric approaches remain fragmented with no unified standard; CMU peer-reviewed analysis (April 2025) showed benchmark structures systematically overestimate unlearning effectiveness; comprehensive auditing frameworks (May 2025) found six algorithms fail to demonstrate true knowledge removal. CSA assessed right-to-be-forgotten as unresolved with no proven scalable solutions. Financial services drove governance adoption, treating data provenance documentation as contractual requirement. Core tension remained: governance platforms advanced for transparency/lineage, but deletion-from-model verification stayed unproven at scale.
  • 2025-Q3: Enterprise governance deployment stalled; only 30% of organizations advanced beyond experimentation to production, with just 13% managing multiple deployments and 48% failing to monitor production systems. Federal government cited data governance and security as critical AI adoption barriers, despite regulatory mandates. The quarter revealed persistent infrastructure gaps: enterprises struggled with governance platform integration, data quality remained a blocker for 60%+ of organizations, and no new breakthroughs in deletion-from-model verification emerged. Governance remained a recognized adoption blocker and competitive requirement, but deployment maturity plateaued.
  • 2025-Q4: Unlearning research advanced with new frameworks (OBLIVIATE, LUNE) addressing efficiency and deletion quality, but no resolution emerged for verification gaps or scalable proof-of-deletion. Governance platform deployments remained operational for lineage and access control (Databricks, Azure, AWS), yet financial sector contracts still relied on documentation and provenance rather than technical deletion guarantees. Federal agencies continued struggling with governance infrastructure adoption. The year ended with governance platforms mature and research active, but the core tension—between regulatory deletion mandate and technical verification inability—unresolved at production scale.
  • 2026-Jan: EU AI Act and OMB M-25-22 enforcement drove governance from emerging practice to market license. Vendor governance frameworks matured (Databricks, Azure, AWS); strategic analysis from data leaders confirmed governance as 2026 priority and enablement layer for scaling AI. Simultaneously, peer-reviewed research published critical assessments: Columbia Law Review analyzed unlearning's policy limitations, new economic audit models exposed verification challenges, and GhostDrift analysis identified accountability evaporation risks in static compliance frameworks. Governance infrastructure and documentation standardized; deletion-from-model verification remained unsolved at scale.
  • 2026-Feb: Regulatory enforcement and compliance barriers continued to intensify. New peer-reviewed research (February arXiv papers) exposed fundamental verification gaps in unlearning: representation-level analysis questioned whether methods truly delete vs. suppress training information; perfect retraining attacks revealed deletion claims may inadvertently expose undeleted elements. OpenAI case study documented immense technical challenges of purging user data from complex ML pipelines. GDPR enforcement reached €5B cumulative fines with 20 US states enacting comprehensive privacy laws. Persistent gap between opt-out expectations and technical reality in training data governance underscored compliance obstacles. Governance infrastructure commoditized; deletion-from-model verification and audit methodologies remained fragmented and unproven.
  • 2026-Apr: Enterprise governance platforms advanced with Collibra launching dedicated AI Governance covering use cases, models, and agents, and Immuta treating AI agents as first-class governed data users with zero standing privileges — addressing a critical surface as 80% of Fortune 500 firms deploy GenAI but fewer than 40% have adequate governance. OpenMetadata reached GitHub Trending #1 (13,535 stars) driven by AI governance and semantic data features, while a production case study of an ungoverned customer support agent encountering SSNs in tickets illustrated the real costs of governance gaps. Unlearning remained practically unreliable: ICLR 2026 research showed adversarial prefix attacks cause 1,150x information leakage surges, EACL 2026 auditing frameworks revealed residual knowledge persists post-unlearning, and production quantization masks standard unlearning methods — while the EDPS TechSonar assessment confirmed GDPR-aligned deletion mechanisms remain unverifiable at scale.
  • 2026-May: Governance deployment gaps widened further: Gartner data (57% of IT leaders pushed to adopt AI before ready; only 14% confident data is secured/governed) and Observer analysis of real production failures reinforced the governance-adoption lag, while Agentics research quantified a 12x production success multiplier for enterprises with governance frameworks. On the technical deletion front, two concurrent ICML 2026 papers (D² paradigm and ALU framework) advanced unlearning theory — D² addressing latent knowledge re-emergence, ALU enabling mass deletion via public-data augmentation — but a May 2026 SoK survey concluded both unlearnability and unlearning still suffer shallow dememorization with no formal deletion guarantees at scale. IAPP legal analysis flagged a structural GDPR consent gap: processing designs that make withdrawal impossible render the original consent legally questionable, adding a new regulatory pressure layer on top of the unresolved technical problem.
  • 2026-Jun: Agentic AI governance moved from emerging to operational. Snowflake-Collibra partnership (June 2) delivers production agentic data access with ephemeral role provisioning and dual-identity audit trails; Immuta's agentic data access deployment demonstrates zero-standing-privileges governance at scale. Regulatory authorities operationalized guidance: CNIL (January 2026) published proportionate implementation framework for GDPR rights on models; EDPS (June 8) issued formal orientations to EU institutions on gen AI data governance, signaling enforcement posture. Rights exercise moved to scale: DataGrail data shows 567% surge in deletion requests since 2021, now 87% of all DSRs. Governance effectiveness quantified: 12x production multiplier for projects with governance; Gartner found 57% of IT leaders pushed to deploy before ready, only 14% confident data secured/governed. Technical advances in unlearning published: UMD MSA research enables selective deletion via training checkpoints without retraining (ICLR 2026); yet NIST-validated research on reconstruction attacks against synthetic tabular data finds differential privacy protection plateaus at high epsilon and synthesizer choice dominates risk — a critical finding for governance tool selection and compliance claims. Hong Kong Privacy Commissioner audit of 60 organizations reveals governance-adoption gap: 95% use AI but only 29% retained personal data for rights exercise, only 29% disclosed AI in privacy notices. Core tension persists: governance platforms mature, deletion-from-model mechanisms show early feasibility without formal verification guarantees, regulatory authorities demand proportionate compliance by August 2, 2026.