The State of Play

A living index of AI adoption across industries — where established practice meets the bleeding edge
UPDATED DAILY
← All domains

🎯 Product & Design

AI applied from user research through to shipped product experience. Wide maturity spread: A/B testing and analytics are established, prototyping and design systems are good practice, but nearly half the domain is bleeding-edge — generative UI, autonomous UX research, and AI-native product frameworks are experimental. Most practices are stalled, with more energy in tooling announcements than production adoption.

13 practices: 2 established, 4 good practice, 4 leading edge, 3 bleeding edge

The Headline

AI now drafts specs, screens and copy in minutes, but written rules don't bind it and reviewing its work eats the time saved. The teams pulling ahead enforce their standards with automated checks.

13
practices tracked
7
at the frontier
0
moving this fortnight
2,612
evidence items

The Picture

Almost every product tool you already pay for now ships an AI assistant or agent (software that acts on its own without being prompted), and nearly every individual on product teams uses AI. Very few organizations have built it into how the whole team works. In Zeroheight's design systems survey, 10 percent of teams have built AI into their design-system process, against 56 percent that are experimenting. The teams pulling ahead put machine checks around the AI: build gates that reject code using components outside the approved design system, pixel comparisons against the approved design, and experimentation assistants that choose and run validated tools but never calculate a result themselves. Everyone else is running into what Figma's own leadership has admitted: teams produce 17 times as much code but ship only 1.3 times as many releases.

This Fortnight

  • Written rules failed to keep AI agents inside the design system. In NordVPN's tests, models built their own buttons, dialogs and toggles in 41 of 57 runs, even though the design system shipped an instruction file written for agents. A style guide describes your standard but does not enforce it; only checks that fail the build do that, so ask your teams which of the two they actually have.

  • A review of 50 studies on AI agents that write software requirements found no production deployments. Practitioner reports point the same way: one AI-generated spec passed all its tests for an app that did not run in a browser. Treat AI-written specs as drafts, and expect more value from having AI list the open questions than from letting it answer them.

  • Figma rolled out AI-built, code-backed prototypes to all users and says its MCP (a connector standard for AI tools) has about a million weekly users. Pendo also made its AI assistant generally available in Slack and Teams, and AlphaSense opened a single MCP connection to its research library on November 4. These features are landing inside contracts you already hold, and Figma says AI credit charges may now apply to its prototypes, so check what your renewals meter.

  • In a vendor-commissioned survey of 573 professionals, 87 percent had hit at least one personalization error in the past year. Most of those who hit errors heard about them from customer complaints, not from their own data, and buyer interviews describe slow rollouts and one recommendations product still unused after nearly two years. Before funding personalization, ask how you will catch a wrong recommendation before a customer does.

  • University researchers found three of the 20 most popular chatbots sending user conversations in plain text through Microsoft Clarity's session-replay tool. Separately, a lawsuit filed in October alleges FullStory kept recording a visitor after they opted out; the claims are unproven. If you record user sessions, have counsel confirm what the recordings mask and whether opt-outs hold mid-session.

Coming Up

  • Anthropic closes the standalone Claude Design site on December 14 and moves design systems into Claude itself. Users have reported that its GitHub and Figma sync misses components and variables on larger libraries. If your designers rely on it, confirm the migration and check that your full component library made it across.

  • Accessibility deadlines for US public bodies under ADA Title II now fall between April 2027 and 2028. When it extended them, the Justice Department said generative AI does not yet reliably automate fixing inaccessible content at scale. If you build for public bodies or sell to them, budget for human audits rather than assuming AI-built interfaces will pass.

  • France's data regulator, CNIL, has issued draft guidance that would treat session replay as a purpose needing its own separate consent. Together with US wiretap suits that test whether consent holds mid-session, this will shrink the share of users you can record and tilt it toward those who opt in. Watch for the final guidance, and expect journey maps built on recordings to describe a self-selected slice of your customers.

What's Hard About This

  • Drafting got cheap; checking did not. In Atlassian's survey of 1,000 product professionals, 82 percent say teammates hand in low-effort AI work that has to be revised, and most say decisions have not sped up. The savings only appear where teams put as much into checking AI output as into producing it.

  • Data readiness, not the choice of tool, decides whether AI analytics pays off. In a 10,000-business survey cited by Pendo, 81 percent of firms with fully AI-ready data reported broad or strong returns, against 12 percent of those whose data was only partly ready. Meanwhile, none of GA4, Amplitude or Mixpanel publishes an accuracy rate for its AI assistant, so the cleanup work falls on you.

  • Most of the success figures in this field come from vendors. Where independent measurement exists, it shrinks them: Microsoft Research estimates that at least 75 percent of activity credited to product recommendations would have happened anyway. Ask for a control group before accepting any lift figure in a business case.

2012View this domain's timeline →Today

Practices in this Domain (13)

PRACTICETIERTREND
A/B test design & analysisESTABLISHED— Steady
Accessibility auditing & remediationGOOD PRACTICE— Steady
Behavioural analytics — session replay & interaction patternsESTABLISHED— Steady
Competitive product analysisGOOD PRACTICE— Steady
Design system generation & enforcementLEADING EDGE— Steady
Feature prioritisation & roadmap supportBLEEDING EDGE— Steady
Personalisation engine design & tuningLEADING EDGE— Steady
Product analytics interpretation & insightBLEEDING EDGE— Steady
Requirements, PRD & user story generationLEADING EDGE— Steady
User journey mapping from behavioural dataGOOD PRACTICE— Steady
User research & feedback synthesisGOOD PRACTICE— Steady
UX copy generation & voice enforcementLEADING EDGE— Steady
Wireframe generation & design-to-code conversionBLEEDING EDGE— Steady
Read the full technical briefing (1,753 words) →

Where AI Stands in Product & Design

Across the toolchain from research interview to shipped screen, product and design has adopted AI further than it has improved outcomes. Almost every platform now ships an assistant, an agent or an MCP server: Dovetail, Amplitude, Mixpanel, Pendo, Figma, Optimizely, Kameleoon, Klue, Crayon, AlphaSense and Productboard all do. Individual use is close to universal. In Nielsen Norman Group's survey of 521 UX practitioners, 67% of those who analyse their own research data use AI often or always for that task. Integration across whole organisations is far thinner. Zeroheight's design systems report finds 10% of teams have built AI into their design-system process, against 56% experimenting with it. In one product-leader survey only 17% rated AI's impact on roadmapping as high. Figma's own leadership has put the gap in one number: teams produce 17 times as many lines of code but only 1.3 times as many releases.

The domain's maturity follows a clear pattern. Where AI sits on top of established methods with deterministic scaffolding, such as experimentation, session replay and accessibility scanning, it is incremental and increasingly disciplined. The builders of one large in-house A/B testing agent let the model choose and sequence validated tools but never compute a number. Deque confines AI to narrow yes-or-no classifications inside a harness that makes the pass/fail call itself. Where AI is asked to make the judgement call, the evidence thins and the failures become specific. Those judgement calls include ranking the roadmap, explaining why a metric moved, writing the spec, holding the voice and composing the screen. In each, models fill gaps with plausible inventions: undecided requirements written up as decided, RICE scores fabricated when the data is missing, microcopy promising confirmation emails that do not exist. Momentum is building in enforcement infrastructure: lint gates in CI, MCP servers that query a system of record, and semantic layers, which Atlan reports lift analytics-agent accuracy from 21% to 95%. Momentum has stalled where judgement cannot be scaffolded. A systematic review of agentic requirements engineering found no production deployments at all.

Two things set this domain apart from its neighbours. Its output is judged by people who notice when it is wrong: users, auditors, regulators and readers. And its evidence base is mostly written by vendors. Merkle's 30% faster iteration in Figma, Michael Hill's 64% rise in average transaction value credited to a Contentsquare-informed redesign, and Grez's 62.6% revenue-per-visitor lift are all self-reported, and almost none come with controls or significance tests. Where independent measurement exists, it deflates the vendor figures. Microsoft Research estimates that at least 75% of recommendation-driven activity would have happened without the recommendations. The constraint in this domain is not model capability. It is verification, data readiness and trust, and announcements are arriving faster than any of the three is improving.

What's New, 2026-10-24 to 2026-11-07

The fortnight's most useful evidence showed written rules failing to bind agents, and review effort eating the time saved on drafting. In NordVPN's tests, reported via ProdFolks, models built their own buttons, dialogs and toggles in 41 of 57 runs even though the design system shipped an AGENTS.md. Supernova logged an agent using the shadcn MCP without a rules file breaking a team's rules 20 times across 9 runs. Definition gave Gemini the tone-of-voice guidelines of eight brands and got near-identical newsletters back. Requirements generation took a heavier blow. Bandi and Demir's review of 50 LLM-agent requirements-engineering studies found no production deployments, and cites earlier work showing over 90% of generative-AI requirements research never passed proof-of-concept. In a 182-story test, GPT-5-mini caught 51 of 63 confirmed defects, but GPT-5 produced 85 false positives at 0.26 precision. That study ran one pass per model with one annotator. Practitioners reported the same failures in practice. A Kiro-generated spec passed its tests for an app that did not run in a browser. A Claude Design handover to Claude Code produced a patch containing no CSS. A first Claude Design draft lacked failure states entirely. In experimentation, Octo's simulation showed weekly peeking at a 95% Bayesian chance-to-win threshold raising false winners from 10.5% to 45.3%. Purdue statisticians showed in simulation that continually validating AI analyst updates cut Type I error from 0.183 to at most 0.044 while keeping 96% of power. An ACM poster found GPT-5.5 and LLaMA-3.3 fixed structural WCAG violations on ten live homepages but failed on alt text and empty links, and fabricated elements when the code gave them no textual clues.

Distribution kept widening even so. On its Q3 call Figma said Code Layers had rolled out to all users and put its MCP server at about a million weekly users. Amplitude reported $111M in Q3 revenue, up 25%, and $431M in ARR, and demonstrated an Agent Analytics product without giving adoption figures. Pendo made Ask Pendo generally available in Slack and Teams, and AlphaSense launched a single MCP endpoint over 500M+ documents on 4 November. Dovetail's Channels 2.0 reached general availability after classifying 61 million data points. Deque cited a workflow that remediated and validated 80% of 171 WCAG issues, 70% of them with no human edits, while arguing in the same piece that AI alone cannot produce a reliable conformance report. The counter-evidence also accumulated. Alium's buyer interviews described personalisation projects taking six months to eighteen, and one recommendations product still unused after about twenty months. A Fullstory-commissioned survey of 573 professionals found 87% had hit personalisation errors, most of which came to light through customer complaints. Lumivero found only 21% of qualitative researchers use AI regularly, with distrust the top barrier at 57%. Privacy pressure on behavioural data sharpened. UC Davis researchers found three of the 20 most popular chatbots sending plaintext conversations through Microsoft Clarity session replay. Munoz v. LG Electronics, filed in October, alleges that FullStory kept recording after an opt-out; the allegations are unproven. No practice moved materially. The movement this cycle was in the evidence, not in where the practices stand.

Key Tensions

  • Drafting got cheap, review did not. Atlassian's survey of 1,000 product professionals found 82% say teammates produce low-effort AI work that has to be revised, and 69% say decision-making has not sped up. Figma's ratio of 17 times the code to 1.3 times the releases is the same problem at product scale. The teams getting reliable output, such as Callstack with its sub-2% pixel-diff gate and the in-house A/B agent whose model computes no numbers, spend their effort on verification rather than generation.

  • Written guidance versus enforced constraints. AGENTS.md files, pasted tone-of-voice guides and rules lists describe a system without holding agents to it, as NordVPN's 41-of-57 result and Definition's eight-brand test show. Even exact-match rules leak: in one comparison a ban on 'guarantee' let 'guaranteed' through. The answer the field is converging on is machine enforcement, through lint and type checks set to error in CI and MCP servers backed by a system of record, as IBM's Carbon, SAP and PatternFly now publish.

  • Confident gaps. Models settle decisions nobody made, and the result reads as finished. ChatPRD filled in both deliberately omitted decisions in a vendor's comparison, chat assistants invent RICE reach and impact values when a backlog lacks them, and AI microcopy promises behaviour the product does not have. Tabelog's switch from AI-drafted requirements to AI-raised questions, which cut more than 50 open items to 8, shows where the value actually lies.

  • Data readiness decides returns. A survey of 10,000 businesses cited by Pendo found only 8% have data fully ready for AI. Of those, 81% report broad or strong ROI, against 12% of businesses with partially ready data. Atlan's figures, analytics-agent accuracy rising from 21% to 95% with a semantic layer, make the same point, and none of GA4, Amplitude or Mixpanel publishes an accuracy rate for its assistant.

  • The observed user is shrinking and changing. Consent rules are narrowing the behavioural record: CNIL's draft guidance treats session replay as a separately consented purpose, and Munoz v. LG tests whether consent flags hold mid-session. At the same time, agents are becoming visitors. Microsoft Clarity counted 5.23 million agentic sessions between mid-August and early September, and MCP-based agents leave no trace in traditional replays, so journey maps and frustration scores built from human clickstreams increasingly describe a self-selected subset of the people actually using a product.

Top 10 Evidence Items

  1. Figma Q3 2026 Earnings Call Transcript - Figma (NYSE:FIG) (news-coverage) — Figma's own 17x-code-to-1.3x-releases figure is the single number the summary uses to pin down the drafting-versus-outcomes gap. https://www.benzinga.com/news/26/11/62900371/figma-q3-2026-earnings-call-transcript
  2. who approved that popup? (opinion) — The 41-of-57 result is the clearest demonstration that written guidance (AGENTS.md) doesn't bind agent behaviour, motivating the push to machine enforcement. https://prodfolks.substack.com/p/who-approved-that-popup
  3. Requirements Engineering with Generative AI: A Systematic Literature Review of Using AI Agents for Requirements Engineering Automation (research-paper) — An independent systematic review finding zero production deployments is the strongest evidence that judgement-heavy requirements work hasn't matured. https://www.mdpi.com/2673-4052/7/5/169/pdf
  4. Should an LLM Review Your User Stories? A 182-Story Test Says Not Alone | Specira AI (opinion) — Shows the judgement-call failure mode concretely: higher-capability GPT-5 produced more false positives than the smaller model, undermining a 'bigger model fixes it' assumption. https://www.specira.ai/blog/llm-user-story-quality-review
  5. The Hidden Work (and Sweat) Powering AI Productivity (news-coverage) — Direct evidence for the 'drafting got cheap, review did not' tension, with Atlassian's own numbers on unsped-up decisions and padded output. https://www.newsweek.com/atlassian-ai-productivity-hidden-work-12641692
  6. A/B Testing Tools for Low Traffic: Bayesian Options Compared (opinion) — A vendor's own simulation showing its recommended Bayesian threshold multiplies false winners four-fold is a rare case of a vendor publishing evidence against its own method. https://www.octoagency.io/blog/ab-testing-tools-low-traffic/
  7. Evaluating Large Language Models for Web Accessibility Remediation in Real-World Websites (research-paper) — Independent, peer-reviewed confirmation that AI accessibility remediation fails precisely where judgement is needed (semantics) and fabricates content absent clues. https://dl.acm.org/doi/10.1145/3797867.3841243
  8. Nearly 3 in 4 enterprises hear customer complaints on AI personalization (adoption-metric) — Deflates the vendor-reported personalisation gains elsewhere in the briefing by showing most errors reach teams only via customer complaints, not monitoring. https://www.fullstory.com/blog/ai-powered-personalization/
  9. Lumivero Report: Qualitative Researchers Adopting AI, But Lack Confidence In It (adoption-metric) — Captures adoption without confidence directly: regular use stuck at 21% with distrust as the named barrier, a sharper read than any raw usage statistic. https://www.socialsciencespace.com/2026/10/lumivero-report-qualitative-researchers-adopting-ai-but-lack-confidence-in-it/
  10. Munoz v. LG Electronics: the lawsuit built on a half-second consent flip (news-coverage) — Represents the regulatory/legal pressure narrowing the behavioural record that session-replay and analytics tools depend on, even though the claims are unproven. https://consentpixel.com/blogs/munoz-lg-session-replay-consent-flag/