Perly Consulting │ Beck Eco

The State of Play

A living index of AI adoption across industries — where established practice meets the bleeding edge
UPDATED DAILY

The AI landscape doesn't move in one direction — it lurches. Some techniques leap from experiment to table stakes in a single quarter; others stall against regulatory walls, technical ceilings, or organisational inertia that no amount of hype can dislodge. Knowing which is which is the hard part. The State of Play cuts through the noise with a rigorously maintained index of AI techniques across every major business domain — classified by maturity, evidenced by real-world adoption, and updated daily so you always know where you stand relative to the field. Stop guessing. Start knowing.

The Daily Dispatch

A daily newsletter distilling the past two weeks of movement in a domain or two — delivered to your inbox while the index updates in the background.

AI Maturity by Domain

Each dot marks the weighted maturity of practices within a domain — hover for a brief summary, click for more detail

DOMAIN
BLEEDING EDGEESTABLISHED

Architecture documentation & specification writing

LEADING EDGE

TRAJECTORY

Advancing

AI that generates architecture diagrams, system design documents, and technical specifications from codebases and requirements. Includes C4 diagram generation and design doc drafting; distinct from code documentation which targets inline and API-level references.

OVERVIEW

AI-assisted architecture documentation has crossed an inflection point: AI agents now consume 66% of documentation platform traffic, a shift from the 45% threshold reached earlier in 2026. Claude Code alone generated 199 million documentation requests in a single month, confirming machine-readable architecture documentation as primary consumption layer for AI systems. Yet production deployments reveal a hard ceiling: peer-reviewed benchmarks show near-zero accuracy on complex diagrams beyond 30-40 components, models lack pragmatic architectural reasoning, and hallucination rates across benchmarks span 0.7-94% depending on task complexity. This creates an uncomfortable reality: simple artifacts (service diagrams, ADRs, draft specifications) benefit from automation, while complex systems architecting and documentation maintenance demand human judgment. Senior engineers are shifting from craft execution to architectural governance: specification writing, constraint design, and invariant definition now consume 4.8 hours per workday as AI accelerates code generation. Tools can generate diagrams and draft specifications at scale; Google's deployment demonstrates autonomous agents can identify critical system-level issues when architecture documentation is committed to CI/CD pipelines. Documentation drift—the gap between live code and documented architecture—has accelerated from weekly to daily misalignment in AI-accelerated teams. Real deployments surface harder problems: untraced AI-generated code (engineering teams found significant shares with no requirement linkage after six months), silent specification drift where agents execute stale specs without noticing actual API divergence, and hallucinated compliance requirements appearing correct but violating standards. Specification-driven development frameworks promise precision but reveal a fundamental limitation: they automate syntax but not semantics—specs give guardrails but guardrails are not judgment. The compensating discipline remains specification engineering: structured, machine-readable specifications as executable contracts, but now paired with information architecture discipline (taxonomies, semantic layers, controlled vocabularies) as foundation for reliable agent reasoning. Architecture documentation transitions from passive reference to active control interface, constrained by the reality that AI throughput gains routinely pair with increased change failure rates unless architectural governance is explicit.

CURRENT LANDSCAPE

Commercial tooling matured sharply in April 2026. Mintlify announced a $500M Series B valuation, revealing that 45% of its documentation traffic now comes from AI agents—significantly exceeding human browser access at 46%. Claude Code alone generated 199 million documentation requests in one month. The platform serves 100+ million monthly users across 20,000+ customers including Microsoft, Anthropic, Coinbase, and PayPal, with $10M ARR at end of 2025 (10x growth YoY). Eraser continues ecosystem expansion with official AI agent integrations (Claude Code, Cursor, Windsurf) and community MCP servers. Architecture diagram generation has become a crowded market segment: 10+ active AI diagram platforms now offer prompt-to-diagram generation, cloud infrastructure templates, CI/CD integration, and version-tracked living documentation—signaling ecosystem maturity and scale of AI-assisted architecture visualization. Architecture-specific case studies are emerging: Google deployed autonomous AI agents to generate ARCHITECTURE.md files across a microservices mesh, with AI-powered CI/CD quality gates identifying critical system-level issues (distributed tracing blackouts, storage leaks) undetected for months—demonstrating that architecture documentation can serve as an automated reasoning layer for infrastructure assurance. Adobe Commerce describes a four-phase production methodology (capture, gap analysis, documentation, ticket linkage) reducing discovery-to-developer-ready timeline from weeks to days with the architect as editor-in-chief—validating the pattern where AI handles high-volume synthesis and architects focus on judgment and validation.

However, real-world deployments surface deeper failure modes that tools cannot solve. Documentation drift has accelerated from weekly misalignment to daily architectural divergence in AI-accelerated teams. More critically: engineering teams conducting traceability audits six months into AI-assisted development found significant AI-generated code had no requirement linkage, making specification compliance enforceable only through governance, not automation. Apiiro reports a 322% spike in privilege-escalation flaws in repositories with high AI contribution rates, signaling architectural security gaps. Specification drift—where documented APIs diverge from actual behavior—becomes a silent failure mode because AI agents execute exactly what specs say without noticing discrepancies humans would catch; 46% of engineering teams cite integration with existing systems as their primary deployment blocker. Architectural debt is rising with AI-assisted code generation—teams produce code faster than architecture evolves to accommodate it—requiring portfolio-wide architecture visibility and continuous documentation as a governance discipline. The hard constraint surfaces: AI models hallucinate compliance requirements with plausible confidence; larger models show elevated hallucination rates relative to smaller ones, and verification cannot be automated—requiring separate validation pipelines. Specification-driven development frameworks (GitHub Spec Kit, OpenSpec, BMAD) position architecture specifications as executable machine-readable contracts: Sheriff linting rules, API schema validation, and automated constraint checks guide agent reasoning and provide deterministic feedback. DORA and Forrester research confirm: uneven adoption of specification governance raises throughput but increases change failure rates; the productive pattern pairs architectural judgment (review, trade-off evaluation, constraint engineering) with AI-driven artifact generation under explicit specification control.

TIER HISTORY

ResearchJan-2023 → Jul-2024
Bleeding EdgeJul-2024 → May-2026
Leading EdgeMay-2026 → present

EVIDENCE (124)

— Critical finding: SDD automates syntax but not semantics. Specs enable interfaces and scaffolding but cannot capture shared intent or evolving business context—documenting hard ceiling on automation-first approaches.

— 66% of documentation traffic now from AI agents (213M requests in July alone); 52-point growth in 7 months. Signals architectural docs must be optimized for agent consumption as primary reader.

— Meta-analysis: 95% AI pilots fail, 5% achieve ROI, 861% code churn post-adoption. Contextualizes why robust specification discipline and architecture documentation governance become critical as counterbalance.

— Hallucination benchmarks across 26+ models: 0.7-94% range depending on task and model. Quality concern directly affecting reliability of AI-generated architecture documentation and specifications.

How to Do Spec-Driven DevelopmentCase Studies

— Larridin case study: specification structure explicitly includes Architecture section (data model, module boundaries, interfaces). Production workflow treating architecture specs as executable contracts.

— Anthropic's context engineering principles for Claude 5: shift from simple specs to rich references (test suites, rubrics, full codebases). Demonstrates evolution of specification writing for effective agent guidance.

— Information architecture (taxonomies, semantic layers, controlled vocabularies) is foundation for AI system reliability. Shows why architecture documentation structure and metadata matter for agent reasoning.

— Mintlify deployed 'Docs from GitHub' feature with Daytona infrastructure: 1,000+ parallel sandboxes, <90ms creation times, 3 months engineering time saved. Multi-agent orchestration generating production-ready documentation at scale.

HISTORY

  • 2023-H1: Research into automated architecture documentation validation gained visibility through ICSA 2023 publication on inconsistency detection. C4 model adoption visible in practitioner tutorials and vendor tooling. Diagram-as-code approaches (PlantUML) emerging for version control integration.

  • 2023-H2: SARIF architecture recovery research published (arXiv, 36.1% accuracy improvement). Practitioner tutorials demonstrate hands-on C4/Structurizr DSL adoption for versioned documentation. Practitioner discourse highlights documentation's role in mitigating technical debt from AI-generated code, renewing focus on spec-driven development practices.

  • 2024-Q1: Commercial AI diagram tools enter product-GA phase (e.g., AI Diagram Maker). Conversational diagram generation reduces production time from 30+ minutes to 20 seconds, beginning to address the manual labor bottleneck in diagram creation. Practitioner adoption of diagrams-as-code remains steady, supported by open-source tooling.

  • 2024-Q2: Research continues on automated architectural knowledge extraction and organization using generative AI, with ICSA 2024 poster demonstrating techniques for mining architecture information from dispersed sources (code, logs, documentation). The practical challenge remains addressing the organizational problem—many teams still lack systematic architecture documentation despite emerging tooling.

  • 2024-Q3: DORA 2024 survey confirms majority developer adoption of AI for documentation tasks; Mintlify achieves 3,000 customer traction and $18.5M Series A. Critical gap emerges: Zhejiang University benchmarking shows AI achieves only 55-65% accuracy on diagrams vs. 82% human performance. Accessibility experts document stability and compliance failures in AI-generated technical artifacts. Quality assurance becomes the blocking factor as review overhead offsets creation time savings.

  • 2024-Q4: Commercial adoption accelerates: Mintlify quintuples customer base with Fortune 500 deployments (Anthropic, Cursor, Perplexity). AWS releases Amazon Q Developer for diagram generation. DORA 2024 final report quantifies productivity-stability trade-off: 25% AI adoption → 7.5% documentation quality gain but 1.5% throughput loss and 7.2% stability decrease. EU AI Act spurs academic work on automatable compliance documentation. Empirical testing reveals persistent hallucinations in AI diagram generation; model collapse risk emerges as long-term threat to AI training data quality.

  • 2025-Q1: Commercial consolidation continues: Mintlify scales to 15+ named enterprise customers serving 2M+ monthly developers. Eraser AI demonstrates production ROI (10x diagram speedup, documentation scaling). However, industry-wide AI failure rates spike to 42% of businesses scrapping initiatives due to specification and governance gaps. Tooling advances: CI/CD-native diagram automation (Eraser, Amazon Q) reduces documentation drift. Practitioners adopting AI for Architecture Decision Records document productivity gains alongside persistent limitations (hallucinations, context loss). Quality assurance and human-in-the-loop governance remain critical blocking factors for scaled deployment.

  • 2025-Q2: Limited new evidence emerges, with architectural tooling gains focused on incremental improvements. Eraser.io tutorial documentation highlights DiagramGPT feature maturity for natural-language-to-diagram generation and diagram-as-code approach with CI/CD synchronization, confirming continued emphasis on reducing manual drift. Commercial API documentation platforms (Mintlify, Scalar, Bump) dominate market discourse; architecture documentation remains secondary narrative. Industry evidence scarce—suggests either consolidation phase or temporary pause in narrative generation within this specialized segment.

  • 2025-Q3: Peer-reviewed research (ASEM 2024) evaluates ChatGPT's diagram generation capabilities, finding competent outputs for simple diagram types but significant limitations on complex systems architecting scenarios. Vendor analysis (IcePanel) reveals persistent architectural reasoning gaps: LLMs design like junior programmers, fixating on popular technologies over pragmatic choices. Diagram software market shows continued growth momentum ($843M in 2024 to projected $1.8B by 2031). Limited public evidence reflects ongoing consolidation in the commercial tooling space; research focus remains on capability assessment rather than large-scale production deployments.

  • 2025-Q4: Spec-driven development emerges as key methodology, with ThoughtWorks and multiple practitioners analyzing AI agents' role in transforming specifications into implementation. Commercial tooling advances: AI-powered ArchiMate modeling reduces documentation cycles from weeks to hours. Market discourse shifts toward documentation-as-machine-readable-infrastructure; Mintlify positions documentation as 50% AI-optimized. Critical counterpoint surfaces: scaling challenges in spec-driven development reveal fundamental limitations—natural language ambiguity, AI's lack of contextual reasoning, and architectural judgment gaps remain persistent obstacles. Ethics research highlights hallucinations, bias, and IP concerns as adoption blockers. Stack Overflow survey data shows 84% developer adoption of AI tools but only 46% favorable sentiment, citing accuracy concerns. Deployment momentum continues but governance and quality assurance requirements intensify.

  • 2026-Jan: AI-powered architecture documentation tooling reaches production maturity with ecosystem expansion. Eraser launches official AI agent integrations (Claude Code, Cursor, Windsurf) for IDE-native diagram generation; community extends tooling via open-source MCP servers. Mintlify scales to 20M monthly users serving enterprise customers (Microsoft, Anthropic, Coinbase) demanding machine-readable documentation as AI agent input. Product development accelerates: Mintlify adds repo-based auto-generation and multi-modal assistant input. However, negative signals intensify: peer-reviewed research finds GenAI image models achieve only 42% accuracy on architectural visuals (vs. 82% human baseline); practitioner research surfaces risks of homogenization, authorship loss, and contract/regulatory constraints limiting real-world adoption. Vendor momentum remains strong but governance challenges and quality assurance requirements deepen.

  • 2026-Feb: Enterprise documentation tooling consolidates around specification discipline as coordinating practice. Mintlify advances with enterprise security features (SSO, RBAC) signaling mature B2B adoption; independent benchmarking reveals AI models fail on complex diagrams at scale (near-zero accuracy beyond 30+ components), reinforcing need for human-in-the-loop governance. Practitioner consensus crystallizes: specification engineering—the discipline of writing agent-executable blueprints—emerges as the critical bottleneck and compensating control for reliable AI-assisted architecture work. Comparative analysis surfaces trade-offs: AI-native platforms gain speed but lose collaborative depth and approval workflows.

  • 2026-Mar: Practical deployment evidence surfaces as critical practice maturation marker. Helicone (16k+ organizations, 14.2 trillion tokens processed) documents that documentation quality ('the knowledge layer') is the limiting factor for AI system performance—not model capability. First unified benchmarking platform (ArchBench) launches for measuring LLM capabilities on architecture tasks (ICSA 2026), establishing research infrastructure for the practice. Commercial maturation accelerates: Ardoq GA releases AI Chat for architecture data querying and AI Visual Importer for diagram-to-structured-data conversion, with Tenneco case study eliminating 1.25 FTE through AI-assisted workflows. Apache SkyWalking documents how AI economics reshape architecture decision-making: runnable PoCs become cheap enough that architects can pursue optimal designs instead of early compromises. However, critical signals persist: specification synchronization challenges documented by practitioners; hallucinations and knowledge-cutoff remain blocking factors for specification generation; organizational prerequisites limit SDD applicability outside founder-led contexts. Market assessment shows 80%+ of enterprise architecture artifacts currently unstructured but AI-driven consolidation tools achieving 94.4% accuracy on document parsing, with ROI evidence (15% IT cost reduction) in regulated industries.

  • 2026-Apr: Spec-driven development consolidates with measurable deployment evidence and research validation. Talk Think Do publishes Q1 2026 AI Velocity Report showing 84% AI-authored code with OpenSpec achieving 40-50% faster delivery and 55% cost advantage in competitive tender. Peer-reviewed research (OmniDiagram, ACL 2026) establishes SOTA benchmarks for AI diagram code generation with 196k-instance dataset and RL-based visual feedback validation, validating technical feasibility of specification-driven diagramming. Independent third-party evaluation documents 70% reduction in C4 diagram creation time in enterprise production deployment. Palo Alto Networks engineer evaluates three SDD frameworks (BMAD, Spec-Kit, OpenSpec), finding OpenSpec highest-scoring (4.0/5) for specification quality and AI tool compatibility. Pulumi and Forte Group demonstrate CI/CD-integrated diagram automation and strategic positioning of Spec Layer as durable constraint interface for AI execution. Late-month signals reinforce commercial and production maturity: Mintlify announced $500M Series B valuation with 45% of documentation traffic now from AI agents (Claude Code alone generated 199M requests in one month), confirming machine-readable documentation has crossed the threshold where AI agent consumption exceeds human browsing; Google deployed autonomous agents generating standardized ARCHITECTURE.md across a microservices mesh, with AI-powered CI quality gates catching critical issues (distributed tracing blackout, storage leak) undetected for months; ICLR 2026 Text2Arch research validated fine-tuned models matching GPT-4o on scientific architecture diagram generation; practitioners documented that AI coding accelerates documentation drift from weekly to daily misalignment, intensifying demand for automated synchronization. Emerging pattern: specification engineering transitions from theoretical discipline to operational practice across consulting, enterprise, and tooling sectors, with commercial scale (Mintlify 20,000+ customers) and production CI/CD deployments (Google) validating the shift from AI-as-autonomous-architect to AI-as-implementation-executor-constrained-by-specs.

  • 2026-Jun: Spec drift and traceability failures surface as the defining operational risk. Jama Software documented a real failure case where engineering teams found a "significant share of AI-generated code had no traceable requirement linkage" after six months, confirming SDD traceability as compliance-critical in regulated domains. Adobe Commerce published a four-phase AI-assisted architecture methodology compressing discovery-to-developer-ready from weeks to days, validating the architect-as-editor-in-chief pattern at named enterprise scale. Augment Code's SDLC analysis (citing DORA and Forrester) frames specifications as the AI-era control plane: AI throughput gains routinely pair with increased change failure rates unless architectural governance is explicit. Angular Architects practitioner guide demonstrates architecture documentation as executable contract—AGENTS.md and Sheriff linting rules serving as machine-readable constraints guiding AI coding agents with deterministic validation. Critical negative signal: 46% of enterprise teams cite integration with existing systems as primary AI deployment blocker, and spec drift (agents executing stale specs without noticing actual API divergence) is identified as the predominant silent failure mode in agentic development workflows.

  • 2026-May: Specification-driven development confirmed as mainstream coordinating discipline, with governance urgency intensified by AI-accelerated architectural debt. State of Docs Report 2026 (1,131+ practitioners) documents 76% of technical writers using AI regularly—up 16 YoY—with SDD tooling (AWS Kiro, GitHub Spec Kit at 93k+ stars, OpenSpec, BMAD) reaching enterprise production use. SDLC AI Radar 2026 (LTM analyst report) identifies specifications, context engineering, and architectural judgment as critical rigor shifts in AI-native SDLC, directly validating specification-driven architecture as the coordinating practice. Architectural debt rising with AI adoption: SIG analysis documents AI coding tools lack architectural context and produce hard-to-change software, positioning portfolio-wide architecture visibility as essential governance—not optional tooling. AI architecture diagram generator ecosystem matured to 10+ platforms with prompt-to-diagram, cloud templates, CI/CD integration, and version tracking. Reversa framework (arXiv May 2026) establishes reverse documentation engineering for converting legacy software into operational specifications for AI agents—extending specification practice to existing systems. Critical boundary signals persist: formal governance model documents productivity-reliability paradox (20-56% gains in controlled studies vs. 19% slowdown in RCT); 60% of AI pilots generate no value outside high-maturity specification contexts. Synthesis: commercial scaling (documentation AI adoption mainstream) confirmed, but specification-driven architecture's value depends on organizational maturity to write and maintain effective constraints—the requirements problem remains unsolved.

  • 2026-Jun/Jul: Specification-driven development tooling reaches vendor consolidation and real deployment signals. Microsoft GitHub Spec Kit GA confirms enterprise SDD maturity with named deployment: brownfield asset-type onboarding compressed from 2–3 weeks to days using parameterized specifications, validating production ROI. Critical balance shift in evidence base: Pluralsight's 6-level maturity model and Brief HQ's analysis expose the hard limits of SDD automation: specifications address delivery speed but cannot solve upstream decision clarity, and governance discipline requires tracking decisions outside version control—Brief HQ quantifies waste at 82 cents of every AI coding dollar never reaching production (44¢ bug fixes, 27¢ rework, 11¢ friction). Real failure documented: deployment teams conducting traceability audits found significant AI-generated code with zero requirement linkage after six months—demonstrating that SDD frameworks (GitHub Spec Kit, OpenSpec, BMAD) are necessary but insufficient without organizational rigor on documentation freshness and decision tracking. Technical barriers persist: empirical testing shows AI diagram generation fails on XML syntax (unclosed tags, spatial positioning) and accuracy collapses on diagrams >20 components, necessitating manual refinement loops that offset creation speed gains. Market inflection: AI agents now consume 45-50% of documentation traffic, exceeding human browsing; GitBook identifies 2026 as the inflection point where AI agents are the primary documentation consumer—poorly structured docs are skipped entirely, driving platform consolidation around dual optimization for human and machine readers. Anthropic's Claude Code documentation team operationalizes feedback-driven documentation improvement via automated agent PR generation (Mintlify), showing signal velocity becomes viable only with automated triage at scale. Emerging synthesis: architecture documentation transitions from static reference to active control interface; tooling maturation (GitHub Spec Kit, AWS Kiro, Eraser integrations) enables automation but value realization depends on organizational capacity for specification engineering discipline and continuous decision tracking.

  • 2026-Jul: Deployment evidence and ecosystem consolidation accelerate. ZOZO (major e-commerce) deployed Claude Code for automated architecture diagram generation with CI/CD integration, achieving >50% reduction in diagram creation time and improved code-architecture consistency—demonstrating production-stage adoption with measurable productivity gains. Eraser published MCP Server v0.2.0 enabling agent-driven diagram generation and editing as artifacts, signaling product maturity for agentic architecture documentation. Independent ecosystem growth visible: open-source drawio-skill reached 6.2k GitHub stars, LikeC4 achieved 22k VSCode extension installs, and Classmethod documented engineering constraints and vendor-specific solutions for cloud platform icon integration. Market analysis shows documentation platforms now compete on AI-readiness as first-class differentiator (llms.txt, MCP analytics, agent-aware content structure). Critical negative signals persist: research documents domain-specific hallucination rates (69-88% in legal domains) creating adoption barriers; game-theoretic approaches demonstrate 79.46% hallucination reduction is achievable through structured reasoning but requires specialized training and domain-specific techniques. Quantified metrics from Mintlify show 64% improvement in AI agent answer precision and 50% token reduction from better documentation structure, confirming that architecture documentation quality (not model capability alone) limits AI agent performance in integration scenarios. A comprehensive buyer's guide (Forasoft) assessed the maturing vendor landscape across eight distinct AI-native architecture categories—diagram generation, ADR authoring, threat modeling, fitness functions—benchmarking 250+ products, confirming the practice has moved from scattered point tools to a categorized, comparable market.

  • 2026-Aug: Mintlify's midyear report showed AI agents now generate 66% of documentation traffic (213M requests in July, up 52 points in seven months), while Anthropic's Claude Code context-engineering shift—stripping 80% of its system prompt in favor of rich references (test suites, full codebases)—signaled that effective specs increasingly mean grounding material for agents rather than static prose. Countervailing critique (spec-driven-development ceiling analysis, Larridin case study) argued SDD automates syntax but not semantic intent, while industry data (95% AI pilot failure rate, 861% code churn) reinforced the case for rigorous architecture documentation as a governance counterweight.