The State of Play

— Negative: SHAP and gradient saliency on Meta's Prompt Guard 2 guided edits that flipped its predictions and sometimes jailbroke the LLM. Explanations can lower the cost of attacks.

— Malware triage prototype that uses SHAP–LIME agreement as a confidence signal and disagreement as uncertainty, reaching 96.17% accuracy on 1,080,000 EMBER 2024 samples.

— A 100-participant Co-12 evaluation finds SHAP suits developers while LLM explanations suit end users. This supports audience-specific hybrid explanations.

— University of Melbourne benchmark: LIME and SHAP produce conflicting explanations on 1,200 loan applications; LIME identifies features coefficient analysis and SHAP dismiss, undermining assumption that XAI provides stable explanations.

— Market research: XAI market growing from $9.1B (2025) to $52.9B (2034) at 21.7% CAGR; regulatory mandates (GDPR, EU AI Act, US GAO) drive adoption across fraud/healthcare/finance; model-agnostic methods (SHAP/LIME) command 64.8% share.

— BNY Mellon enterprise case: LLM self-reported decision factors (Spearman ρ ~0.35) weakly correlate with causal drivers; proves LLM explanations are post-hoc rationalizations, not reliable audit evidence for compliance.

— Named org governance: TransUnion operationalises explainability via reason codes, model monitoring, and drift detection in regulated credit/fraud/identity decisions; identifies governance as non-negotiable for regulated AI deployment.

— Governance framework: regulatory audits cite insufficient decision traceability ($4.5M avg penalty); proposes 4-layer interpretability architecture (logging, attribution, documentation, audit access) to prevent post-deployment compliance failure.

— Manufacturing production deployment: XGBoost+SHAP model (ROC-AUC 0.928) integrated with Power BI dashboard; expert validation confirmed interpretability enables procurement teams to operationalise AI-driven risk decisions.

— Dual-cohort ICU clinical study (18,526 MIMIC-IV + 314 external validation): XGBoost+SHAP identified clinical mortality drivers (BUN, age, WBC, PT); external validation across independent hospital demonstrates robustness.

— Critical analysis: SHAP's formal guarantees (local accuracy, missingness) fail in malware; dilutes feature credit across correlated features and reverses attribution signs with distribution changes—domain-specific limitation undermining security auditing.

— Telecom industry deployment (7,043 customers): XGBoost+SHAP+Tableau framework (80.41% accuracy) identifies churn drivers (contract type, tenure); demonstrates explainability+visualisation converting model outputs to business insight.

— Product launch for agentic AI in banking with named deployments (KeyBank, Paragon Bank, Axos Bank, Grihum) emphasising explainability and auditability as core architectural requirements, not bolt-on features.

— SIGIR 2026 paper on fine-grained XAI for patent novelty prediction addressing explainability gaps via constituent-element-level passage retrieval with auditable mappings to prior art.

— Philips patent on interpretable medical imaging combining saliency maps with segmentation for clinical explainability precision; signals 4-year product development cycle positioning vendor to commercialise XAI in clinical decision support.

— Systematic review of 116 studies on XAI in precision medicine documenting SHAP, attention mechanisms, and saliency maps in cancer and biomarker domains; identifies clinical validation as primary adoption barrier despite transparency gains.

— Practitioner critique arguing LIME/SHAP visualisation demos are 'theatre' without replayable inspection objects; critical assessment documenting that current XAI methods fail auditability requirements for data governance workflows.

— Governance-focused practitioner guidance positioning explainability as control-plane requirement with audit-ready artifacts, scenario testing, and explanation reproducibility for high-impact decisions.

— Audit consulting framework embedding explainability and interpretability as regulatory controls with specific drift/bias thresholds (PSI >0.25) and annual audit frequencies for high-risk models under SR 11-7 and EU AI Act.

— Case study of insurance fraud model with perfect SHAP explanations but wrong decision, documenting XAI's blind spot: explains reasoning but not upstream knowledge or policy integrity, critical limitation for high-stakes governance.

— Peer-reviewed LIME vs SHAP comparison for intrusion detection on UNSW-NB15 benchmark (95.3% accuracy); cautions that explanation-guided improvements require downstream workflow changes, not automatic performance gains.

— Google Research study: advanced LLMs encode 95-98% of facts but retrieve only 26-34% directly; reveals XAI testing blind spot—explanations incomplete without retrieval-robustness testing across word-order variants.

— Frontiers systematic review (PRISMA-guided, 2014–2025, 21 studies) documenting 39-52% adoption of SHAP and LIME in GAN-based IoT intrusion detection; identifies critical gaps in explanation stability, real-time scalability, and cross-dataset robustness limiting production deployment.

— Enterprise survey (600 CIOs, April-May 2026 fieldwork): 92% have been asked to defend AI outcomes they cannot fully explain; 85% report explainability gaps have already delayed or blocked production deployment due to inadequate audit trails and decision logs.

— Empirical user study (N=200) revealing explanation fidelity exhibits ceiling effect at 70-85% correctness; even fully correct explanations fail to guarantee human understanding (bimodal outcomes)—challenges core assumption that improving XAI method fidelity alone solves governance problems.

— CTO at major automotive OEM (covering commercial vehicles, passenger, electric vehicle divisions) articulating explainability as production-deployment requirement alongside regulatory and cyber governance; emphasizes reversibility and human-in-the-loop as non-negotiable governance mandates.

— Multi-jurisdictional regulatory adoption signal: 65% of Kenyan commercial banks deployed AI for credit-risk assessment with SHAP-based decision explanation; Central Bank approved 227 Digital Credit Providers under explainability and transparency requirements; EU enforcement deadline extended to December 2027.

— Critical technical analysis distinguishing mechanistic interpretability achievements from unproven assumptions: documents coverage gap (methods explain only a fraction of computation), verification gap (ground-truth feature enumeration exists only at toy scale), and frontier model superposition remains inference, not proof.

— UAI 2026 framework operationalizing SHAP concentration as pre-deployment governance control for model-risk assessment; validated across 16 multiclass tasks in 9 domains (supply chain, healthcare, finance), establishing SHAP-based diagnostics as actionable operational tool for preventing distribution-shift failures.

— Nature Medicine study (MIT, Stanford, Columbia) reveals critical limitation: explainability increases automation bias and overreliance, especially in non-expert users; LLM explanations most prone to deference—users weakest without AI most susceptible to harm—contradicting assumptions that explanation improves outcomes.

— Sichuan University and Tongji University deployed SHAP with XGBoost for fly ash geopolymer prediction (355 samples, R²=0.930 validation); experimental verification achieved 2.41 MPa mean error, demonstrating SHAP enabling both transparency and actionable domain optimization in materials science.

— IBM watsonx.governance reaches production maturity with local explanations, feature attribution, and LLM token-level provenance; deployed in regulated industries (banking, insurance, healthcare) for EU AI Act compliance and data exposure risk reduction.

— Applied SHAP to seismic severity classification on 75 years of USGS data (1947-2022, 91% accuracy); SHAP diagnostics revealed hard physical boundary—secondary sensors fail during extreme events—providing actionable infrastructure insight beyond regulatory compliance.

— Taibah University and Higher Colleges of Technology embedded SHAP/LIME into CNN for multiclass lesion detection in wireless capsule endoscopy (3,301 images, precision=recall=0.97); interpretability designed to enable clinician trust and workflow integration in high-stakes diagnostics.

— ACL 2026 empirical study (6k annotated segments) validates SHAP-computed Shapley values outperform LLM-generated explanations in faithfulness; deletion tests confirm SHAP-identified features reliably drive predictions while LLM rationales show inconsistent influence and poor transfer across architectures.

— Systematic evaluation of 13 XAI methods (SHAP, LIME, Grad-CAM) across 4 ECG classifiers against clinical guidelines; 9 methods performed below chance on at least one cardiac condition—gradient-based techniques systematically misrepresent model behavior, raising fundamental trustworthiness concerns in medical deployment.

— Stanford HAI's 2026 AI Index documents critical market divergence: model capabilities rising (SWE-Bench 60→100%) while transparency disclosure fell (Foundation Model Index 58→40); major vendors (OpenAI, Anthropic, Google) ceased publication of training parameters, dataset sizes, and training costs—opacity rising despite regulatory pressure.

— Peer-reviewed systematic survey at ACL 2026 documenting five design paradigms for intrinsic interpretability in LLMs (functional transparency, concept alignment, representational decomposability, explicit modularization, latent sparsity induction), signaling mainstream field recognition.

— Regulatory enforcement signal: EU AI Act high-risk system requirements (transparency, audit trails, human oversight) become fully enforceable August 2, 2026, with €35M penalties or 7% global turnover; explainability now legally mandated for compliance-sensitive domains.

— Frontiers healthcare deployment case study using SHAP for clinical outcome prediction across 15,420 patients; ensemble ML achieved AUC ≈0.94 for weight loss and 0.79 for glycemic control with SHAP feature attribution enabling personalized treatment strategies.

— ACL 2026 position paper documenting critical maturity gap: mechanistic interpretability field lacks standardized auditing frameworks, limiting adoption in safety-critical applications requiring correctness guarantees.

— IBM showcased production-grade vLLM-Hook mechanistic interpretability plugin at ICML 2026 enabling inspection and intervention on LLM internal states; demonstrated use cases include prompt injection detection, retrieval enhancement, and activation steering.

— Technical analysis operationalizing mechanistic interpretability for governance: J-space research enables reading and auditing LLM deliberate reasoning through internal workspace probing; moves interpretability from philosophy to instrumentation-grade auditability.

— Survey of UK CIOs: 84% report explainability gaps have delayed or blocked AI projects; 97% exploring agentic AI but only 36% have centralized governance. Adoption barrier evidence showing interpretability/explainability remain material obstacles to AI deployment at scale.

— Empirical study of 15 cardiovascular professionals comparing SHAP, counterfactual explanations, and Anchors: no single method meets clinician needs; recommends layered explanation designs combining quick overviews with optional deeper insights. Practical deployment constraints in high-stakes healthcare.

— Production systems optimize for speed with explainability as afterthought: decision outputs logged but reasoning is not. Structural governance gap in deployed AI; EU AI Act enforcement (August 2026) now requires mandatory logging, traceability, and human oversight infrastructure that most systems lack.

— Risk-tiered enterprise XAI implementation guide addressing emerging agentic AI challenge: agents emit step-by-step reasoning traces, verifier models check logic, and all tool calls logged to tamper-evident audit trails. Explainability infrastructure extends beyond post-hoc methods to trace decision workflows.

— Controlled experiment (921 subjects) reveals feature-importance explanations increase overconfidence in flawed models (−0.14 SD financial outcome), while uncertainty-aware explanations reduce overconfidence (+0.29 SD benefit). Critical maturity evidence: XAI methods have asymmetric effects depending on model quality, requiring design interventions.

— NIST AI RMF (updated July 7, 2026) establishes explainability and interpretability as foundational trustworthiness characteristics. Authoritative governance standard adopted widely by enterprises as de facto requirement for AI deployment.

— Dataiku/Harris Poll survey of 800 senior data executives: 95% lack full visibility into AI decision-making. Adoption metric documenting enterprise governance gap despite widespread AI deployment; explainability remains critical barrier to scale.

— Bank of England analysis: SHAP, LIME, and PDP all assume feature independence—a violation in real-world financial models with correlated features. Median rank-agreement coefficients collapse from 0.95 to 0.3–0.8 under correlation; Apple Card example shows attribution methods miss discriminatory outcomes.

— Comprehensive taxonomy of SHAP/LIME applied to malware detection across Windows PE, PDF, Linux, and hardware platforms; demonstrates XAI deployment breadth for actionable security decision-making.

— TokenSHAP applied to understand LLM error detection; demonstrates practical deployment of explainability tools for illuminating model behavior in domain-specific text-based tasks.

— Anthropic production deployment of explainability: API stop_details field returns refusal category (cyber, bio, or null) and human-readable explanation, enabling developers to understand and route refusal decisions.

— Proposes architectural approach: localized ML models with higher per-node expressivity achieve both greater interpretability and computational efficiency than DNNs; addresses core accuracy-interpretability tradeoff through design-time commitment.

— Multi-center clinical deployment using 2461 RA patients (2011-2022 EHR data) with SHAP for cardiovascular risk prediction achieving C-index 0.8771; demonstrates SHAP enabling transparent risk interpretation in real clinical workflows.

— Governance maturity gap: 75% of UK financial services firms use AI but only 2% operate without human sign-off; explainability frameworks exist but real-world demonstrability at scale remains limited.

— XAI framework deployment (SHAP, LIME with XGBoost/Random Forest) for critical infrastructure intrusion detection (energy, healthcare, transportation, financial, communications); real-world application for governance of high-stakes systems.

— Critical governance risk: post-hoc explanations drift when models undergo quantization during deployment optimization; fragility of explanations to even minor model modifications threatens compliance justifications.

— Standards-based analysis shows SHAP insufficient for 62% of autonomous driving safety lifecycle stages; causal XAI required for hazard identification and incident investigation, establishing real deployment requirement gap.

— Psychophysics study (377 participants, 15K+ responses) establishes foundation models consistently less interpretable than supervised counterparts; interpretability independent of task performance, requiring explicit design interventions.

— Federal Reserve SR 26-2 (April 2026) and NIST framework with emphasis on lifecycle governance; governance structures requiring board-level oversight, model risk committees, and accountability for explainability as baseline practice.

— Federal Reserve SR 11-7 (January 2026) mandates explainability for AI systems using SHAP, LIME, counterfactual analysis; establishes regulatory baseline requiring institutions to treat AI models as high-risk assets demanding governance oversight.

— Expert panel consensus: model explainability (plain-language narratives, counterfactual analysis, fairness snapshots, immutable audit logs) now non-negotiable for regulatory defense; compliance failures common despite tool availability.

— Clinical deployment of EfficientNet with Grad-CAM achieving 99.37% accuracy on diabetic foot ulcer classification; demonstrates interpretability visualization enabling clinical decision support in medical imaging.

— FDA finalized 2025 guidance on AI/ML medical devices requiring disclosure of training data demographics, external validation datasets, and performance differentials by subgroup; establishes explainability as mandated compliance requirement in healthcare.

— Novel method (ELUDe) disentangles polysemantic neurons while preserving model outputs exactly, addressing core governance tension of interpretability without downstream accuracy loss in production systems.

— Critical finding in medical imaging: no XAI method (CAM, attribution) robust to corruption even when predictions accurate; explanations unreliable at noise levels plausible in clinical practice—documents fundamental governance limitation.

— Professional synthesis of SHAP/LIME deployment in high-stakes contexts (healthcare, lending, hiring) under ECOA and compliance mandates; positions interpretability as converting black-box liability into auditable governance.

— Demonstrates XAI interfaces exploitable for model extraction attacks on graph neural networks, documenting security risk and unintended consequences of transparency deployment in regulated platforms.

— Real-world deployment of SHAP and LIME to 50,718 Italian SMEs (2015–2024 financial data) for credit-risk prediction; demonstrates interpretability enabling transparent decision-making in regulated financial governance.

— Critical limitation: institutions have perfect logs of autonomous agent behavior but zero explainability for why agents converge; documents governance gap between machine-speed decisions and institutional accountability frameworks.

— Frontier model interpretability limitation: chain-of-thought alone insufficient for monitoring Opus 4.8; model reasons about evaluation in internal activations without text output—documents XAI method gap at scale.

— Regulatory shift: supervisors now examining model internals not just outputs; explainability and bias documentation baseline requirements across US, EU, UK; accountability shifted from compliance reporting to liability function.

— Governance gap: hybrid interpretable models route demographic groups differently to interpretable vs. black-box components; requires audit for interpretability coverage disparity beyond predictive fairness metrics.

— nEGXAI-V framework accepted at IEEE IRI 2026 for vision-based XAI; extends negation-based explainability to CNN applications with focus on early detection of prediction failures.

— Named financial services deployment using explainability as enforced production gate for regulated workloads; governance integration signals operational maturity.

— Healthcare deployment with dual-XAI validation framework (SHAP+LIME); demonstrates 100% concordance on top-3 feature rankings and clinical utility.

— Systematic review of 371 XAI studies in cancer imaging reveals critical deployment gap: 82.2% post-hoc methods, only 12.1% clinical deployment, 5.2% quantitative validation.

— Health Canada ML device guidance with PCCP, UK MHRA AI Airlock funding, EU €63.2M funding, and 770 NHS evidence responses emphasizing interpretability trust.

— Formalizes production XAI specifications (latency, fidelity, resource budgets) with empirical measurements; bridges research-practice gap in deployment requirements.

— Systematic review of 43 financial services studies (2018-2024) showing SHAP/LIME dominate implementations; identifies underexplored fairness integration.

— Identifies critical clinical adoption barriers: automation bias, post-hoc explanation instability, and socio-technical governance needs beyond technical explainability.

— Analyzes XAI adoption in financial services with regulatory drivers (ECOA, EU AI Act, UK Consumer Duty) and adoption metrics (54% of European banks).

— ICML 2026 position paper identifying fundamental limitations of current XAI methods in regulated financial contexts; empirical evidence of adoption barriers.

— Mixed-methods study (N=157) in South Asian mHealth revealing Western-centric XAI frameworks fail; identifies five context-specific requirements for resource-constrained settings.

— Operational deployment of SHAP+LIME in energy forecasting; demonstrates accuracy-stability trade-offs in production interpretability workflows.

— SHAP explainability framework applied to 70,000 clinical records with measured outcomes (AUC 0.773–0.794), validating interpretability in high-stakes healthcare.

— Critical practitioner assessment: explainability techniques (feature importance, local explanations, attention, chain-of-thought) show technical limitations and deployment risks in high-stakes domains; candid analysis of what explanations can and cannot deliver.

— Market adoption evidence: XAI grew to $11.74B (2026) from $9.73B (2025), 20.6% CAGR; EU AI Act August 2026 deadline with €35M penalties driving enterprise adoption; only 20% of orgs seeing AI ROI—governance/explainability the missing link.

— Materials science deployment: SHAP-guided feature selection in self-driving laboratories achieving 33% reduction in experimental effort; demonstrates operational efficiency gains from interpretability in production automation.

— Survey of 950 banking executives: 50% report governance/compliance barriers limiting AI performance; only 18% confident in audit readiness; explainability positioned as critical missing link for scaled deployment.

— Manufacturing case study: explainable ML (including SHAP) optimizing 3D-printed composites with quantified outcomes (14.6% tensile strength, 9.2% thermal conductivity improvements, R²=0.937).

— Peer-reviewed clinical deployment: SHAP-interpreted random forest for stroke-associated pneumonia prediction at Huizhou Central People's Hospital (290-patient cohort); demonstrates production integration of interpretability in high-stakes medical decision support.

— SHAP applied to Alzheimer's diagnosis and prognosis using 53,318 participants from NACC-UDS; high accuracy diagnostic models with empirical performance metrics; evidence of clinical-scale SHAP adoption in healthcare ML.

— Enterprise adoption barrier data: 30% cite lack of explainability as top barrier to AI trust; 28% cite model transparency; 46% of planned AI investments stalled due to trust concerns.

— Design science study (n=344) reveals critical limitation: XAI explanations produce NO significant improvement in user understandability, trust, or usability; introducing explanations reduces agreement with system classifications.

— Peer-reviewed deployment across four Wisconsin wastewater facilities (3.7-250 MGD capacity) validates SHAP and LIME effectiveness for predicting effluent quality, demonstrating production adoption across operational scales.

— Named regulatory enforcement ($89M Apple/Goldman Sachs penalty), market scale ($11.28B 2025 XAI market, projected $57.9B by 2035), and business ROI (2.84x vs 0.84x); explainability reduces loan processing time 70%.

— Named implementations: JPMorgan Chase uses SHAP for credit card approvals; Wells Fargo processes 400,000 mortgage applications annually with SHAP-generated explanations; CFPB found 60% of AI-based credit decisions lack explainable reasoning.

— Clinical validation study (400 physician-adjudicated vignettes) reveals 53-percentage-point knowledge-action gap: mechanistic interpretability achieves 98.2% AUROC but corrects only 45.1% of model errors, challenging assumptions about interpretability enabling error correction.

— BaFin and FINMA regulatory mandate: SHAP/LIME implementation required before production for credit, AML, fraud models; explainability is non-discretionary for DACH banking regulators.

— Field metrics: 34M+ features extracted from Claude 3 Sonnet (90% accuracy) yet <5% of frontier model computations understood; $75-150M annual investment; deception detection only 25-39% hint rate; 3-7 year timeline to safety applications.

— Critical assessment: sparse autoencoders underperform linear probes; interpretability methods lack generalization, remain unused in real engineering, produce incomplete explanations, and don't scale to frontier models.

— Google DeepMind engineer describes vendor deployment: activation probes added to live Gemini deployments for misuse detection, demonstrating production adoption of mechanistic interpretability techniques for AI governance.

— PLoS One study applying SHAP and LIME to CKD prediction achieves 88.4% (AUC=0.904) on hospital data and 94.6% (AUC=0.948) on UCI data, validating XAI utility in clinical contexts.

— Thales and IBM launch AI governance solution built with watsonx.governance for EU AI Act compliance, integrating explainability with cybersecurity expertise to manage AI lifecycle risks.

— Research framework evaluating SHAP and LIME robustness in geophysics reveals explanations can disagree on complex data, proposing causal necessity/sufficiency for more reliable interpretability.

— Analysis of regulatory shift in financial compliance: EU AI Act transparency mandates and heightened scrutiny on explainability moving from automation to accountability with mandatory human oversight.

— Eat Weight Disord study using SHAP/LIME on NHANES nutrition data (n=8914) with Random Forest achieving 0.991 AUC, identifying key predictors and confirming interpretability value in health-related ML.

— Critical assessment: 42% of companies abandoned AI initiatives in 2025 (up from 17% in 2024) due to compliance gaps; highlights that retrofitting explainability costs 2-3x more than building in from the start.

— Survey of 600 global data leaders shows 7 in 10 adopted GenAI but 75% acknowledge governance and data literacy gaps; adoption barriers reveal need for explainability and trustworthy governance.

— Named enterprise deployment (e&) of IBM watsonx.governance with embedded agentic AI for governance and compliance; proof-of-concept delivered in eight weeks, demonstrating explainability-by-design integration.

— Advances LIME for NLP by replacing random masking with hypothesis-driven LLM-based perturbations, showing significant improvements in local explanation fidelity and addressing semantic validity gaps.

— Third-party journalism covering 2026 XAI landscape: mechanistic interpretability maturation, corporate adoption across IBM/Palantir/ServiceNow, EU AI Act enforcement (Jan 2026), expert warnings of interpretability illusions.

— Critical analysis of black-box AI compliance failures in banking, identifying inability to produce audit-ready explanations and regulatory risks; cites Gartner prediction that 60% of enterprises adopt XAI tools by 2026.

— NeurIPS coverage of mechanistic interpretability's transition to production via Sparse Autoencoders; Rakuten case study demonstrates 500x cost reduction vs. GPT-5 for PII detection with higher recall.

— Microsoft support thread documenting product limitation: Azure ML AutoML/Designer does not support XAI/RAI dashboards for multi-level classification; signals real-world deployment constraints and vendor tooling gaps.

— Industry commentary from Corlytics and b-next on XAI's role in financial regulatory compliance; highlights practical adoption barriers and emphasizes human-in-the-loop necessity over technical transparency alone.

— Peer-reviewed comparative study of SHAP and LIME applied to XGBoost and TabNet for intrusion detection; XGBoost achieved 97.8% accuracy with stable explanations, validating XAI reliability in security domain.

— Moody's analysis of global financial regulatory mandates for AI explainability; documents OSFI Guideline E-23 (May 2027) requirement for enterprise-wide XAI compliance and FSI emphasis on explainability for managing core model risks.

— 18-month longitudinal case study of clinical AI system for cerebral palsy risk prediction; demonstrates clinicians trust explainability when it enables evaluation against their own assessments, introducing 'Evaluative Requirements' framework.

— Explores accuracy-interpretability trade-off with industry case studies in finance, healthcare, and algorithmic trading; proposes strategic framework for model selection balancing performance and explainability requirements.

— Peer-reviewed roadmap identifying three desiderata for XAI in medical practice: context-dependent explanations, genuine dialogue, and social capabilities; critical assessment of current method limitations in clinical integration.

— Industry data showing XAI market valued at USD 7.94B (2024) projected to reach USD 30.26B by 2032 (18.2% CAGR); signals sustained market growth and business adoption (83% of firms).

— IBM watsonx GA product page highlights explainable AI governance workflows with Vodafone case study showing 99% improvement in testing turnaround, signaling continued vendor maturity.

— Philosophical critique arguing demand for AI explainability may be overreaching, proposing reframe toward sound epistemic development practices rather than post-hoc model transparency.

— Peer-reviewed research addressing SHAP/LIME limitations in spectral analysis, proposing grouped feature analysis to enhance explainability and reduce noise in specialized domains.

— Practitioner tutorial detailing XAI method limitations: LIME explanation stability variance from randomness, SHAP/LIME faithfulness concerns, computational cost trade-offs in production deployment.

— InterpretML library v0.6.10 (March 2025) with enhanced ARM support and reordered class functions; demonstrates active maintenance and ecosystem maturity in open-source interpretability tooling.

— Technical comparison of SHAP and LIME consistency across dimensionality, finding LIME consistency drops to 0.72 in high dimensions but recovers with increased samples; practical guidance on method reliability.

— CSET analysis finding inconsistent definitions of explainability/interpretability and lack of real-world effectiveness testing; critical assessment revealing evaluation gaps at scale.

— Christoph Molnar reflects on sustainability challenges in interpretability education and technical debt in implementation; signals burnout and maintenance challenges in field maturity.

— CMU Machine Learning in Production course lecture notes integrating Cynthia Rudin's work on interpretable models, GDPR/ECOA regulatory references, and critical trade-off analysis.

— IBM watsonx.governance v2.1.2 (March 2025) adds Evaluation Studio for comparing AI assets and improved inventory navigation, signaling continued vendor investment in interpretability and model governance.

— MIT research introducing EXPLINGO, which uses LLMs to convert SHAP visualizations into narrative text explanations, addressing accessibility and complexity in communicating model decisions.

— PRISMA-compliant systematic review of LIME and SHAP in Alzheimer's detection across 23 studies, documenting XAI's role in clinical decision support and trustworthiness enhancement in healthcare.

— Financial services case study comparing LIME and SHAP in loan approval, achieving 85% accuracy; documents trade-offs (SHAP deeper attributions, LIME faster) and practical performance metrics.

— XAI 2024 conference paper comparing SHAP, LIME, ANCHORS, and DiCE for fraud detection, finding SHAP/LIME/ANCHORS superior in stability and separability; empirical validation in high-stakes financial domain.

— Consultancy case study applying SHAP and LIME to financial prediction model for Infineon Technologies; demonstrates real-world deployment with specific metrics and outperformed buy-and-hold baseline.

— Empirical evaluation across 20 datasets (68,500 model runs) shows interpretable GAMs match black-box performance for tabular data, dispelling performance-interpretability trade-off misconception.

— Critical assessment arguing post-hoc XAI methods (LIME/SHAP) are fundamentally insufficient due to spurious correlations, fragility, and late-stage explanations, calling for paradigm alternatives.

— Case study integrating XAI into cybersecurity analyst workflows reveals SHAP/LIME outputs 'lost in translation' with non-technical users, highlighting practical usability gaps in production deployments.

— IBM Research demonstrates XAI tools (LIME/SHAP) can be exploited for model extraction attacks, revealing security vulnerabilities in interpretable models under black-box settings.

— Journal review of XAI techniques in healthcare domains (radiology, oncology); identifies domain-specific challenges balancing fidelity with usability and proposes multidisciplinary framework.

— PLOS ONE case study applying SHAP and LIME to cancer detection achieving 96.89% accuracy; demonstrates real-world healthcare adoption of interpretability techniques in high-stakes medical diagnostics.

— MDPI Algorithms peer-reviewed analysis proposing XAIE framework for evaluating XAI tools; addresses critical gaps in framework selection and transparency within the XAI tool ecosystem.

— Position paper from McGill and Harvard arguing intrinsic and post-hoc interpretability paradigms fail to ensure faithfulness; highlights theoretical limitations and need for paradigm evolution.

— Qualitative study of user interactions with XAI in loan decisions; documents limitations of purely technical transparency and critical need for contextual explanations for user trust.

— Practitioner analysis of CFPB regulatory risks using SHAP/LIME for credit adverse actions; highlights accuracy validation gaps and practical adoption barriers in financial services compliance.

— Legal analysis linking LIME/SHAP to CFPB regulatory compliance requirements for credit decisions; signals emerging regulatory demand for interpretability in financial services.

— IEEE Access paper applying LIME, SHAP, PDP, and ICE to predictive maintenance in Industry 4.0; demonstrates practical XAI deployment in industrial manufacturing contexts.

— ArXiv research proposing framework to evaluate robustness of LIME and SHAP explanations in geophysics; demonstrates method disagreement and limitations in complex high-dimensional data.

— Microsoft Research position paper on LLM interpretability, discussing opportunities (natural language explanations) and new challenges (hallucinated explanations, computational costs) in evolving landscape.

— CVPR 2024 workshop paper reviewing XAI trends and limitations in remote sensing, identifying challenges in both application domain and XAI methodology, pointing to novel strategies and promising research directions.

— EU-funded research proposing Constrainable Neural Additive Models (CNAM) benchmarked on 56 datasets; demonstrates technical advancement in balancing interpretability and predictive performance.

— Stanford Social Innovation Review article debating necessity and trade-offs of explainability across domains, arguing targeted explainability rather than universal requirement.

— IBM releases watsonx.governance GA in December 2023 with integrated explainability features for monitoring fairness, bias, and drift, signaling sustained vendor investment in AI governance tooling.

— Survey paper documenting LIME's foundational limitations in fidelity, stability, and applicability; provides critical assessment of XAI method reliability amid pragmatic adoption.

— Retrospective study on 15,612 Swedish hospital admissions showing explainable ML models match deep learning performance (AUPRC 68% vs 66%) while providing actionable clinical insights.

— Empirical study across materials science datasets showing interpretable linear models achieve only 5% higher error than black-box models in extrapolation, contradicting assumed trade-offs.

— Azure Machine Learning documentation detailing built-in interpretability capabilities including SHAP and LIME for global and local explanations in production ML workflows.

— IBM watsonx.governance GA with built-in explainability features for model evaluation, fairness assessment, and interpretable AI governance at scale.

— Research paper identifying critical limitations in SHAP and LIME: high sensitivity to model choice and feature collinearity, raising caution about reliability and validity of popular XAI methods.

— Critical perspective paper arguing current XAI approaches fail to account for human decision-making processes and agency; proposes paradigm shift to evaluative AI framework.

— CMU SEI research framework for implementing XAI in practice, with case studies in wildlife conservation and space systems, advancing research-to-practice adoption pathways.

— Healthcare application evaluating LIME and SHAP for disease detection, achieving 99.79% accuracy with counterfactual explanations in high-stakes medical decision support.

— Application of LIME and SHAP to explain neural networks for breast cancer classification, demonstrating practical XAI adoption in biomedical high-stakes decision domains.

— Survey of employee perspectives on XAI in enterprises, showing recognition of XAI as critical for AI adoption but highlighting real-world adoption challenges and organizational barriers.

— Nature Communications survey showing public prioritizes accuracy over interpretability in trade-offs, revealing adoption barriers for interpretability tools in practice.

— Empirical study evaluating LIME and SHAP explanations for bug prediction models in software engineering, assessing explanation quality and alignment with human expectations.

— Survey of interpretability techniques with application to insurance, illustrating XAI tools for explaining actuarial models and addressing regulatory requirements in high-stakes domains.

— Healthcare research evaluating LIME and SHAP for ICD-10 code prediction in clinical text, with expert validation showing SHAP outperforms LIME in medical domain.

— KDD 2022 paper presenting GAM Changer, an interactive system for editing interpretable models; validated with 7 data scientists and deployed by physicians for pneumonia/sepsis risk prediction.

— Research paper revealing LIME and SHAP exhibit different bias-variance trade-offs depending on data sparsity; proposes CLIMB method, highlighting method reliability limitations.

— Microsoft announces Responsible AI Dashboard and Scorecard GA, integrating SHAP and LIME for model interpretability, fairness assessment, and error analysis in Azure ML.

— Deloitte analyst report on explainable AI adoption in banking, discussing LIME/SHAP techniques, regulatory drivers, and challenges in integrating XAI into banking operations.

— IUI 2022 case study with 14 physicians using interpretability tools for ECG classification, showing improved alignment with domain factors and building physician intuition about model limitations.

— Fortune critical assessment of XAI in healthcare, citing experts and research showing LIME/SHAP produce unreliable explanations; highlights risks of vendor overpromises.

— Methodological paper proposing design principles for inherently interpretable models in finance; includes real case study designing interpretable ReLU DNN for credit default prediction in home lending.

— Brookings Institution critical analysis argues XAI has not met practical governance goals; highlights that explainability often prioritizes engineering over user needs and fails to reduce power asymmetries.

— IBM Watson announces enhanced explainability for planning forecasts and federated learning; IBM-commissioned survey finds 84% of AI professionals value transparency, positioning explainability as mainstream requirement.

— Microsoft ships interpretability features in Azure ML with named deployments: SAS reduced fraud detection false positives using model understanding; EY improved loan fairness from 7% to 0.5% accuracy disparity.

— Real-world fraud detection study with analysts shows all explainers (LIME, SHAP, TreeInterpreter) reduced accuracy vs. data-only baseline; challenges practical utility claims.

— Study of 25 data scientists reveals state-of-the-art explanation methods (LIME, SHAP) often disagree; practitioners use ad hoc heuristics to resolve conflicts, risking unreliable explanations.

— NeurIPS 2020 paper proving linear/tree models strictly more interpretable than neural networks under computational complexity; provides formal theoretical foundation for comparing interpretability.

— IBM Watson OpenScale sample demonstrating enterprise-grade explainability implementation; shows operationalization of XAI capabilities in AI governance platforms.

— Critical analysis of SHAP and LIME showing sensitivity to model choice and feature collinearity; demonstrates limitations of popular post-hoc explanation methods.

— Comprehensive XAI survey proposing taxonomy of techniques, evaluating 8 algorithms on image data, discussing limitations and future directions for the field.

— Neural-Backed Decision Trees achieve neural network accuracy while preserving interpretability; set new SOTA for interpretable models on ImageNet (75.30% accuracy).

— CSCW 2020 study of 22 ML practitioners revealing organizational roles, adoption challenges, and gaps between academic XAI methods and real-world practice needs.

— Application of SHAP and LIME in agricultural AI systems; domain-specific case study demonstrating real-world deployment and comparison of interpretation methods.

— Comprehensive XAI survey by Arrieta et al. establishing foundational taxonomy of explainability concepts, opportunities, and challenges; authoritative reference consolidating field knowledge.

— Study of industry ML practitioners and product teams reveals real-world interpretability challenges, user needs, and adoption barriers; bridges research-to-practice gap.

— MIT Lincoln Lab analysis showing <1% of XAI papers validate explanations with humans; critical assessment exposing field maturity gap and validation methodologies.

— Polish government GDPR implementation requiring explanations for negative credit decisions; shows regulatory drivers creating real organizational demand for interpretability in finance.

— Human subject research isolating effect of algorithmic explanations on model simulatability; empirical evidence on explanation effectiveness and user understanding.

— R Journal paper introducing live and breakDown packages for model explanation, comparing with LIME and Shapley values, advancing practitioner tooling.

— NeurIPS 2018 paper proposing Bayesian framework incorporating human feedback to build interpretable models; advances human-centered interpretability research.

— Cynthia Rudin's influential position paper arguing post-hoc explanations for black-box models in high-stakes domains are problematic and favor inherently interpretable models instead.

— IBM launches AI OpenScale, a vendor-agnostic platform for real-time bias detection and AI decision explanation, signaling major vendor commitment to interpretability tooling.

— Critical opinion piece by Rudina Seseri questioning XAI feasibility, highlighting performance trade-offs and risks to IP disclosure, providing skeptical counterpoint.

— Comprehensive survey of explainable AI defining interpretability, classifying XAI approaches, and identifying standardization gaps; presented at IEEE DSAA 2018.