ConceptSMILE Audits Whether Concept-Based AI Explanations Are Trustworthy
A new arXiv preprint introduces ConceptSMILE, a model-agnostic audit layer for concept-based explainable AI. The framework perturbs image regions, measures how concept responses change, applies locality weighting, and fits an XGBoost surrogate around the prediction being explained. It then evaluates attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency rather than assuming that human-readable concepts are reliable. In retinal fundus experiments, the authors report that MedSAM concepts led spatial attribution and produced fidelity scores of 0.8503 and 0.8465 while the vision-language pathway performed better on selected vessel-faithfulness and artefact-stability tests. LDS views the work as a useful audit template, but the evidence remains author-run, task-specific, and unvalidated for clinical use.
What happened
A new arXiv preprint proposes ConceptSMILE, a perturbation-based framework for auditing concept explanations produced by AI systems. Concept-based explainability tries to describe a model through human-understandable ideas rather than raw pixels or feature values. The paper's central warning is that understandable labels are not automatically faithful descriptions of what drove a prediction.
ConceptSMILE extends the logic of SMILE from feature or image-region attribution to concept-level auditing. It perturbs input regions, measures how concept responses shift, weights observations by locality, and trains an XGBoost surrogate to approximate behavior around the specific example. The audit then measures attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency.
Technical context
The authors compare MedSAM-derived visual concepts with semantic concepts produced through a vision-language pathway on retinal fundus images. MedSAM concepts led spatial attribution and produced fidelity scores of 0.8503 and 0.8465 in the reported evaluation. The vision-language pathway showed stronger vessel faithfulness and stronger stability under selected artefact conditions. The mixed result matters: a concept method can look reliable on one dimension while failing another.
| Audit dimension | Question | Warning sign |
|---|---|---|
| Attribution | Is the concept linked to the correct region? | Plausible label on irrelevant pixels |
| Fidelity | Does the local surrogate track the model? | Explanation diverges from prediction behavior |
| Faithfulness | Does changing evidence change the concept appropriately? | Concept remains despite removed evidence |
| Stability | Does small noise preserve the explanation? | Large concept drift |
| Consistency | Do comparable cases receive comparable concepts? | Arbitrary case-to-case changes |
For practitioners
A concept explanation should be treated as a testable output, not a user-interface decoration. Teams should define acceptable thresholds per audit dimension, preserve the perturbed samples and surrogate configuration, and repeat tests across institutions, devices, populations, and image artefacts. Aggregate scores should not hide failure on clinically important concepts or rare cases.
Editorial analysis
ConceptSMILE's practical value is the separation of interpretability from trustworthiness. A concept label can sound medically meaningful while remaining spatially wrong, unstable, or weakly connected to the model's decision. The framework turns those concerns into observable tests. Its present limitation is evidence scope: the preprint reports one medical-imaging evaluation and no independent reproduction.
What to watch
Watch for released code, independent validation, tests outside retinal imaging, comparisons with expert concept annotations, and evidence that audit failures predict real downstream model errors.
Key Points
- 1ConceptSMILE perturbs input regions and audits concept explanations across attribution, fidelity, faithfulness, stability, and consistency instead of assuming readability implies trustworthiness.
- 2MedSAM concepts led spatial attribution and produced fidelity scores of 0.8503 and 0.8465 in the authors' retinal-image evaluation.
- 3LDS recommends thresholding each audit dimension separately and repeating tests across institutions, devices, populations, artefacts, and clinically important concepts.
Scoring Rationale
An impact score of 6.3 reflects a useful multidimensional audit framework and concrete medical-imaging evaluation, tempered by preprint status, narrow scope, and missing independent replication.
Sources
Primary source and supporting public references used for this report.
Practice with real Health & Insurance data
90 SQL & Python problems · 15 industry datasets
250 free problems · No credit card
See all Health & Insurance problems
