From Interpretability Methods to Interpretable Models

Abstract

Deep learning models are increasingly deployed for high-stakes settings, yet their decision-making remains largely opaque to the people who rely on them. This talk argues that, alongside developing better tools to explain those opaque models, explainable AI (XAI) should seek to understand the principles that make models less opaque in the first place. To this end, it develops a framework grounded in large-scale human experiments to measure, compare, and shape the interpretability of vision models. First, the talk investigates the appropriate representational basis for interpretability, showing that dictionary-learned features are more interpretable than individual neurons. Next, it introduces two behavioral protocols to measure human understanding of visual features and systematically compare interpretability across models. These reveal that vision foundation models are less interpretable than their supervised baseline, a gap best predicted by feature locality and alignment with human semantic judgments. Finally, the talk investigates whether interpretability can be improved by aligning models with human semantic judgments, finding that this approach benefits some models but not others. Together, these results provide the first behaviorally grounded account of how interpretable vision foundation models are, why they differ, and how far their interpretability can be deliberately improved.

Bio

Julien Colin is an ELLIS PhD student at ELLIS Alicante, working under the supervision of Nuria Oliver (ELLIS Alicante) and Thomas Serre (brown University). He holds a Master's degree in Artificial and Natural Cognition (2021, Grenoble INP - Phelma). His research focuses on what makes AI models understandable to humans: how to measure it, how to shape it, and the mechanisms behind it.