Version 1
: Received: 29 May 2024 / Approved: 29 May 2024 / Online: 29 May 2024 (12:06:10 CEST)
How to cite:
Bhattacharya, A.; Verbert, K. "How Good Is Your Explanation?": Towards a Standardised Evaluation Approach for Diverse XAI Methods on Multiple Dimensions of Explainability. Preprints2024, 2024051964. https://doi.org/10.20944/preprints202405.1964.v1
Bhattacharya, A.; Verbert, K. "How Good Is Your Explanation?": Towards a Standardised Evaluation Approach for Diverse XAI Methods on Multiple Dimensions of Explainability. Preprints 2024, 2024051964. https://doi.org/10.20944/preprints202405.1964.v1
Bhattacharya, A.; Verbert, K. "How Good Is Your Explanation?": Towards a Standardised Evaluation Approach for Diverse XAI Methods on Multiple Dimensions of Explainability. Preprints2024, 2024051964. https://doi.org/10.20944/preprints202405.1964.v1
APA Style
Bhattacharya, A., & Verbert, K. (2024). "How Good Is Your Explanation?": Towards a Standardised Evaluation Approach for Diverse XAI Methods on Multiple Dimensions of Explainability. Preprints. https://doi.org/10.20944/preprints202405.1964.v1
Chicago/Turabian Style
Bhattacharya, A. and Katrien Verbert. 2024 ""How Good Is Your Explanation?": Towards a Standardised Evaluation Approach for Diverse XAI Methods on Multiple Dimensions of Explainability" Preprints. https://doi.org/10.20944/preprints202405.1964.v1
Abstract
Artificial Intelligence (AI) systems involve diverse components, such as data, models, users and predicted outcomes. To elucidate these different aspects of AI systems, multifaceted explanations that combine diverse explainable AI (XAI) methods are beneficial. However, popularly adopted user-centric XAI evaluation methods do not measure these explanations across the different components of the system. In this position paper, we advocate for an approach tailored to evaluate XAI methods considering the diverse dimensions of explainability within AI systems using a normalised scale. We argue that the prevalent user-centric evaluation methods fall short of facilitating meaningful comparisons across different types of XAI methodologies. Moreover, we discuss the potential advantages of adopting a standardised approach, which would enable comprehensive evaluations of explainability across systems. By considering various dimensions of explainability, such as data, model, predictions, and target users, a standardised evaluation approach promises to facilitate both inter-system and intra-system comparisons for user-centric AI systems.
Keywords
Explainable AI; XAI; Explainable AI Evaluation
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
