Quantitative and Qualitative Evaluation on Local Explainability Models for Anomaly Detection Algorithms

Abstract

[Abstract]: There is an increasingly urgent need to address the lack of transparency and clarity in the internal processes of AI (Artificial Intelligence) algorithms. In this paper, we explore local explainability techniques, LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations), to create a new layer of explanations on top of any anomaly detection model. This layer helps human supervisors better understand model behavior and the rationale behind its classification decisions. To assess the quality of these explanations, we conducted a qualitative analysis through a survey and a quantitative analysis using Quantus, a robust Python toolkit for evaluating explainability. The results of our experiments underscore the subtle trade-offs among various explainability techniques and emphasize the importance of carefully considering the context when applying explainability techniques.