Evaluation of XAI on ALS 6-months mortality prediction
Published in Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, 2022
Recommended citation: Buonocore et al. (2022). "Evaluation of XAI on ALS 6-months mortality prediction" Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. https://www.researchgate.net/profile/Tommaso-Buonocore/publication/362761796_Evaluation_of_XAI_on_ALS_6-months_mortality_prediction/links/62fe1002eb7b135a0e422dfd/Evaluation-of-XAI-on-ALS-6-months-mortality-prediction.pdf
In the article we present a comparative evaluation study of three different model agnostic XAI methods, namely SHAP, LIME and AraucanaXAI. The prediction task considered consists in predicting mortality for ALS patients based on observations carried out during a period of 6-months. The different XAI approaches are compared according to four quantitative evaluation metrics consisting in identity, fidelity, separability and time to compute an explanation. Furthermore, a qualitative comparison of post-hoc generated explanations is carried out on specific scenarios where the ML model correctly predicted the outcome, vs when it predicted it incorrectly. The combination of the results of the qualitative and quantitative evaluations carried out in the experiment form the basis for a critical discussion of XAI methods properties and desiderata for healthcare applications, advocating for more inclusive and extensive XAI evaluation studies involving human experts.
Recommended citation: Buonocore et al. (2022). “Evaluation of XAI on ALS 6-months mortality prediction” Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum.