This is the repository of A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future, a systematic review of Multimodal Explainable Artificial Intelligence (MXAI) studies in a historical perspective. For details, please refer to:
A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future Paper
- ☄️ [2024/12/26] We have revised the table of contents, and the remaining sections will be updated progressively.
Artificial intelligence (AI) has rapidly developed through advancements in computational power and the growth of massive datasets. However, this progress has also heightened challenges in interpreting the "black-box" nature of AI models.
To address these concerns, eXplainable AI (XAI) has emerged with a focus on transparency and interpretability to enhance human understanding and trust in AI decision-making processes. In the context of multimodal data fusion and complex reasoning scenarios, the proposal of Multimodal eXplainable AI (MXAI) integrates multiple modalities for prediction and explanation tasks. Meanwhile, the advent of Large Language Models (LLMs) has led to remarkable breakthroughs in natural language processing, yet their complexity has further exacerbated the issue of MXAI.
To gain key insights into the development of MXAI methods and provide crucial guidance for building more transparent, fair, and trustworthy AI systems, we review the MXAI methods from a historical perspective and categorize them across four eras: traditional machine learning, deep learning, discriminative foundation models, and generative LLMs. We also review evaluation metrics and datasets used in MXAI research, concluding with a discussion of future challenges and directions.
We truly appreciate any contributions through PRs, issues, emails, or other means.
- Related Surveys
- 1. The Era of Traditional Machine Learning (2000-2009)
- 2. The Era of Deep Learning (2011-2016)
- 3. The Era of Discriminative Foundation Models (2017-2021)
- 4. The Era of Generative Large Language Models (2022-2024)
- 5. Evaluation datasets and metrics
Click to expand the XAI Survey Comparison Table
If you find this repository useful, please cite our paper:
@article{sun2024review,
title={A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future},
author={Sun, Shilin and An, Wenbin and Tian, Feng and Nan, Fang and Liu, Qidong and Liu, Jun and Shah, Nazaraf and Chen, Ping},
journal={arXiv preprint arXiv:2412.14056},
year={2024}
}