Update README.md

InterpretDL: 基于『飞桨』的模型可解释性算法库

InterpretDL(全称interpretations of deep learning models), 是基于飞桨的模型可解释性算法库，其中集成了许多可解释性算法，包括LIME, Grad-CAM, Integrated Gradients等等，还添加和更新许多SOTA算法和最新的可解释性算法。

InterpretDL持续更新中，欢迎所有贡献。

为什么选择InterpretDL

随着深度学习模型的越发复杂，人们很难去理解其内部的工作原理。“黑盒”的可解释性正成为许多优秀研究者的焦点。InterpretDL不仅支持经典的可解释性算法，也对最新的可解释性算法持续更新。

通过这些非常有效的可解释性方法，人们可以更好知道模型为什么好，为什么不好，进而可以针对性提高模型性能。

对于正在开发新可解释性算法的研究者而言，利用InterpretDL和已有算法比较也非常方便。

🔥 🔥 🔥 News 🔥 🔥 🔥

(2022/04/27) 除了已提供的每个算法的具体使用案例，新增一个快速上手教程：可从 GitHub 或者 NBViewer 查看。我们也在准备更多的教程以方便InterpretDL的用户使用。教程和使用案例均可以在 tutorial 目录下查看。
(2022/01/06) 新增 Cross-Model Consensus Explanation 算法. 简单来说，这个可解释性算法将多个模型的解释结果进行平均，来定位数据中最具代表性的特征，并且定位的精度相比单个模型的结果更为准确。更多细节请查看文章。
- Consensus: Xuhong Li, Haoyi Xiong, Siyu Huang, Shilei Ji, Dejing Dou. Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study. arXiv:2109.00707.

我们准备了一个简单的demo，用了6个模型进行平均，原文中建议用15个以上的模型进行解释，可以得到一个更好的结果。实现代码可以参考使用案例。

Consensus Result

Demo

可解释性算法对“黑盒”的预测做出了解释。

下表是使用一些可解释性算法对原始图片的可视化展示，告诉我们为什么模型做出了”bull_mastiff”的预测。

Original Image	IntGrad (demo)	SG (demo)	LIME (demo)	Grad-CAM (demo)

文本情感分类任务中模型给出积极或消极预测的原因也可以被可视化。这里是demo. 对中文的样本数据也同样适用。这里是demo.

InterpretDL: Interpretation of Deep Learning Models based on PaddlePaddle
Why InterpretDL
🔥 🔥 🔥 News 🔥 🔥 🔥
Demo
Contents
Installation
- Pip installation
- Developer installation
Documentation
Usage Guideline
Roadmap
Copyright and License
Recent News

Installation

安装百度「飞桨」深度学习框架 paddlepaddle, 建议选择CUDA支持版本。

Pip installation

pip install interpretdl

# or with tsinghua mirror
pip install interpretdl -i https://pypi.tuna.tsinghua.edu.cn/simple

Developer installation

git clone https://github.com/PaddlePaddle/InterpretDL.git
# ... fix bugs or add new features
cd InterpretDL && pip install -e .
# welcome to propose pull request and contribute

Unit Tests

# run gradcam unit tests
python -m unittest -v tests.interpreter.test_gradcam
# run all unit tests
python -m unittest -v

Documentation

在线链接: interpretdl.readthedocs.io.

或者下载到本地:

git clone https://github.com/PaddlePaddle/InterpretDL.git
cd docs
make html
open _build/html/index.html

Usage Guideline

所有解释器都继承类 Interpreter, 借助 interpret(**kwargs) 以调用。

# an example of SmoothGradient Interpreter.

import interpretdl as it
from paddle.vision.models import resnet50
paddle_model = resnet50(pretrained=True)
sg = it.SmoothGradInterpreter(paddle_model, use_cuda=True)
gradients = sg.interpret("test.jpg", visual=True, save_path=None)

使用细节在 tutorials 文件夹。

Roadmap

我门希望构建一个强大的模型解释工具库，并提供评估方式。以下是我们已实现的算法，我们也将继续加入新的算法。欢迎贡献算法，或者告诉我们想用的算法，我们将尽可能地实现。

已实现的算法

以输入特征为解释对象
- SmoothGrad
- IntegratedGradients
- Occlusion
- GradientSHAP
- LIME
- GLIME (LIMEPrior)
- NormLIME/FastNormLIME
- LRP
以中间过程特征为解释对象
- CAM
- GradCAM
- ScoreCAM
- Rollout
- TAM
数据层面的可解释性算法
- Forgetting Event
- SGDNoise
- TrainIng Data analYzer (TIDY)
跨模型的解释
- Consensus

计划中的算法

数据层面的可解释性算法
- Influence Function
评估方式
- Perturbation Tests
- Deletion & Insertion
- Localization Ablity
- Local Fidelity
- Sensitivity

Tutorials

我们计划为每种可解释性算法提供至少一个例子，涵盖CV和NLP的应用。

现有的教程在 tutorials 文件夹。

References of Algorithms

SGDNoise: On the Noisy Gradient Descent that Generalizes as SGD, Wu et al 2019
IntegratedGraients: Axiomatic Attribution for Deep Networks, Mukund Sundararajan et al. 2017
CAM, GradCAM: Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, Ramprasaath R. Selvaraju et al. 2017
SmoothGrad: SmoothGrad: removing noise by adding noise, Daniel Smilkov et al. 2017
GradientShap: A Unified Approach to Interpreting Model Predictions, Scott M. Lundberg et al. 2017
Occlusion: Visualizing and Understanding Convolutional Networks, Matthew D Zeiler and Rob Fergus 2013
Lime: “Why Should I Trust You?”: Explaining the Predictions of Any Classifier, Marco Tulio Ribeiro et al. 2016
NormLime: NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks, Isaac Ahern et al. 2019
ScoreCAM: Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks, Haofan Wang et al. 2020
ForgettingEvents: An Empirical Study of Example Forgetting during Deep Neural Network Learning, Mariya Toneva et al. 2019
LRP: On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation, Bach et al. 2015
Rollout: Quantifying Attention Flow in Transformers, Abnar et al. 2020
TAM: Explaining Information Flow Inside Vision Transformers Using Markov Chain. Yuan et al. 2021
Consensus: Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study. Li et al 2021
Perturbation: Evaluating the visualization of what a deep neural network has learned.
Deletion&Insertion: RISE: Randomized Input Sampling for Explanation of Black-box Models.
PointGame: Top-down Neural Attention by Excitation Backprop.

Copyright and License

InterpretDL 基于 Apache-2.0 license 提供。

Recent News

(2021/10/20) 新增 Transition Attention Maps (TAM) explanation method for PaddlePaddle Vision Transformers. 一如往常，几行代码即可调用！详情查看使用案例, 以及 paper:
- TAM: Tingyi Yuan, Xuhong Li, Haoyi Xiong, Hui Cao, Dejing Dou. Explaining Information Flow Inside Vision Transformers Using Markov Chain. In Neurips 2021 XAI4Debugging Workshop.

import paddle
import interpretdl as it

# load vit model and weights
# !wget -c https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch16_224_pretrained.pdparams -P assets/
from assets.vision_transformer import ViT_base_patch16_224
paddle_model = ViT_base_patch16_224()
MODEL_PATH = 'assets/ViT_base_patch16_224_pretrained.pdparams'
paddle_model.set_dict(paddle.load(MODEL_PATH))

# Call the interpreter.
tam = it.TAMInterpreter(paddle_model, use_cuda=True)
img_path = 'samples/el1.png'
heatmap = tam.interpret(
        img_path,
        start_layer=4,
        label=None,  # elephant
        visual=True,
        save_path=None)
heatmap = tam.interpret(
        img_path,
        start_layer=4,
        label=340,  # zebra
        visual=True,
        save_path=None)

image	elephant	zebra

(2021/07/22) 新增 Rollout Explanations for PaddlePaddle Vision Transformers. 点击 notebook 浏览可视化结果。

import paddle
import interpretdl as it

# wget -c https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_small_patch16_224_pretrained.pdparams -P assets/
from assets.vision_transformer import ViT_small_patch16_224
paddle_model = ViT_small_patch16_224()
MODEL_PATH = 'assets/ViT_small_patch16_224_pretrained.pdparams'
paddle_model.set_dict(paddle.load(MODEL_PATH))

img_path = 'assets/catdog.png'
rollout = it.RolloutInterpreter(paddle_model, use_cuda=True)
heatmap = rollout.interpret(img_path, start_layer=0, visual=True)