SR networks are mysterious and little works make attempt to understand them. In this work, we perform attribution analysis of SR networks, which aims at finding the input pixels that strongly influence the SR results. We propose a novel attribution approach called local attribution map (LAM) to interpret SR networks. Our work opens new directions for designing SR networks and interpreting low-level vision deep models.

Local Attribution Maps (LAM)

LAM is build based on the following Auxiliary Principles:

1. Interpreting local not global.
2. Interpreting hard not simple.
3. Interpreting features not pixels.

The framework of LAM:

LAM inherits the integral gradient method yet with two unique features.

1. We use the blurred image as the baseline input.
2. We adopt a novel progressive blurring function as the path function.

Results

Interesting Findings

Based on LAM, we have several interesting conclusions:

1. SR networks with a wider range of involved input pixels could achieve better performance.
2. Attention networks and non-local networks extract features from a wider range of input pixels.
3. Comparing with the range that actually contributes, the receptive field is large enough for most deep networks.
4. For SR networks, textures with regular stripes or grids are more likely to be noticed, while complex semantics are difficult to utilize.

More information please refer to our paper.

Try Our Online Demo on Colab!

We build an online demo hosted on Google Colab. You can try LAM attribution results to see its interesting results.

Try Here!

Resources

• Newest Version (arXiv)
• Paper CVPR2021
• Technical Report (Full resolution, 50Mb)
• Colab Demo
• Code on GitHub (Coming soon)
• Extended Version (Coming soon)
• Video [Youtube][Bilibili]
• Talk at Extreme Mart, Shenzhen, 2021.3. bilibili (CN)
• Talk at CSIG-Guangdong Province CVPR 2021 Online Academic Report, 2021.5. bilibili (CN)
• Invited talk at Tsinghua AI TIME Online Academic Report, 2021.7. bilibili (CN)
• Invited Talk at Student Forum on Frontiers of AI (SFFAI), Shenzhen, 2021.9. bilibili (CN) Slides

Citation

If you find our work inspiring, please cite our work:

@inproceedings{gu2021interpreting,
  title={Interpreting Super-Resolution Networks with Local Attribution Maps},
  author={Gu, Jinjin and Dong, Chao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={9199--9208},
  year={2021}
}