For everyone interested in what makes a classifier put an image in a certain class I can recommend the excellent paper by Robert Geirhos et al. "ImageNet-traine

For everyone interested in what makes a classifier put an image in a certain class I can recommend the excellent paper by Robert Geirhos et al. "ImageNet-trained CNNs are biased towards texture" [1]. The paper is a bit older but it's an excellent read. [1] https://arxiv.org/pdf/1811.12231 https://arxiv.org/pdf/1811.12231