Weakly Supervised Localization using Min-Max Entropy: an Interpretable Framework
Weakly supervised object localization (WSOL) models aim to locate objects of interest in an image after being trained only on data with coarse image level labels. Deep learning models for WSOL rely typically on convolutional attention maps with no constraints on the regions of interest which allows these models to select any region, making them vulnerable to false positive regions and inconsistent predictions. This issue occurs in many application domains, e.g., medical image analysis, where interpretability is central to the prediction process. In order to improve the localization reliability, we propose a deep learning framework for WSOL with pixel level localization. Our framework is composed of two sequential sub-networks: a localizer that localizes regions of interest; followed by a classifier that classifies these regions. Within its end-to-end training, we incorporate the prior knowledge stating that, in an agnostic-class setup, an image is more likely to contain relevant –i.e., object of interest– and irrelevant regions –i.e., noise, background–. Based on the conditional entropy measured at the classifier level, the localizer is driven to spot relevant regions identified with low conditional entropy, and irrelevant regions identified with high conditional entropy. Our framework is able to recover large and even complete discriminative regions in an image using our recursive erasing algorithm that we incorporate within the backpropagation during training. Moreover, the framework handles intrinsically multi-instances. Experimental results on public datasets with medical images (GlaS colon cancer) and natural images (Caltech-UCSD Birds-200-2011, Oxford flower 102) show that, compared to state of the art WSOL methods, the proposed approach can provide significant improvements in terms of image-level classification and pixel-level localization. Our framework showed robustness to overfitting when dealing with few training samples. Performance improvements are due in large part to our framework effectiveness at disregarding irrelevant regions. A public reproducible PyTorch implementation is provided1. https://github.com/sbelharbi/wsol-min-max-entropy-interpretability Preprint. Under review. ar X iv :1 90 7. 12 93 4v 3 [ cs .C V ] 3 1 A ug 2 01 9