fine-grained recognition
3825202
BoxCars: Improving Fine Grained Recognition of Vehicles Using 3 D Bounding Boxes in Traffic Surveillance
In this paper, we focus on fine grained recognition of vehicles mainly in traffic surveillance applications. We propose an approach that is orthogonal to recent advancements in fine grained recognition (automatic part discovery and bilinear pooling) In addition, in contrast to other methods focused on fine grained recognition of vehicles, we do not limit ourselves to a frontal/rear viewpoint, but allow the vehicles to be seen from any viewpoint. Our approach is based on 3 D bounding boxes built around the vehicles. The bounding box can be automatically constructed from traffic surveillance data. For scenarios where it is not possible to use precise construction, we propose a method for an estimation of the 3 D bounding box. The 3 D bounding box is used to normalize the image viewpoint by "unpacking" the image into a plane. We also propose to randomly alter the color of the image and add a rectangle with random noise to a random position in the image during the training of convolutional neural networks (CNNs) We have collected a large fine grained vehicle data set BoxCars116k, with 116k images of vehicles from various viewpoints taken by numerous surveillance cameras. We performed a number of experiments, which show that our proposed method significantly improves CNN classification accuracy (the accuracy is increased by up to 12% points and the error is reduced by up to 50% compared with CNNs without the proposed modifications) We also show that our method outperforms the state of the art methods for fine grained recognition.
IEEE Transactions on Intelligent Transportation Systems
Jakub Sochor Jakub Spanhel Adam Herout
209376486
Fine grained Recognition: Accounting for Subtle Differences between Similar Classes
The main requisite for fine grained recognition task is to focus on subtle discriminative details that make the subordinate classes different from each other. We note that existing methods implicitly address this requirement and leave it to a data driven pipeline to figure out what makes a subordinate class different from the others. This results in two major limitations: First, the network focuses on the most obvious distinctions between classes and overlooks more subtle inter class variations. Second, the chance of misclassifying a given sample in any of the negative classes is considered equal, while in fact, confusions generally occur among only the most similar classes. Here, we propose to explicitly force the network to find the subtle differences among closely related classes. In this pursuit, we introduce two key novelties that can be easily plugged into existing end to end deep learning pipelines. On one hand, we introduce "diversification block" which masks the most salient features for an input to force the network to use more subtle cues for its correct classification. Concurrently, we introduce a "gradient boosting" loss function that focuses only on the confusing classes for each sample and therefore moves swiftly along the direction on the loss surface that seeks to resolve these ambiguities. The synergy between these two blocks helps the network to learn more effective feature representations. Comprehensive experiments are performed on five challenging datasets. Our approach outperforms existing methods using similar experimental setting on all five datasets.
AAAI
Guolei Sun Hisham Cholakkal Salman Hameed Khan Fahad Shahbaz Khan Ling Shao
52065963
Learning a Discriminative Filter Bank Within a CNN for Fine Grained Recognition
Compared to earlier multistage frameworks using CNN features, recent end to end deep approaches for fine grained recognition essentially enhance the mid level learning capability of CNNs. Previous approaches achieve this by introducing an auxiliary network to infuse localization information into the main classification network, or a sophisticated feature encoding method to capture higher order feature statistics. We show that mid level representation learning can be enhanced within the CNN framework, by learning a bank of convolutional filters that capture class specific discriminative patches without extra part or bounding box annotations. Such a filter bank is well structured, properly initialized and discriminatively learned through a novel asymmetric multi stream architecture with convolutional filter supervision and a non random layer initialization. Experimental results show that our approach achieves state of the art on three publicly available fine grained recognition datasets (CUB 200 2011, Stanford Cars and FGVC Aircraft) Ablation studies and visualizations are provided to understand our approach.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Yaming Wang Vlad I Morariu Larry S Davis
11041575
SPDA CNN: Unifying Semantic Part Detection and Abstraction for Fine Grained Recognition
Most convolutional neural networks (CNNs) lack midlevel layers that model semantic parts of objects. This limits CNN based methods from reaching their full potential in detecting and utilizing small semantic parts in recognition. Introducing such mid level layers can facilitate the extraction of part specific features which can be utilized for better recognition performance. This is particularly important in the domain of fine grained recognition. In this paper, we propose a new CNN architecture that integrates semantic part detection and abstraction (SPDACNN) for fine grained classification. The proposed network has two sub networks: one for detection and one for recognition. The detection sub network has a novel top down proposal method to generate small semantic part candidates for detection. The classification sub network introduces novel part layers that extract features from parts detected by the detection sub network, and combine them for recognition. As a result, the proposed architecture provides an end to end network that performs detection, localization of multiple semantic parts, and whole object recognition within one framework that shares the computation of convolutional filters. Our method outperforms state of theart methods with a large margin for small parts detection (e.g. our precision of 93.40% vs the best previous precision of 74.00% for detecting the head on CUB 2011) It also compares favorably to the existing state of the art on finegrained classification, e.g. it achieves 85.14% accuracy on CUB 2011.
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Han Zhang Tao Xu Mohamed Elhoseiny Xiaolei Huang Shaoting Zhang A Elgammal Dimitris N Metaxas
218763610
Interpretable and Accurate Fine grained Recognition via Region Grouping
We present an interpretable deep model for fine grained visual recognition. At the core of our method lies the integration of region based part discovery and attribution within a deep neural network. Our model is trained using image level object labels, and provides an interpretation of its results via the segmentation of object parts and the identification of their contributions towards classification. To facilitate the learning of object parts without direct supervision, we explore a simple prior of the occurrence of object parts. We demonstrate that this prior, when combined with our region based part discovery and attribution, leads to an interpretable model that remains highly accurate. Our model is evaluated on major fine grained recognition datasets, including CUB 200, CelebA and iNaturalist. Our results compares favourably to state of the art methods on classification tasks, and outperforms previous approaches on the localization of object parts.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Zixuan Huang Yin Li
211730948
An Adversarial Domain Adaptation Network For Cross Domain Fine Grained Recognition
In this paper, we tackle a valuable yet very challenging visual recognition task, where the instances are within a subordinate category, and the target domain undergoes a shift with the source domain. This task, termed as cross domain fine grained recognition, relates closely to many real life scenarios, e.g. recognizing retail products in storage racks by models trained with images collected in controlled environments. To deal with this problem, we design a new algorithm and propose a corresponding fine grained domain adaptation dataset. Firstly, we propose a novel end to end CNN architecture that integrates two specialized modules: an adversarial module for domain alignment and a self attention module for fine grained recognition. The adversarial module is used to handle domain shift by gradually aligning the different domains with domain level and class level alignments, and strive to help the classifier learn with domain invariant features generated by nets. The self attention module is designed to capture discriminative image regions which are crucial for fine grained visual recognition. Secondly, we collect a large scale fine grained domain adaptation dataset of retail products, which contains 52,011 images of 263 classes from 3 domains. Thirdly, we validate the effectiveness of our method on three datasets, showing that the proposed method can yield significant improvements over baseline methods on fine grained datasets. Besides, we also evaluate the effectiveness of the self attention module by performing visualization, which can capture the discriminative image regions in both source and target domains.
2020 IEEE Winter Conference on Applications of Computer Vision (WACV)
Yimu Wang Ren-Jie Song Xiu-Shen Wei Lijun Zhang
15080437
The Unreasonable Effectiveness of Noisy Data for Fine Grained Recognition
Current approaches for fine grained recognition do the following: First, recruit experts to annotate a dataset of images, optionally also collecting more structured data in the form of part annotations and bounding boxes. Second, train a model utilizing this data. Toward the goal of solving fine grained recognition, we introduce an alternative approach, leveraging free, noisy data from the web and simple, generic methods of recognition. This approach has benefits in both performance and scalability. We demonstrate its efficacy on four fine grained datasets, greatly exceeding existing state of the art without the manual collection of even a single label, and furthermore show first results at scaling to more than 10,000 fine grained categories. Quantitatively, we achieve top 1 accuracies of (92.3\ on CUB 200 2011, (85.4\ on Birdsnap, (93.4\ on FGVC Aircraft, and (80.8\ on Stanford Dogs without using their annotated training sets. We compare our approach to an active learning approach for expanding fine grained datasets.
ECCV
Jonathan Krause Benjamin Sapp Andrew G Howard Howard Zhou Alexander Toshev Tom Duerig James Philbin Li Fei-Fei
219813075
Bi Modal Progressive Mask Attention for Fine Grained Recognition
Traditional fine grained image recognition is required to distinguish different subordinate categories (e.g. birds species) based on the visual cues beneath raw images. Due to both small inter class variations and large intra class variations, it is desirable to capture the subtle differences between these sub categories, which is crucial but challenging for fine grained recognition. Recently, language modality aggregation has been proved as a successful technique to improve visual recognition in the experience. In this paper, we introduce an end to end trainable Progressive Mask Attention (PMA) model for fine grained recognition by leveraging both visual and language modalities. Our Bi Modal PMA model can not only stage by stage capture the most discriminative part in the visual modality by our mask based fashion, but also explore the out of visual domain knowledge from the language modality in an interactional alignment paradigm. Specifically, at each stage, a self attention module is proposed to attend to the key patch from images or text descriptions. Besides, a query relational module is designed to seize the key words/phrases of texts and further bridge the connection between two modalities. Later, the learned representations of bi modality from multiple stages are aggregated as the final features for recognition. Our Bi Modal PMA model only needs raw images and raw text descriptions, without requiring bounding boxes/part annotations in images or key word annotations in texts. By conducting comprehensive experiments on fine grained benchmark datasets, we demonstrate that the proposed method achieves superior performance over the competing baselines, on either vision and language bi modality or single visual modality.
IEEE Transactions on Image Processing
Kaitao Song Xiu-Shen Wei Xiangbo Shu Ren-Jie Song Jianfeng Lu
220363872
End to end Learning of a Fisher Vector Encoding for Part Features in Fine grained Recognition
Part based approaches for fine grained recognition do not show the expected performance gain over global methods, although being able to explicitly focus on small details that are relevant for distinguishing highly similar classes. We assume that part based methods suffer from a missing representation of local features, which is invariant to the order of parts and can handle a varying number of visible parts appropriately. The order of parts is artificial and often only given by ground truth annotations, whereas viewpoint variations and occlusions result in parts that are not observable. Therefore, we propose integrating a Fisher vector encoding of part features into convolutional neural networks. The parameters for this encoding are estimated jointly with those of the neural network in an end to end manner. Our approach improves state of the art accuracies for bird species classification on CUB 200 2011 from 90.40\ to 90.95\ on NA Birds from 89.20\ to 90.30\ and on Birdsnap from 84.30\ to 86.97\
ArXiv
Dimitri Korsch P Bodesheim Joachim Denzler
219573772
Towards Robust Fine grained Recognition by Maximal Separation of Discriminative Features
Adversarial attacks have been widely studied for general classification tasks, but remain unexplored in the context of fine grained recognition, where the inter class similarities facilitate the attacker's task. In this paper, we identify the proximity of the latent representations of different classes in fine grained recognition networks as a key factor to the success of adversarial attacks. We therefore introduce an attention based regularization mechanism that maximally separates the discriminative latent features of different classes while minimizing the contribution of the non discriminative regions to the final class prediction. As evidenced by our experiments, this allows us to significantly improve robustness to adversarial attacks, to the point of matching or even surpassing that of adversarial training, but without requiring access to adversarial samples.
ACCV
Krishna Kanth Nakka Mathieu Salzmann