R-CNN
54465873
Mask R CNN
We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high quality segmentation mask for each instance. The method, called Mask R CNN, extends Faster R CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R CNN is simple to train and adds only a small overhead to Faster R CNN, running at 5 fps. Moreover, Mask R CNN is easy to generalize to other tasks, e.g. allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding box object detection, and person keypoint detection. Without bells and whistles, Mask R CNN outperforms all existing, single model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance level recognition. Code has been made available at: https:/github.com/facebookresearch/Detectron.
IEEE Transactions on Pattern Analysis and Machine Intelligence
Kaiming He Georgia Gkioxari Piotr Dollar Ross B Girshick
10328909
Faster R CNN: Towards Real Time Object Detection with Region Proposal Networks
State of the art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet and Fast R CNN have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full image convolutional features with the detection network, thus enabling nearly cost free region proposals. An RPN is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained end to end to generate high quality region proposals, which are used by Fast R CNN for detection. We further merge RPN and Fast R CNN into a single network by sharing their convolutional features using the recently popular terminology of neural networks with 'attention' mechanisms, the RPN component tells the unified network where to look. For the very deep VGG 16 model, our detection system has a frame rate of 5fps (including all steps) on a GPU, while achieving state of the art object detection accuracy on PASCAL VOC 2007, 2012, and MS COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015 competitions, Faster R CNN and RPN are the foundations of the 1st place winning entries in several tracks. Code has been made publicly available
IEEE Transactions on Pattern Analysis and Machine Intelligence
Shaoqing Ren Kaiming He Ross B Girshick Jian Sun
206770307
Fast R CNN
This paper proposes a Fast Region based Convolutional Network method (Fast R CNN) for object detection. Fast R CNN builds on previous work to efficiently classify object proposals using deep convolutional networks. Compared to previous work, Fast R CNN employs several innovations to improve training and testing speed while also increasing detection accuracy. Fast R CNN trains the very deep VGG16 network 9x faster than R CNN, is 213x faster at test time, and achieves a higher mAP on PASCAL VOC 2012. Compared to SPPnet, Fast R CNN trains VGG16 3x faster, tests 10x faster, and is more accurate. Fast R CNN is implemented in Python and C+ (using Caffe) and is available under the open source MIT License at https:/github.com/rbgirshick/fast rcnn.
2015 IEEE International Conference on Computer Vision (ICCV)
Ross B Girshick
206771194
Mask R CNN
We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high quality segmentation mask for each instance. The method, called Mask R CNN, extends Faster R CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R CNN is simple to train and adds only a small overhead to Faster R CNN, running at 5 fps. Moreover, Mask R CNN is easy to generalize to other tasks, e.g. allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding box object detection, and person keypoint detection. Without tricks, Mask R CNN outperforms all existing, single model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance level recognition. Code will be made available.
2017 IEEE International Conference on Computer Vision (ICCV)
Kaiming He Georgia Gkioxari Piotr Dollar Ross B Girshick
206596979
Cascade R CNN: Delving Into High Quality Object Detection
In object detection, an intersection over union (IoU) threshold is required to define positives and negatives. An object detector, trained with low IoU threshold, e.g. 0.5, usually produces noisy detections. However, detection performance tends to degrade with increasing the IoU thresholds. Two main factors are responsible for this: 1) overfitting during training, due to exponentially vanishing positive samples, and 2) inference time mismatch between the IoUs for which the detector is optimal and those of the input hypotheses. A multi stage object detection architecture, the Cascade R CNN, is proposed to address these problems. It consists of a sequence of detectors trained with increasing IoU thresholds, to be sequentially more selective against close false positives. The detectors are trained stage by stage, leveraging the observation that the output of a detector is a good distribution for training the next higher quality detector. The resampling of progressively improved hypotheses guarantees that all detectors have a positive set of examples of equivalent size, reducing the overfitting problem. The same cascade procedure is applied at inference, enabling a closer match between the hypotheses and the detector quality of each stage. A simple implementation of the Cascade R CNN is shown to surpass all single model object detectors on the challenging COCO dataset. Experiments also show that the Cascade R CNN is widely applicable across detector architectures, achieving consistent gains independently of the baseline detector strength. The code is available at https:/github.com/zhaoweicai/cascade rcnn.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Zhaowei Cai Nuno Vasconcelos
102354217
Libra R CNN: Towards Balanced Learning for Object Detection
Compared with model architectures, the training process, which is also crucial to the success of detectors, has received relatively less attention in object detection. In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels sample level, feature level, and objective level. To mitigate the adverse effects caused thereby, we propose Libra R CNN, a simple but effective framework towards balanced learning for object detection. It integrates three novel components: IoU balanced sampling, balanced feature pyramid, and balanced L1 loss, respectively for reducing the imbalance at sample, feature, and objective level. Benefitted from the overall balanced design, Libra R CNN significantly improves the detection performance. Without bells and whistles, it achieves 2.5 points and 2.0 points higher Average Precision (AP) than FPN Faster R CNN and RetinaNet respectively on MSCOCO.
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Jiangmiao Pang Kai Chen Jianping Shi Huajun Feng Wanli Ouyang Dahua Lin
67856179
Mask Scoring R CNN
Letting a deep network be aware of the quality of its own predictions is an interesting yet important problem. In the task of instance segmentation, the confidence of instance classification is used as mask quality score in most instance segmentation frameworks. However, the mask quality, quantified as the IoU between the instance mask and its ground truth, is usually not well correlated with classification score. In this paper, we study this problem and propose Mask Scoring R CNN which contains a network block to learn the quality of the predicted instance masks. The proposed network block takes the instance feature and the corresponding predicted mask together to regress the mask IoU. The mask scoring strategy calibrates the misalignment between mask quality and mask score, and improves instance segmentation performance by prioritizing more accurate mask predictions during COCO AP evaluation. By extensive evaluations on the COCO dataset, Mask Scoring R CNN brings consistent and noticeable gain with different models and outperforms the state of the art Mask R CNN. We hope our simple and effective approach will provide a new direction for improving instance segmentation. The source code of our method is available at \url{https:/github.com/zjhuang22/maskscoring_rcnn}
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Zhaojin Huang Lichao Huang Yongchao Gong Chang Huang Xinggang Wang
210873750
Faster R CNN for multi class fruit detection using a robotic vision system
Abstract An accurate and real time image based multi class fruit detection system is important for facilitating higher level smart farm tasks such as yield mapping and robotic harvesting. Robotic harvesting can reduce the costs of labour and increase fruit quality. This paper proposes a deep learning framework for multi class fruits detection based on improved Faster R CNN. The proposed framework includes fruits image library creation, data argumentation, improved Faster RCNN model generation, and performance evaluation. This work is a pioneer to create a multi labeled and knowledge based outdoor orchard image library using 4000 images in the real world. Also, improvement of the convolutional and pooling layers is achieved to have a more accurate and faster detection. The test results show the proposed algorithm has achieved higher detecting accuracy and lower processing time than the traditional detectors, which has excellent potential to build an autonomous and real time harvesting or yield mapping/estimation system.
Comput. Networks
Shaohua Wan Sotirios K Goudos
3780471
Domain Adaptive Faster R CNN for Object Detection in the Wild
Object detection typically assumes that training and test data are drawn from an identical distribution, which, however, does not always hold in practice. Such a distribution mismatch will lead to a significant performance drop. In this work, we aim to improve the cross domain robustness of object detection. We tackle the domain shift on two levels: 1) the image level shift, such as image style, illumination, etc. and 2) the instance level shift, such as object appearance, size, etc. We build our approach based on the recent state of the art Faster R CNN model, and design two domain adaptation components, on image level and instance level, to reduce the domain discrepancy. The two domain adaptation components are based on divergence theory, and are implemented by learning a domain classifier in adversarial training manner. The domain classifiers on different levels are further reinforced with a consistency regularization to learn a domain invariant region proposal network (RPN) in the Faster R CNN model. We evaluate our newly proposed approach using multiple datasets including Cityscapes, KITTI, SIM10K, etc. The results demonstrate the effectiveness of our proposed approach for robust object detection in various domain shift scenarios.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Yuhua Chen Wen Li Christos Sakaridis Dengxin Dai Luc Van Gool
208512936
Siam R CNN: Visual Tracking by Re Detection
We present Siam R CNN, a Siamese re detection architecture which unleashes the full power of two stage object detection approaches for visual object tracking. We combine this with a novel tracklet based dynamic programming algorithm, which takes advantage of re detections of both the first frame template and previous frame predictions, to model the full history of both the object to be tracked and potential distractor objects. This enables our approach to make better tracking decisions, as well as to re detect tracked objects after long occlusion. Finally, we propose a novel hard example mining strategy to improve Siam R CNN's robustness to similar looking objects. Siam R CNN achieves the current best performance on ten tracking benchmarks, with especially strong results for long term tracking. We make our code and models available at www.vision.rwth aachen.de/page/siamrcnn.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Paul Voigtlaender Jonathon Luiten Philip H S Torr B Leibe