consistency loss
212644488
Unpaired Image to Image Translation using Adversarial Consistency Loss
Unpaired image to image translation is a class of vision problems whose goal is to find the mapping between different image domains using unpaired training data. Cycle consistency loss is a widely used constraint for such problems. However, due to the strict pixel level constraint, it cannot perform geometric changes, remove large objects, or ignore irrelevant texture. In this paper, we propose a novel adversarial consistency loss for image to image translation. This loss does not require the translated image to be translated back to be a specific source image but can encourage the translated images to retain important features of the source images and overcome the drawbacks of cycle consistency loss noted above. Our method achieves state of the art results on three challenging tasks: glasses removal, male to female translation, and selfie to anime translation.
ECCV
Yihao Zhao Ruihai Wu Hao Dong
215828163
AdaCoSeg: Adaptive Shape Co Segmentation With Group Consistency Loss
We introduce AdaCoSeg, a deep neural network architecture for adaptive co segmentation of a set of 3D shapes represented as point clouds. Differently from the familiar single instance segmentation problem, co segmentation is intrinsically contextual: how a shape is segmented can vary depending on the set it is in. Hence, our network features an adaptive learning module to produce a consistent shape segmentation which adapts to a set. Specifically, given an input set of unsegmented shapes, we first employ an offline pre trained part prior network to propose per shape parts. Then the co segmentation network iteratively and jointly optimizes the part labelings across the set subjected to a novel group consistency loss defined by matrix ranks. While the part prior network can be trained with noisy and inconsistently segmented shapes, the final output of AdaSeg is a consistent part labeling for the input set, with each shape segmented into up to (a user specified) K parts. Overall, our method is weakly supervised, producing segmentations tailored to the test set, without consistent ground truth segmentations. We show qualitative and quantitative results from AdaSeg and evaluate it via ablation studies and comparisons to state of the art co segmentation methods.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Chenyang Zhu Kai Xu Siddhartha Chaudhuri Li Yi Leonidas J Guibas Hao Zhang
210473383
Structured Consistency Loss for semi supervised semantic segmentation
The consistency loss has played a key role in solving problems in recent studies on semi supervised learning. Yet extant studies with the consistency loss are limited to its application to classification tasks; extant studies on semi supervised semantic segmentation rely on pixel wise classification, which does not reflect the structured nature of characteristics in prediction. We propose a structured consistency loss to address this limitation of extant studies. Structured consistency loss promotes consistency in inter pixel similarity between teacher and student networks. Specifically, collaboration with CutMix optimizes the efficient performance of semi supervised semantic segmentation with structured consistency loss by reducing computational burden dramatically. The superiority of proposed method is verified with the Cityscapes; The Cityscapes benchmark results with validation and with test data are 81.9 mIoU and 83.84 mIoU respectively. This ranks the first place on the pixel level semantic labeling task of Cityscapes benchmark suite. To the best of our knowledge, we are the first to present the superiority of state of the art semi supervised learning in semantic segmentation.
ArXiv
Jongmok Kim Jooyoung Jang Hyunwoo Park
218487290
Multi Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesis
Magnetic Resonance (MR) images of different modalities can provide complementary information for clinical diagnosis, but whole modalities are often costly to access. Most existing methods only focus on synthesizing missing images between two modalities, which limits their robustness and efficiency when multiple modalities are missing. To address this problem, we propose a multi modality generative adversarial network (MGAN) to synthesize three high quality MR modalities (FLAIR, T1 and T1ce) from one MR modality T2 simultaneously. The experimental results show that the quality of the synthesized images by our proposed methods is better than the one synthesized by the baseline model, pix2pix. Besides, for MR brain image synthesis, it is important to preserve the critical tumor information in the generated modalities, so we further introduce a multi modality tumor consistency loss to MGAN, called TC MGAN. We use the synthesized modalities by TC MGAN to boost the tumor segmentation accuracy, and the results demonstrate its effectiveness.
2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI)
Bingyu Xin Yifan Hu Yefeng Zheng Hongen Liao
215768878
A recurrent cycle consistency loss for progressive face to face synthesis
This paper addresses a major flaw of the cycle consistency loss when used to preserve the input appearance in the face to face synthesis domain. In particular, we show that the images generated by a network trained using this loss conceal a noise that hinders their use for further tasks. To overcome this limitation, we propose a "recurrent cycle consistency loss" which for different sequences of target attributes minimises the distance between the output images, independent of any intermediate step. We empirically validate not only that our loss enables the re use of generated images, but that it also improves their quality. In addition, we propose the very first network that covers the task of unconstrained landmark guided face to face synthesis. Contrary to previous works, our proposed approach enables the transfer of a particular set of input features to a large span of poses and expressions, whereby the target landmarks become the ground truth points. We then evaluate the consistency of our proposed approach to synthesise faces at the target landmarks. To the best of our knowledge, we are the first to propose a loss to overcome the limitation of the cycle consistency loss, and the first to propose an "in the wild" landmark guided synthesis approach. Code and models for this paper can be found in https:/github.com/ESanchezLozano/GANnotation.
2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)
Enrique Sanchez Michel F Valstar
218798055
Semantic related image style transfer with dual consistency loss
Abstract Recently, image style transfer has drawn great attention in computer vision area with its successful applications in digital entertainment and social media. Much of the research aimed at rendering an image with artistic style of single artwork. However in certain applications, the target is to render the image as it were painted by a certain artist. Even though collection based image style transfer methods were proposed to address the single artwork problem, they ignored the relation between the subject and the style of an artwork, which is an important factor in mimicking an artist's creation mechanism. Based on this consideration, we assume the semantic relationship between the content and style by constructing a subject related photo artwork dataset and seek to learn that relationship. We propose a dual consistency loss to train an encoder decoder network with adversarial discriminator. The dual consistency loss encourages the output to be semantically and stylistically consistent with subject related content and style image pair. Thus the visual appearance of the stylized outputs would adapt to the subject of the content image, which increased the diversity of the generated results. Both quantitative and qualitative experimental results demonstrate the effectiveness of the proposed approach for retaining semantic and style consistency in style transfer process.
Neurocomputing
Zhuoqi Ma Jie Li Nannan Wang Xinbo Gao
226226724
Unsupervised Metric Relocalization Using Transform Consistency Loss
Training networks to perform metric relocalization traditionally requires accurate image correspondences. In practice, these are obtained by restricting domain coverage, employing additional sensors, or capturing large multi view datasets. We instead propose a self supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration. Guided by this intuition, we derive a novel transform consistency loss. Using this loss function, we train a deep neural network to infer dense feature and saliency maps to perform robust metric relocalization in dynamic environments. We evaluate our framework on synthetic and real world data, showing our approach outperforms other supervised methods when a limited amount of ground truth information is available.
ArXiv
Mike Kasper Fernando Nobre C Heckman Nima Keivan
210843160
Distribution Consistency Loss for Large Scale Remote Sensing Image Retrieval
Remote sensing images are featured by massiveness, diversity and complexity. These features put forward higher requirements for the speed and accuracy of remote sensing image retrieval. The extraction method plays a key role in retrieving remote sensing images. Deep metric learning (DML) captures the semantic similarity information between data points by learning embedding in vector space. However, due to the uneven distribution of sample data in remote sensing image datasets, the pair based loss currently used in DML is not suitable. To improve this, we propose a novel distribution consistency loss to solve this problem. First, we define a new way to mine samples by selecting five in class hard samples and five inter class hard samples to form an informative set. This method can make the network extract more useful information in a short time. Secondly, in order to avoid inaccurate feature extraction due to sample imbalance, we assign dynamic weight to the positive samples according to the ratio of the number of hard samples and easy samples in the class, and name the loss caused by the positive sample as the sample balance loss. We combine the sample balance of the positive samples with the ranking consistency of the negative samples to form our distribution consistency loss. Finally, we built an end to end fine tuning network suitable for remote sensing image retrieval. We display comprehensive experimental results drawing on three remote sensing image datasets that are publicly available and show that our method achieves the state of the art performance.
Remote. Sens.
Lili Fan Hongwei Zhao Haoyu Zhao
58014138
Certainty Driven Consistency Loss for Semi supervised Learning
The recently proposed semi supervised learning methods exploit consistency loss between different predictions under random perturbations. Typically, a student model is trained to predict consistently with the targets generated by a noisy teacher. However, they ignore the fact that not all training data provide meaningful and reliable information in terms of consistency. For misclassified data, blindly minimizing the consistency loss around them can hinder learning. In this paper, we propose a novel certainty driven consistency loss (CCL) to dynamically select data samples that have relatively low uncertainty. Specifically, we measure the variance or entropy of multiple predictions under random augmentations and dropout as an estimation of uncertainty. Then, we introduce two approaches, i.e. Filtering CCL and Temperature CCL to guide the student learn more meaningful and certain/reliable targets, and hence improve the quality of the gradients backpropagated to the student. Experiments demonstrate the advantages of the proposed method over the state of the art semi supervised deep learning methods on three benchmark datasets: SVHN, CIFAR10, and CIFAR100. Our method also shows robustness to noisy labels.
ArXiv
Yingting Li Lu Liu Robby T Tan
203209122
Decoupled Certainty Driven Consistency Loss for Semi supervised Learning
One of the successful approaches in semi supervised learning is based on the consistency regularization. Typically, a student model is trained to be consistent with teacher prediction for the inputs under different perturbations. To be successful, the prediction targets given by teacher should have good quality, otherwise the student can be misled by teacher. Unfortunately, existing methods do not assess the quality of the teacher targets. In this paper, we propose a novel Certainty driven Consistency Loss (CCL) that exploits the predictive uncertainty in the consistency loss to let the student dynamically learn from reliable targets. Specifically, we propose two approaches, i.e. Filtering CCL and Temperature CCL to either filter out uncertain predictions or pay less attention on them in the consistency regularization. We further introduce a novel decoupled framework to encourage model difference. Experimental results on SVHN, CIFAR 10, and CIFAR 100 demonstrate the advantages of our method over a few existing methods.
Lu Liu Yiting Li Robby T Tan