feature alignment
221637473
PDAM: A Panoptic Level Feature Alignment Framework for Unsupervised Domain Adaptive Instance Segmentation in Microscopy Images
In this work, we present an unsupervised domain adaptation (UDA) method, named Panoptic Domain Adaptive Mask R CNN (PDAM) for unsupervised instance segmentation in microscopy images. Since there currently lack methods particularly for UDA instance segmentation, we first design a Domain Adaptive Mask R CNN (DAM) as the baseline, with cross domain feature alignment at the image and instance levels. In addition to the image and instance level domain discrepancy, there also exists domain bias at the semantic level in the contextual information. Next, we, therefore, design a semantic segmentation branch with a domain discriminator to bridge the domain gap at the contextual level. By integrating the semantic and instance level feature adaptation, our method aligns the cross domain features at the panoptic level. Third, we propose a task re weighting mechanism to assign trade off weights for the detection and segmentation loss functions. The task re weighting mechanism solves the domain bias issue by alleviating the task learning for some iterations when the features contain source specific factors. Furthermore, we design a feature similarity maximization mechanism to facilitate instance level feature adaptation from the perspective of representational learning. Different from the typical feature alignment methods, our feature similarity maximization mechanism separates the domain invariant and domain specific features by enlarging their feature distribution dependency. Experimental results on three UDA instance segmentation scenarios with five datasets demonstrate the effectiveness of our proposed PDAM method, which outperforms state of the art UDA methods by a large margin.
IEEE Transactions on Medical Imaging
Dongnan Liu Donghao Zhang Yang Song Fangfang Zhang Lauren J O'Donnell Heng Huang Mei Chen Weidong (Tom) Cai
218793899
Feature Alignment and Aggregation Siamese Networks for Fast Visual Tracking
Siamese networks have been successfully introduced into visual tracking, which match the best candidate and a target template via a couple of networks with shared parameters. However, most Siamese network based trackers (SNTs) are tailored to best match the canonical posture of the template and the search region images, resulting in inferior performance when the target objects have large scale pose variations. Besides, SNTs fail to discriminate distractors well because they only leverage high level semantic features as target representations that cannot well tell from different targets of the same category. To address these issues, this paper presents an efficient and effective SNT that is based on feature alignment and aggregation networks. Specifically, we first design an effective feature alignment network module to calibrate the search region image. This module results in a more reliable matching response that is robust to severe target pose variations. Then, we develop an effective shallow level and high level feature aggregation network module to complement the feature characteristics, making the learned feature representation not only well differentiate the target from distractors, but also robust to target appearance variations. Afterwards, we employ a channel attention mechanism to further strengthen the discriminative capability of the aggregated feature representation. Finally, both the alignment and the aggregation modules are seamlessly integrated into the Siamese networks for robust tracking. Meanwhile, we offline learn the network parameters end to end without time consuming fine tuning. Extensive evaluations on a variety of benchmarks including VOT 2017, OTB 100, UAV123 and GOT 10k demonstrate favorable performance of our tracker against state of the art ones with a speed of 60 fps.
IEEE Transactions on Circuits and Systems for Video Technology
Jiaqing Fan Huihui Song Kaihua Zhang Kang Yang Qingshan Liu
221497794
Curriculum Feature Alignment Domain Adaptation for Epithelium Stroma Classification in Histopathological Images
In recent years, deep learning methods have received more attention in epithelial stroma (ES) classification tasks. Traditional deep learning methods assume that the training and test data have the same distribution, an assumption that is seldom satisfied in complex imaging procedures. Unsupervised domain adaptation (UDA) transfers knowledge from a labelled source domain to a completely unlabeled target domain, and is more suitable for ES classification tasks to avoid tedious annotation. However, existing UDA methods for this task ignore the semantic alignment across domains. In this paper, we propose a Curriculum Feature Alignment Network (CFAN) to gradually align discriminative features across domains through selecting effective samples from the target domain and minimizing intra class differences. Specifically, we developed the Curriculum Transfer Strategy (CTS) and Adaptive Centroid Alignment (ACA) steps to train our model iteratively. We validated the method using three independent public ES datasets, and experimental results demonstrate that our method achieves better performance in ES classification compared with commonly used deep learning methods and existing deep domain adaptation methods.
IEEE Journal of Biomedical and Health Informatics
Qi Qi Xin Lin Chaoqi Chen Weiping Xie Yue Huang Xinghao Ding Xiaoqing Liu Yizhou Yu
230104791
SoFA: Source data free Feature Alignment for Unsupervised Domain Adaptation
Applying a trained model on a new scenario may suffer from domain shift. Unsupervised domain adaptation (UDA) has been proven to be an effective approach to solve the problem of domain shift by leveraging both data from the scenario that the model was trained on (source) and the new scenario (target) Although the source data are available for training the source model, there is no guarantee that the source data will still be available when applying UDA in the future due to emerging regulations on privacy of data. This results in the in applicability of most existing UDA methods in the absence of source data. This paper proposes a source data free feature alignment (SoFA) method to address this problem by only using the trained source model and unlabeled target data. The source model is used to predict the labels for target data, and we model the generation process from predicted classes to input data to infer the latent features for alignment. Specifically, a mixture of Gaussian distributions is induced from the predicted classes as the reference distribution. The encoded target features are then aligned to the reference distribution via variational inference to extract class semantics without accessing source data. Relationship of the proposed method and the theory of domain adaptation is provided to verify the performance. Experimental results show the proposed method achieves higher or comparable accuracy compared to the existing methods in several cross dataset classification tasks. Ablation studies are also conducted to confirm the importance of latent feature alignment to adaptation performance.
2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
Hao-Wei Yeh Baoyao Yang Pong C Yuen Tatsuya Harada
232304206
Towards Cross Dataset Palmprint Recognition Via Joint Pixel and Feature Alignment
Deep learning based palmprint recognition algorithms have shown great potential. Most of them are mainly focused on identifying samples from the same dataset. However, they may be not suitable for a more convenient case that the images for training and test are from different datasets, such as collected by embedded terminals and smartphones. Therefore, we propose a novel Joint Pixel and Feature Alignment (JPFA) framework for such cross dataset palmprint recognition scenarios. Two stage alignment is applied to obtain adaptive features in source and target datasets. 1) Deep style transfer model is adopted to convert source images into fake images to reduce the dataset gaps and perform data augmentation on pixel level. 2) A new deep domain adaptation model is proposed to extract adaptive features by aligning the dataset specific distributions of target source and target fake pairs on feature level. Adequate experiments are conducted on several benchmarks including constrained and unconstrained palmprint databases. The results demonstrate that our JPFA outperforms other models to achieve the state of the arts. Compared with baseline, the accuracy of cross dataset identification is improved by up to 28.10% and the Equal Error Rate (EER) of cross dataset verification is reduced by up to 4.69% To make our results reproducible, the codes are publicly available at http:/gr.xjtu.edu.cn/web/bell/resource.
IEEE Transactions on Image Processing
Huikai Shao Dexing Zhong
232763419
AFAN: Augmented Feature Alignment Network for Cross Domain Object Detection
Unsupervised domain adaptation for object detection is a challenging problem with many real world applications. Unfortunately, it has received much less attention than supervised object detection. Models that try to address this task tend to suffer from a shortage of annotated training samples. Moreover, existing methods of feature alignments are not sufficient to learn domain invariant representations. To address these limitations, we propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain adversarial training into a unified framework. An intermediate domain image generator is proposed to enhance feature alignments by domain adversarial training with automatically generated soft domain labels. The synthetic intermediate domain images progressively bridge the domain divergence and augment the annotated source domain training data. A feature pyramid alignment is designed and the corresponding feature discriminator is used to align multi scale convolutional features of different semantic levels. Last but not least, we introduce a region feature alignment and an instance discriminator to learn domain invariant features for object proposals. Our approach significantly outperforms the state of the art methods on standard benchmarks for both similar and dissimilar domain adaptations. Further extensive experiments verify the effectiveness of each component and demonstrate that the proposed network can learn domain invariant representations.
IEEE Transactions on Image Processing
Hongsong Wang Shengcai Liao Ling Shao
235358458
Semi Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment
Contemporary domain adaptive semantic segmentation aims to address data annotation challenges by assuming that target domains are completely unannotated. However, annotating a few target samples is usually very manageable and worthwhile especially if it improves the adaptation performance substantially. This paper presents SSDAS, a SemiSupervised Domain Adaptive image Segmentation network that employs a few labeled target samples as anchors for adaptive and progressive feature alignment between labeled source samples and unlabeled target samples. We position the few labeled target samples as references that gauge the similarity between source and target features and guide adaptive inter domain alignment for learning more similar source features. In addition, we replace the dissimilar source features by high confidence target features continuously during the iterative training process, which achieves progressive intra domain alignment between confident and unconfident target features. Extensive experiments show the proposed SSDAS greatly outperforms a number of baselines, i.e. UDA based semantic segmentation and SSDAbased image classification. In addition, SSDAS is complementary and can be easily incorporated into UDA based methods with consistent improvements in domain adaptive semantic segmentation.
ArXiv
Jiaxing Huang Dayan Guan Aoran Xiao Shijian Lu
233877844
Cross Domain Semantic Segmentation of Urban Scenes via Multi Level Feature Alignment
Semantic segmentation is an essential task in plenty of real life applications such as virtual reality, video analysis, autonomous driving, etc. Recent advancements in fundamental vision based tasks ranging from image classification to semantic segmentation have demonstrated deep learning based models' high capability in learning complicated representation on large datasets. Nevertheless, manually labeling semantic segmentation dataset with pixel level annotation is extremely labor intensive. To address this problem, we propose a novel multi level feature alignment framework for cross domain semantic segmentation of urban scenes by exploiting generative adversarial networks. In the proposed multi level feature alignment method, we first translate images from one domain to another one. Then the discriminative feature representations extracted by the deep neural network are concatenated, followed by domain adversarial learning to make the intermediate feature distribution of the target domain images close to those in the source domain. With these domain adaptation techniques, models trained with images in the source domain where the labels are easy to acquire can be deployed to the target domain where the labels are scarce. Experimental evaluations on various mainstream benchmarks confirm the effectiveness as well as robustness of our approach.
2020 25th International Conference on Pattern Recognition (ICPR)
Bin Zhang Shengjie Zhao Rongqing Zhang
235300500
Unsupervised Domain Adaptation Based Image Synthesis and Feature Alignment for Joint Optic Disc and Cup Segmentation.
Due to the discrepancy of different devices for fundus image collection, a well trained neural network is usually unsuitable for another new dataset. To solve this problem, the unsupervised domain adaptation strategy attracts a lot of attentions. In this paper, we propose an unsupervised domain adaptation method based image synthesis and feature alignment (ISFA) method to segment optic disc and cup on the fundus image. The GAN based image synthesis (IS) mechanism along with the boundary information of optic disc and cup is utilized to generate target like query images, which serves as the intermediate latent space between source domain and target domain images to alleviate the domain shift problem. Specifically, we use content and style feature alignment (CSFA) to ensure the feature consistency among source domain images, target like query images and target domain images. The adversarial learning is used to extract domain invariant features for output level feature alignment (OLFA) To enhance the representation ability of domain invariant boundary structure information, we introduce the edge attention module (EAM) for low level feature maps. Eventually, we train our proposed method on the training set of the REFUGE challenge dataset and test it on Drishti GS and RIM ONE_r3 datasets. On the Drishti GS dataset, our method achieves about 3% improvement of Dice on optic cup segmentation over the next best method. We comprehensively discuss the robustness of our method for small dataset domain adaptation. The experimental results also demonstrate the effectiveness of our method. Our code is available at https:/github.com/thinkobj/ISFA.
IEEE journal of biomedical and health informatics
Haijun Lei Weixing Liu Hai Xie Benjian Zhao Guanghui Yue Baiying Lei
233181502
Progressive Temporal Feature Alignment Network for Video Inpainting
Video inpainting aims to fill spatio temporal "corrupted" regions with plausible content. To achieve this goal, it is necessary to find correspondences from neighbouring frames to faithfully hallucinate the unknown content. Current methods achieve this goal through attention, flow based warping, or 3D temporal convolution. However, flow based warping can create artifacts when optical flow is not accurate, while temporal convolution may suffer from spatial misalignment. We propose 'Progressive Temporal Feature Alignment Network' which progressively enriches features extracted from the current frame with the feature warped from neighbouring frames using optical flow. Our approach corrects the spatial misalignment in the temporal feature propagation stage, greatly improving visual quality and temporal consistency of the inpainted videos. Using the proposed architecture, we achieve state of the art performance on the DAVIS and FVI datasets compared to existing deep learning approaches. Code is available at https:/github.com/MaureenZOU/TSAM
CVPR
Xueyan Zou Ding Liu Yong Jae Lee