unsupervised video person re-identification
235390582
Unsupervised Video Person Re identification via Noise and Hard frame Aware Clustering
Unsupervised video based person re identification (re ID) methods extract richer features from video tracklets than image based ones. The state of the art methods utilize clustering to obtain pseudo labels and train the models iteratively. However, they underestimate the influence of two kinds of frames in the tracklet: 1) noise frames caused by detection errors or heavy occlusions exist in the tracklet, which may be allocated with unreliable labels during clustering; 2) the tracklet also contains hard frames caused by pose changes or partial occlusions, which are difficult to distinguish but informative. This paper proposes a Noise and Hard frame Aware Clustering (NHAC) method. NHAC consists of a graph trimming module and a node re sampling module. The graph trimming module obtains stable graphs by removing noise frame nodes to improve the clustering accuracy. The node re sampling module enhances the training of hard frame nodes to learn rich tracklet information. Experiments conducted on two video based datasets demonstrate the effectiveness of the proposed NHAC under the unsupervised re ID setting.
2021 IEEE International Conference on Multimedia and Expo (ICME)
Pengyu Xie T Yamasaki
235306489
SSN3D: Self Separated Network to Align Parts for 3D Convolution in Video Person Re Identification
Temporal appearance misalignment is a crucial problem in video person re identification. The same part of person (e.g. head or hand) appearing on different locations in video sequence weakens its discriminative ability, especially when we apply standard temporal aggregation such as 3D convolution or LSTM. To address this issue, we propose Self Separated network (SSN) to seek out the same parts in different images. As the name implies, SSN, if trained in an unsupervised strategy, guarantees the selected parts distinct. With a few samples of labeled parts to guide SSN training, this semi supervised trained SSN seeks out the parts that are human understandable within a frame and stable across a video snippet. Given the distinct and stable person parts, rather than performing aggregation on features, we then apply 3D convolution across different frames for person re identification. This SSN 3D pipeline, dubbed SSN3D, is proved to be efficient through extensive experiments on both synthetic and real data.
AAAI
Yu Qiao Junjie Yan Qichen Li Wanrong Zheng Dapeng Chen
235339397
End to end Network Embedding Unsupervised Key Frame Extraction for Video based Person Re identification
At present, regarding the task of video based person re identification, the input sequences have subtle differences and large redundancies because there are not enough effective interventions in the extraction of frame sequences. Although some studies have mentioned that key frame should be extracted first, they have not jointed the key frame extraction and the person re identification. Consequently, it is difficult to evaluate whether the extracted key frames are effective for person re identification. In this paper, we introduce an End to end Network Embedding Unsupervised Key Frame Extraction (EKEN) to address the above problems. First, we design a key frame extraction module and train it using pseudo labels generated by hierarchical clustering to extract key frames. Second, we embed the key frame extraction module into the person re identification task. The results of the key frame extraction and the pedestrian re recognition are fed back to each other in time. The instant feedback promotes the synchronization optimization of these two modules. The mAP achieved by our method in the MARS dataset is improved by 0.7% 2.9% 2.1% and 2.3% over the methods based on Random, Evenly, Cluster and Frame difference, respectively. Particularly, our method is more fit for the real world application comparing to existing methods.
2021 11th International Conference on Information Science and Technology (ICIST)
Ye Li Xiao Luo Shaoqi Hou Guangqiang Yin
234742171
Large Scale Unsupervised Person Re Identification with Contrastive Learning
Existing public person Re Identification (ReID) datasets are small in modern terms because of labeling difficulty. Although unlabeled surveillance video is abundant and relatively easy to obtain, it is unclear how to leverage these footage to learn meaningful ReID representations. In particular, most existing unsupervised and domain adaptation ReID methods utilize only the public datasets in their experiments, with labels removed. In addition, due to small data sizes, these methods usually rely on fine tuning by the unlabeled training data in the testing domain to achieve good performance. Inspired by the recent progress of large scale selfsupervised image classification using contrastive learning, we propose to learn ReID representation from large scale unlabeled surveillance video alone. Assisted by off theshelf pedestrian detection tools, we apply the contrastive loss at both the image and the tracklet levels. Together with a principal component analysis step using camera labels freely available, our evaluation using a large scale unlabeled dataset shows far superior performance among unsupervised methods that do not use any training data in the testing domain. Furthermore, the accuracy improves with the data size and therefore our method has great potential with even larger and more diversified datasets.
ArXiv
Weiquan Huang Yan Bai Qiu-Hua Ren Xinbo Zhao Ming Feng Yin Wang
231915796
Unsupervised Person Re Identification via Multi Order Cross View Graph Adversarial Network
Unsupervised person re identification (re id) is an effective analysis for video surveillance in practice, which can train a pedestrian matching model without any annotations, and it is easy to deploy in unseen camera scenarios. The most challenging problem in unsupervised re id task is the huge distribution gap among different camera views, and the intrinsic correlations in unlabeled identities are also complicated to sufficiently explored. This paper proposes a Multi order Cross view Graph adversarial Network (MCGN) to bridge the cross view distribution gap, and mine the inherent discriminative information by multi order triplet correlations. Specifically, MCGN firstly exploits graph representations by a cross view graph convolutional network according to intra view and inter view graph structure, and then encodes each pedestrian image into a view shared feature space, which is iteratively trained by a graph generative adversarial learning strategy to deeply bridge the distribution gap. Finally, this paper proposes a multi order discriminative learning module for composing reasonable triplet samples according to multi order similarity correlations among unlabeled pedestrian images. Furthermore, sufficient experiments are conducted in two large scale person re id datasets (Market 1501 and DukeMTMC reID) The comparison to state of the art methods and ablation study demonstrate the superiority of MCGN and the contribution of each module proposed in this paper.
IEEE Access
Xiang Fu Xinyu Lai
220665632
Learning Person Re Identification Models From Videos With Weak Supervision
Most person re identification methods, being supervised techniques, suffer from the burden of massive annotation requirement. Unsupervised methods overcome this need for labeled data, but perform poorly compared to the supervised alternatives. In order to cope with this issue, we introduce the problem of learning person re identification models from videos with weak supervision. The weak nature of the supervision arises from the requirement of video level labels, i.e. person identities who appear in the video, in contrast to the more precise frame level annotations. Towards this goal, we propose a multiple instance attention learning framework for person re identification using such video level labels. Specifically, we first cast the video person re identification task into a multiple instance learning setting, in which person images in a video are collected into a bag. The relations between videos with similar labels can be utilized to identify persons, on top of that, we introduce a co person attention mechanism which mines the similarity correlations between videos with person identities in common. The attention weights are obtained based on all person images instead of person tracklets in a video, making our learned model less affected by noisy annotations. Extensive experiments demonstrate the superiority of the proposed method over the related methods on two weakly labeled person re identification datasets.
IEEE Transactions on Image Processing
Xueping Wang Min Liu Dripta S Raychaudhuri S Paul Yaonan Wang Amit K Roy-Chowdhury
230590361
Correlation Discrepancy Insight Network for Video Re identification
Video based person re identification (ReID) aims at re identifying a specified person sequence from videos that were captured by disjoint cameras. Most existing works on this task ignore the quality discrepancy across frames by using all video frames to develop a ReID method. Additionally, they adopt only the person self characteristic as the representation, which cannot adapt to cross camera variation effectively. To that end, we propose a novel correlation discrepancy insight network for video based person ReID, which consists of an unsupervised correlation insight model (CIM) for video purification and a discrepancy description network (DDN) for person representation. Concretely, CIM is constructed by using kernelized correlation filters to encode person half parts, which evaluates the frame quality by the cross correlation across frames for selecting discriminative video fragments. Furthermore, DDN exploits the selected video fragments to generate a discrepancy descriptor using a compression network, which aims at employing the discrepancies with other persons' to facilitate the representation of the target person rather than only using the self characteristic. Due to the advantage in handling cross domain variation, the discrepancy descriptor is expected to provide a new pattern for the object representation in cross camera tasks. Experimental results on three public benchmarks demonstrate that the proposed method outperforms several state of the art methods.
ACM Trans. Multim. Comput. Commun. Appl.
Weijian Ruan Chao Liang Yi Yu Zongge Wang Wu Liu Jingyue Chen Jiayi Ma
226226594
Set Augmented Triplet Loss for Video Person Re Identification
Modern video person re identification (re ID) machines are often trained using a metric learning approach, supervised by a triplet loss. The triplet loss used in video re ID is usually based on so called clip features, each aggregated from a few frame features. In this paper, we propose to model the video clip as a set and instead study the distance between sets in the corresponding triplet loss. In contrast to the distance between clip representations, the distance between clip sets considers the pair wise similarity of each element (i.e. frame representation) between two sets. This allows the network to directly optimize the feature representation at a frame level. Apart from the commonly used set distance metrics (e.g. ordinary distance and Hausdorff distance) we further propose a hybrid distance metric, tailored for the set aware triplet loss. Also, we propose a hard positive set construction strategy using the learned class prototypes in a batch. Our proposed method achieves state of the art results across several standard benchmarks, demonstrating the advantages of the proposed method.
2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
Pengfei Fang Pan Ji Lars Petersson Mehrtash Tafazzoli Harandi
233476654
BiCnet TKS: Learning Efficient Spatial Temporal Representation for Video Person Re Identification
In this paper, we present an efficient spatial temporal representation for video person re identification (reID) Firstly, we propose a Bilateral Complementary Network (BiCnet) for spatial complementarity modeling. Specifically, BiCnet contains two branches. Detail Branch processes frames at original resolution to preserve the detailed visual clues, and Context Branch with a down sampling strategy is employed to capture long range contexts. On each branch, BiCnet appends multiple parallel and diverse attention modules to discover divergent body parts for consecutive frames, so as to obtain an integral characteristic of target identity. Furthermore, a Temporal Kernel Selection (TKS) block is designed to capture short term as well as long term temporal relations by an adaptive mode. TKS can be inserted into BiCnet at any depth to construct BiCnetTKS for spatial temporal modeling. Experimental results on multiple benchmarks show that BiCnet TKS outperforms state of the arts with about 50% less computations. The source code is available at this https URL blue blue272/BiCnet TKS.
CVPR 2021
Ruibing Hou Hong Chang Bingpeng Ma Rui Huang S Shan
225280648
Discriminative feature extraction for video person re identification via multi task network
The goal of video based person re identification is to match different pedestrians in various image sequences across non overlapping cameras. A critical issue of this task is how to exploit the useful information provided by videos. To solve this problem, we propose a novel feature learning framework for video based person re identification. The proposed method aims at capturing the most significant information in the spatial and temporal domains and then building a discriminative and robust feature representation for each sequence. More specifically, to learn more effective frame wise features, we apply several attributes to the video based task and build a multi task network for the identity and attribute classifications. In the training phase, we present a multi loss function to minimize intra class variances and maximize inter class differences. After that, the feature aggregation network is employed to aggregate frame wise features and extract the temporal information from the video. Furthermore, considering that adjacent frames typically have similar appearance features, we propose the concept of "non redundant appearance feature extraction" to obtain the sequence level appearance descriptors of pedestrians. Based on the complementarity between the temporal feature and the non redundant appearance feature, we combine them in the distance learning phase by assigning them different distance weighted coefficients. Extensive experiments are conducted on three video based datasets and the results demonstrate the superiority and effectiveness of our method.
Appl. Intell.
Wanru Song Jieying Zheng Yahong Wu Changhong Chen Feng Liu