Manifold Similarity Reconstruction based Video Hashing

Most existing unsupervised video hashing methods are built on unidirectional models with less effective similarity preservation objectives, which underuse the bidirectional correlations among successive frames and are susceptible to noisy data while constructing the similarity matrix. To alleviate the problems above, in this paper, we propose a novel unsupervised video hashing method, called Manifold Similarity Reconstruction based Video Hashing (MSVH). Inspired by the MLM (mask language model) architecture, we design a visual reconstruction loss to fully exploit the bidirectional correlations between frames by forcing the model to extract information from context. Different from the previous researches, a novel similarity reconstruction loss is adopt considering both Euclidean distance and manifold structure to cope with noise. Furthermore, we develop a more reliable classification loss to exploit the global statistics of the whole dataset as a complement to local similarity preservation. Extensive experiments implemented on three public benchmark datasets, FCVID, UCF101 and HMDB51, show that the proposed method outperforms the state-of-the-art baselines.