Many traditional visual voice activity detection systems utilize features extracted from mouth region images which are sensitive to noisy observations of the visual domain. In addition, hyperparameters of the feature extraction process modulating the desired compromise between robustness, efficiency, and accuracy of the algorithm are difficult to be determined. Therefore, a visual voice activity detection algorithm which only utilizes simple lip shape information as features and a Long Short-Term Memory recurrent neural network LSTM-RNN as a classifier is proposed. Face detection is performed by structural SVM based on histogram of oriented gradient HOG features. Detected face template is used to initialize a kernelized correlation filter tracker. Facial landmark coordinates are then extracted from the tracked face. Centroid distance function is applied to the geometrically normalized landmarks surrounding the outer and inner lip contours. Finally, discriminative LSTM-RNN and generative Hidden Markov Model methods are used to model the temporal lip shape sequences during speech and non-speech intervals and their classification performances are compared. Experimental results show that the proposed algorithm using LSTM-RNN can achieve a classification rate of 98i¾?% in labeling speech and non-speech periods. It is robust and efficient for realtime applications.
['Zaw Htet Aung', 'Panrasee Ritthipravat']
Robust Visual Voice Activity Detection Using Long Short-Term Memory Recurrent Neural Network