cnn speech
219591499
Research on an Improved CNN Speech Recognition System Based on Hidden Markov Model
In order to improve the accuracy of speech recognition and speaker identification in the field of production and life, an improved convolutional neural network algorithm based on hidden markov model is proposed, which includes four parts: feature extraction, segmentation, training and clustering. Quantization and clustering were carried out in the recorded sound set, the feature vectors of the obtained sound signal were segmented, and further training was conducted by deep convolutional network. Finally, after the state segmentation of HMM was completed, the recognition was completed by CNN as the input part of CNN. The experimental results show that the improved CNN based on hidden markov model can complete the recognition function and verify the correctness of the algorithm.
2020 International Conference on Intelligent Transportation, Big Data Smart City (ICITBS)
Zhikang Chen Jia Liu Yun-xi Zhang
210043094
A CNN Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition
Speech is the most significant mode of communication among human beings and a potential method for human computer interaction (HCI) by using a microphone sensor. Quantifiable emotion recognition using these sensors from speech signals is an emerging area of research in HCI, which applies to multiple applications such as human reboot interaction, virtual reality, behavior assessment, healthcare, and emergency call centers to determine the speaker's emotional state from an individual's speech. In this paper, we present major contributions for; (i) increasing the accuracy of speech emotion recognition (SER) compared to state of the art and (ii) reducing the computational complexity of the presented SER model. We propose an artificial intelligence assisted deep stride convolutional neural network (DSCNN) architecture using the plain nets strategy to learn salient and discriminative features from spectrogram of speech signals that are enhanced in prior steps to perform better. Local hidden patterns are learned in convolutional layers with special strides to down sample the feature maps rather than pooling layer and global discriminative features are learned in fully connected layers. A SoftMax classifier is used for the classification of emotions in speech. The proposed technique is evaluated on Interactive Emotional Dyadic Motion Capture (IEMOCAP) and Ryerson Audio Visual Database of Emotional Speech and Song (RAVDESS) datasets to improve accuracy by 7.85% and 4.5% respectively, with the model size reduced by 34.5 MB. It proves the effectiveness and significance of the proposed SER technique and reveals its applicability in real world applications.
Sensors
Soonil Kwon
53016344
Speech emotion recognition using deep 1D 2D CNN LSTM networks
Abstract We aimed at learning deep emotion features to recognize speech emotion. Two convolutional neural network and long short term memory (CNN LSTM) networks, one 1D CNN LSTM network and one 2D CNN LSTM network, were constructed to learn local and global emotion related features from speech and log mel spectrogram respectively. The two networks have the similar architecture, both consisting of four local feature learning blocks (LFLBs) and one long short term memory (LSTM) layer. LFLB, which mainly contains one convolutional layer and one max pooling layer, is built for learning local correlations along with extracting hierarchical correlations. LSTM layer is adopted to learn long term dependencies from the learned local features. The designed networks, combinations of the convolutional neural network (CNN) and LSTM, can take advantage of the strengths of both networks and overcome the shortcomings of them, and are evaluated on two benchmark databases. The experimental results show that the designed networks achieve excellent performance on the task of recognizing speech emotion, especially the 2D CNN LSTM network outperforms the traditional approaches, Deep Belief Network (DBN) and CNN on the selected databases. The 2D CNN LSTM network achieves recognition accuracies of 95.33% and 95.89% on Berlin EmoDB of speaker dependent and speaker independent experiments respectively, which compare favourably to the accuracy of 91.6% and 92.9% obtained by traditional approaches; and also yields recognition accuracies of 89.16% and 52.14% on IEMOCAP database of speaker dependent and speaker independent experiments, which are much higher than the accuracy of 73.78% and 40.02% obtained by DBN and CNN.
Biomed. Signal Process. Control.
Jianfeng Zhao Xia Mao Lijiang Chen
220794027
KUISAIL at SemEval 2020 Task 12: BERT CNN for Offensive Speech Identification in Social Media
In this paper, we describe our approach to utilize pre trained BERT models with Convolutional Neural Networks for sub task A of the Multilingual Offensive Language Identification shared task (OffensEval 2020) which is a part of the SemEval 2020. We show that combining CNN with BERT is better than using BERT on its own, and we emphasize the importance of utilizing pre trained language models for downstream tasks. Our system, ranked 4th with macro averaged F1 Score of 0.897 in Arabic, 4th with score of 0.843 in Greek, and 3rd with score of 0.814 in Turkish. Additionally, we present ArabicBERT, a set of pre trained transformer language models for Arabic that we share with the community.
SEMEVAL
Ali Safaya Moutasem Abdullatif Deniz Yuret
148572680
A New Framework for CNN Based Speech Enhancement in the Time Domain
This paper proposes a new learning mechanism for a fully convolutional neural network (CNN) to address speech enhancement in the time domain. The CNN takes as input the time frames of noisy utterance and outputs the time frames of the enhanced utterance. At the training time, we add an extra operation that converts the time domain to the frequency domain. This conversion corresponds to simple matrix multiplication, and is hence differentiable implying that a frequency domain loss can be used for training in the time domain. We use mean absolute error loss between the enhanced short time Fourier transform (STFT) magnitude and the clean STFT magnitude to train the CNN. This way, the model can exploit the domain knowledge of converting a signal to the frequency domain for analysis. Moreover, this approach avoids the well known invalid STFT problem since the proposed CNN operates in the time domain. Experimental results demonstrate that the proposed method substantially outperforms the other methods of speech enhancement. The proposed method is easy to implement and applicable to related speech processing tasks that require time frequency masking or spectral mapping.
IEEE/ACM Transactions on Audio, Speech, and Language Processing
Ashutosh Pandey Deliang Wang
216376855
Speech emotion recognition using fusion of three multi task learning based classifiers: HSF DNN, MS CNN and LLD RNN
Abstract Speech emotion recognition plays an increasingly important role in emotional computing and is still a challenging task due to its complexity. In this study, we developed a framework integrating three distinctive classifiers: a deep neural network (DNN) a convolution neural network (CNN) and a recurrent neural network (RNN) The framework was used for categorical recognition of four discrete emotions (i.e. angry, happy, neutral and sad) Frame level low level descriptors (LLDs) segment level mel spectrograms (MS) and utterance level outputs of high level statistical functions (HSFs) on LLDs were passed to RNN, CNN, and DNN, separately. Three individual models of LLD RNN, MS CNN, and HSF DNN were obtained. In the models of MS CNN and LLD RNN, the attention mechanism based weighted pooling method was utilized to aggregate the CNN and RNN outputs. To effectively utilize the interdependencies between the two approaches of emotion description (discrete emotion categories and continuous emotion attributes) a multi task learning strategy was implemented in these three models to acquire generalized features by simultaneously operating classification of discrete categories and regression of continuous attributes. Finally, a confidence based fusion strategy was developed to integrate the power of different classifiers in recognizing different emotional states. Three experiments on emotion recognition based on the IEMOCAP corpus were conducted. Our experimental results show that the weighted pooling method based on attention mechanism endowed the neural networks with the capability to focus on emotionally salient parts. The generalized features learned in the multi task learning helped the neural networks to achieve higher accuracies in the tasks of emotion classification. Furthermore, our proposed fusion system achieved weighted accuracy of 57.1% and unweighted accuracy of 58.3% which were significantly higher than those of each individual classifier. The effectiveness of the proposed approach based on classifier fusion was thus validated.
Speech Commun.
Zengwei Yao Zihao Wang Weihuang Liu Yaqian Liu Jiahui Pan
86766369
A Hybrid of Deep CNN and Bidirectional LSTM for Automatic Speech Recognition
Abstract Deep neural networks (DNNs) have been playing a significant role in acoustic modeling. Convolutional neural networks (CNNs) are the advanced version of DNNs that achieve 4 12% relative gain in the word error rate (WER) over DNNs. Existence of spectral variations and local correlations in speech signal makes CNNs more capable of speech recognition. Recently, it has been demonstrated that bidirectional long short term memory (BLSTM) produces higher recognition rate in acoustic modeling because they are adequate to reinforce higher level representations of acoustic data. Spatial and temporal properties of the speech signal are essential for high recognition rate, so the concept of combining two different networks came into mind. In this paper, a hybrid architecture of CNN BLSTM is proposed to appropriately use these properties and to improve the continuous speech recognition task. Further, we explore different methods like weight sharing, the appropriate number of hidden units, and ideal pooling strategy for CNN to achieve a high recognition rate. Specifically, the focus is also on how many BLSTM layers are effective. This paper also attempts to overcome another shortcoming of CNN, i.e. speaker adapted features, which are not possible to be directly modeled in CNN. Next, various non linearities with or without dropout are analyzed for speech tasks. Experiments indicate that proposed hybrid architecture with speaker adapted features and maxout non linearity with dropout idea shows 5.8% and 10% relative decrease in WER over the CNN and DNN systems, respectively.
J. Intell. Syst.
Vishal Passricha Rajesh Kumar Aggarwal
221748924
Deep Net: A Lightweight CNN Based Speech Emotion Recognition System Using Deep Frequency Features
Artificial intelligence (AI) and machine learning (ML) are employed to make systems smarter. Today, the speech emotion recognition (SER) system evaluates the emotional state of the speaker by investigating his/her speech signal. Emotion recognition is a challenging task for a machine. In addition, making it smarter so that the emotions are efficiently recognized by AI is equally challenging. The speech signal is quite hard to examine using signal processing methods because it consists of different frequencies and features that vary according to emotions, such as anger, fear, sadness, happiness, boredom, disgust, and surprise. Even though different algorithms are being developed for the SER, the success rates are very low according to the languages, the emotions, and the databases. In this paper, we propose a new lightweight effective SER model that has a low computational complexity and a high recognition accuracy. The suggested method uses the convolutional neural network (CNN) approach to learn the deep frequency features by using a plain rectangular filter with a modified pooling strategy that have more discriminative power for the SER. The proposed CNN model was trained on the extracted frequency features from the speech data and was then tested to predict the emotions. The proposed SER model was evaluated over two benchmarks, which included the interactive emotional dyadic motion capture (IEMOCAP) and the berlin emotional speech database (EMO DB) speech datasets, and it obtained 77.01% and 92.02% recognition results. The experimental results demonstrated that the proposed CNN based SER system can achieve a better recognition performance than the state of the art SER systems.
Sensors
Tursunov Anvarjon Soonil Kwon
218763263
ASAPP ASR: Multistream CNN and Self Attentive SRU for SOTA Speech Recognition
In this paper we present state of the art (SOTA) performance on the LibriSpeech corpus with two novel neural network architectures, a multistream CNN for acoustic modeling and a self attentive simple recurrent unit (SRU) for language modeling. In the hybrid ASR framework, the multistream CNN acoustic model processes an input of speech frames in multiple parallel pipelines where each stream has a unique dilation rate for diversity. Trained with the SpecAugment data augmentation method, it achieves relative word error rate (WER) improvements of 4% on test clean and 14% on test other. We further improve the performance via N best rescoring using a 24 layer self attentive SRU language model, achieving WERs of 1.75% on test clean and 4.46% on test other.
INTERSPEECH
Jing Pan Joshua Shapiro Jeremy Wohlwend Kyu J Han Tao Lei Tao Ma
216051357
Time Frequency Loss for CNN Based Speech Super Resolution
Speech super resolution (SR) also called speech bandwidth extension (BWE) aims to increase the sampling rate of a given lower resolution speech signal. Recent years have witnessed the successful application of deep neural networks in time or frequency domains, and deep learning has improved the performance considerably compared with conventional approaches. This paper proposes an autoencoder based fully convolutional neural network (CNN) that merges the information from both time and frequency domains. At the training time, we optimize the CNN using a new time frequency loss (T F loss) which combines a time domain loss and a frequency domain loss. The experimental results show that our model trained with the T F loss achieves significantly better results than other state of the art models, and yields balanced performance in terms of time and frequency metrics.
ICASSP 2020 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
He-Ming Wang Deliang Wang