- M. Baskar, Shinji Watanabe, Ramón Fernández Astudillo, L. Burget, J. Černocký. 2020. EAT: Enhanced ASR-TTS Framework for Self-supervised ASR. Abstract: Self-supervised training has drawn major bene-ﬁts for unsupervised speech recognition tasks. In this paper, we propose an enhanced ASR-TTS (EAT) framework using cycle-consistency algo-rithm by introducing two main features: 1) The ASR → TTS pipeline is included with a language model reward to penalise the ASR hypotheses before forwarding to TTS. 2) In TTS → ASR, a hyper-parameter is introduced to scale the attention context from synthesized speech before sending it to ASR to handle out-of-domain data. We explore the effectiveness of the EAT framework under different data conditions and latest architectures. The results show that EAT reduces the performance gap between supervised and unsupervised to 17.1% and 20.1% on Librispeech and BABEL corpus respectively.
