- Photina Jaeyun Jang, Alexander Hauptmann. 2003. TIVA Learning to Recognize Speech by Watching Television. Abstract: Imagine a computer plugged into your television at home or in a foreign hotel. In the morning, it can barely transcribe one out of two words correctly, but by evening, it can provide a largely correct transcript of the evening news show, based on what it learned during the day from the TV. This seems like the core of familiar science fiction shows, but the research described here brings this vision closer to reality by exploiting the multimodal nature of the broadcast signals and the variety of sources available for broadcast data. Our proposed technique gathers large amounts of speech from open broadcast sources and combines it with automatically obtained text or closed captioning to identify suitable speechtraining material. George Zavaliagkos and Thomas Colthurst worked on a different approach to this method that uses confidence scoring on the acoustic data itself to improve performance in the absence of any transcribed data, but their approach only yielded marginal results. Our initial efforts also provided only limited success with small amounts of data. In this article, we describe our approach to collecting almost unlimited amounts of accurately transcribed speech data. This information serves as training data for the acoustic-model component of most high-accuracy speaker-independent speech-recognition systems. The errorridden closed-captioned text aligns with the similarly error-ridden speech-recognizer output. We assume matching segments of sufficient length are reliable transcriptions of the corresponding speech. We then use these segments as the training data for an improved speech recognizer.
