- Robin Ruede, Markus Müller, Sebastian Stüker, A. Waibel. 2017. Enhancing Backchannel Prediction Using Word Embeddings. Abstract: Backchannel responses like “uh-huh”, “yeah”, “right” are used by the listener in a social dialog as a way to provide feedback to the speaker. In the context of human-computer interaction, these responses can be used by an artiﬁcial agent to build rapport in conversations with users. In the past, multiple approaches have been proposed to detect backchannel cues and to predict the most natural timing to place those backchannel utterances. Most of these are based on manually optimized ﬁxed rules, which may fail to generalize. Many systems rely on the location and duration of pauses and pitch slopes of speciﬁc lengths. In the past, we proposed an approach by training artiﬁcial neural networks on acoustic features such as pitch and power and also attempted to add word embeddings via word2vec. In this work, we reﬁned this approach by evaluating different methods to add timed word embeddings via word2vec. Comparing the performance using various feature combinations, we could show that adding linguistic features improves the performance over a prediction system that only uses acoustic features.
