- Lucas Bechberger, Maria Schmidt, A. Waibel, Marcello Federico. 2016. Personalized News Event Retrieval for Small Talk in Social Dialog Systems. Abstract: This thesis explores the area of personalized news event retrieval in the context of social dialog systems. We developed the NewsTeller system which retrieves a relevant news event based on a user query and the user’s general interests (both represented as list of keywords). The retrieved news event can then be used by a social dialog system to initiate news-related small talk. The NewsTeller system is implemented as a pipeline with four stages: In a rst step, a (large) set of potentially relevant news events is retrieved. As about 84% of the found events do not ful ll our syntactic and semantic well-formedness criteria, the second step in the pipeline is concerned with ltering the found events. This ltering is done by a classi er which was trained on a data set of about 6,000 events. The results obtained in a ten-fold cross-validation indicate that a global random forest classi er is superior to an ensemble of specialized classi ers that were trained on speci c subproblems. The global classi er reaches a precision of 63.04% and a recall of 59.55%. The third step in the pipeline is concerned with ranking the remaining events according to their expected relevance and selecting the event with the highest expected relevance. Four ordered classes are used to describe an event’s relevance: Irrelevant, Partially Relevant, Relevant, and Very Relevant. Following the “learning to rank” approach from information retrieval, this task is framed as a regression problem on the relevance values of the events which is solved by training a random forest regressor on a data set of about 3,200 events. Two di erent approaches were compared: using only features de ned on the events and the user query and using also features de ned on the user’s general interests. The results in a user-level leave-one-out evaluation indicate that both regressors have comparable performance in avoiding Irrelevant events and that taking into account the user’s interests helps to improve the detection of Relevant and Very Relevant events. In the fourth step of the pipeline, a summary of the selected news event is created. This is done by extracting the sentence in which the event was mentioned. For evaluating the system, a user study with 48 participants was conducted. The results of this evaluation show that the two regression-based approaches are signi cantly better than a random baseline with respect to avoiding Irrelevant events. We could however not con rm our hypothesis that using information about the users’ general interests helps to improve the detection of Relevant and Very Relevant events.
