task oriented
218487751
A Simple Language Model for Task Oriented Dialogue
Task oriented dialogue is often decomposed into three tasks: understanding user input, deciding actions, and generating a response. While such decomposition might suggest a dedicated model for each sub task, we find a simple, unified approach leads to state of the art performance on the MultiWOZ dataset. SimpleTOD is a simple approach to task oriented dialogue that uses a single causal language model trained on all sub tasks recast as a single sequence prediction problem. This allows SimpleTOD to fully leverage transfer learning from pre trained, open domain, causal language models such as GPT 2. SimpleTOD improves over the prior state of the art by 0.49 points in joint goal accuracy for dialogue state tracking. More impressively, SimpleTOD also improves the main metrics used to evaluate action decisions and response generation in an end to end setting for task oriented dialog systems: inform rate by 8.1 points, success rate by 9.7 points, and combined score by 7.2 points.
NeurIPS
Ehsan Hosseini-Asl Bryan McCann Chien-Sheng Wu Semih Yavuz Richard Socher
160009896
Transferable Multi Domain State Generator for Task Oriented Dialogue Systems
Over dependence on domain ontology and lack of sharing knowledge across domains are two practical and yet less studied problems of dialogue state tracking. Existing approaches generally fall short when tracking unknown slot values during inference and often have difficulties in adapting to new domains. In this paper, we propose a Transferable Dialogue State Generator (TRADE) that generates dialogue states from utterances using copy mechanism, facilitating transfer when predicting (domain, slot, value) triplets not encountered during training. Our model is composed of an utterance encoder, a slot gate, and a state generator, which are shared across domains. Empirical results demonstrate that TRADE achieves state of the art 48.62% joint goal accuracy for the five domains of MultiWOZ, a human human dialogue dataset. In addition, we show the transferring ability by simulating zero shot and few shot dialogue state tracking for unseen domains. TRADE achieves 60.58% joint goal accuracy in one of the zero shot domains, and is able to adapt to few shot cases without forgetting already trained domains.
ACL
Chien-Sheng Wu Andrea Madotto Ehsan Hosseini-Asl Caiming Xiong Richard Socher Pascale Fung
215768835
TOD BERT: Pre trained Natural Language Understanding for Task Oriented Dialogue
The use of pre trained language models has emerged as a promising direction for improving dialogue systems. However, the underlying difference of linguistic patterns between conversational data and general text makes the existing pre trained language models not as effective as they have been shown to be. Recently, there are some pre training approaches based on open domain dialogues, leveraging large scale social media data such as Twitter or Reddit. Pre training for task oriented dialogues, on the other hand, is rarely discussed because of the long standing and crucial data scarcity problem. In this work, we combine nine English based, human human, multi turn and publicly available task oriented dialogue datasets to conduct language model pre training. The experimental results show that our pre trained task oriented dialogue BERT (ToD BERT) surpasses BERT and other strong baselines in four downstream task oriented dialogue applications, including intention detection, dialogue state tracking, dialogue act prediction, and response selection. Moreover, in the simulated limited data experiments, we show that ToD BERT has stronger few shot capacity that can mitigate the data scarcity problem in task oriented dialogues.
EMNLP
Chien-Sheng Wu Steven C H Hoi Richard Socher Caiming Xiong
218581747
SOLOIST: Few shot Task Oriented Dialog with A Single Pre trained Auto regressive Model
This paper presents a new method SOLOIST, which uses transfer learning to efficiently build task oriented dialog systems at scale. We parameterize a dialog system using a Transformer based auto regressive language model, which subsumes different dialog modules (e.g. state tracker, dialog policy, response generator) into a single neural model. We pre train, on large heterogeneous dialog corpora, a large scale Transformer model which can generate dialog responses grounded in user goals and real world knowledge for task completion. The pre trained model can be efficiently adapted to accomplish a new dialog task with a handful of task specific dialogs via machine teaching. Our experiments demonstrate that (i) SOLOIST creates new state of the art results on two well known benchmarks, CamRest and MultiWOZ, (ii) in the few shot learning setting, the dialog systems developed by SOLOIST significantly outperform those developed by existing methods, and (iii) the use of machine teaching substantially reduces the labeling cost. We will release our code and pre trained models for reproducible research.
ArXiv
Baolin Peng Chunyuan Li Jinchao Li Shahin Shayandeh Lars Liden Jianfeng Gao
211532588
Few shot Natural Language Generation for Task Oriented Dialog
As a crucial component in task oriented dialog systems, the Natural Language Generation (NLG) module converts a dialog act represented in a semantic form into a response in natural language. The success of traditional template based or statistical models typically relies on heavily annotated data, which is infeasible for new domains. Therefore, it is pivotal for an NLG system to generalize well with limited labelled data in real applications. To this end, we present FewshotWOZ, the first NLG benchmark to simulate the few shot learning setting in task oriented dialog systems. Further, we develop the SC GPT model. It is pre trained on a large set of annotated NLG corpus to acquire the controllable generation ability, and fine tuned with only a few domain specific labels to adapt to new domains. Experiments on FewshotWOZ and the large Multi Domain WOZ datasets show that the proposed SC GPT significantly outperforms existing methods, measured by various automatic metrics and human evaluations.
FINDINGS
Baolin Peng Chenguang Zhu Chunyuan Li Xiujun Li Jinchao Li Michael Zeng Jianfeng Gao
208267760
Task Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context
Conversations have an intrinsic one to many property, which means that multiple responses can be appropriate for the same dialog context. In task oriented dialogs, this property leads to different valid dialog policies towards task completion. However, none of the existing task oriented dialog generation approaches takes this property into account. We propose a Multi Action Data Augmentation (MADA) framework to utilize the one to many property to generate diverse appropriate dialog responses. Specifically, we first use dialog states to summarize the dialog history, and then discover all possible mappings from every dialog state to its different valid system actions. During dialog system training, we enable the current dialog state to map to all valid system actions discovered in the previous process to create additional state action pairs. By incorporating these additional pairs, the dialog policy learns a balanced action distribution, which further guides the dialog model to generate diverse responses. Experimental results show that the proposed framework consistently improves dialog policy diversity, and results in improved response diversity and appropriateness. Our model obtains state of the art results on MultiWOZ.
AAAI
Yichi Zhang Zhijian Ou Zhou Yu
208202144
Attention Informed Mixed Language Training for Zero shot Cross lingual Task oriented Dialogue Systems
Recently, data driven task oriented dialogue systems have achieved promising performance in English. However, developing dialogue systems that support low resource languages remains a long standing challenge due to the absence of high quality data. In order to circumvent the expensive and time consuming data collection, we introduce Attention Informed Mixed Language Training (MLT) a novel zero shot adaptation method for cross lingual task oriented dialogue systems. It leverages very few task related parallel word pairs to generate code switching sentences for learning the inter lingual semantics across languages. Instead of manually selecting the word pairs, we propose to extract source words based on the scores computed by the attention layer of a trained English task related model and then generate word pairs using existing bilingual dictionaries. Furthermore, intensive experiments with different cross lingual embeddings demonstrate the effectiveness of our approach. Finally, with very few word pairs, our model achieves significant zero shot adaptation performance improvements in both cross lingual dialogue state tracking and natural language understanding (i.e. intent detection and slot filling) tasks compared to the current state of the art approaches, which utilize a much larger amount of bilingual data.
AAAI
Zihan Liu Genta Indra Winata Zhaojiang Lin Peng Xu Pascale Fung
52897360
MultiWOZ A Large Scale Multi Domain Wizard of Oz Dataset for Task Oriented Dialogue Modelling
Even though machine learning has become the major scene in dialogue research community, the real breakthrough has been blocked by the scale of data available.To address this fundamental obstacle, we introduce the Multi Domain Wizard of Oz dataset (MultiWOZ) a fully labeled collection of human human written conversations spanning over multiple domains and topics.At a size of 10k dialogues, it is at least one order of magnitude larger than all previous annotated task oriented corpora.The contribution of this work apart from the open sourced dataset is two fold:firstly, a detailed description of the data collection procedure along with a summary of data structure and analysis is provided. The proposed data collection pipeline is entirely based on crowd sourcing without the need of hiring professional annotators;secondly, a set of benchmark results of belief tracking, dialogue act and response generation is reported, which shows the usability of the data and sets a baseline for future studies.
EMNLP
Pawel Budzianowski Tsung-Hsien Wen Bo-Hsiang Tseng Inigo Casanueva Stefan Ultes Osman Ramadan Milica Gasic
221995891
DialoGLUE: A Natural Language Understanding Benchmark for Task Oriented Dialogue
A long standing goal of task oriented dialogue research is the ability to flexibly adapt dialogue models to new domains. To progress research in this direction, we introduce DialoGLUE (Dialogue Language Understanding Evaluation) a public benchmark consisting of 7 task oriented dialogue datasets covering 4 distinct natural language understanding tasks, designed to encourage dialogue research in representation based transfer, domain adaptation, and sample efficient task learning. We release several strong baseline models, demonstrating performance improvements over a vanilla BERT architecture and state of the art results on 5 out of 7 tasks, by pre training on a large open domain dialogue corpus and task adaptive self supervised training. Through the DialoGLUE benchmark, the baseline methods, and our evaluation scripts, we hope to facilitate progress towards the goal of developing more general task oriented dialogue models.
ArXiv
Shikib Mehri Mihail Eric Dilek Z Hakkani-Tur
211532256
CrossWOZ: A Large Scale Chinese Cross Domain Task Oriented Dialogue Dataset
Abstract To advance multi domain (cross domain) dialogue modeling as well as alleviate the shortage of Chinese task oriented datasets, we propose CrossWOZ, the first large scale Chinese Cross Domain Wizard of Oz task oriented dataset. It contains 6K dialogue sessions and 102K utterances for 5 domains, including hotel, restaurant, attraction, metro, and taxi. Moreover, the corpus contains rich annotation of dialogue states and dialogue acts on both user and system sides. About 60% of the dialogues have cross domain user goals that favor inter domain dependency and encourage natural transition across domains in conversation. We also provide a user simulator and several benchmark models for pipelined task oriented dialogue systems, which will facilitate researchers to compare and evaluate their models on this corpus. The large size and rich annotation of CrossWOZ make it suitable to investigate a variety of tasks in cross domain dialogue modeling, such as dialogue state tracking, policy learning, user simulation, etc.
Transactions of the Association for Computational Linguistics
Qi Zhu Kaili Huang Zheng Zhang Xiaoyan Zhu Minlie Huang