@inproceedings{moradshahi-etal-2023-x,
    title = "{X}-{R}i{SAWOZ}: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents",
    author = "Moradshahi, Mehrad  and
      Shen, Tianhao  and
      Bali, Kalika  and
      Choudhury, Monojit  and
      de Chalendar, Gael  and
      Goel, Anmol  and
      Kim, Sungkyun  and
      Kodali, Prashant  and
      Kumaraguru, Ponnurangam  and
      Semmar, Nasredine  and
      Semnani, Sina  and
      Seo, Jiwon  and
      Seshadri, Vivek  and
      Shrivastava, Manish  and
      Sun, Michael  and
      Yadavalli, Aditya  and
      You, Chaobin  and
      Xiong, Deyi  and
      Lam, Monica",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.174",
    doi = "10.18653/v1/2023.findings-acl.174",
    pages = "2773--2794",
    abstract = "Task-oriented dialogue research has mainly focused on a few popular languages like English and Chinese, due to the high dataset creation cost for a new language. To reduce the cost, we apply manual editing to automatically translated data. We create a new multilingual benchmark, X-RiSAWOZ, by translating the Chinese RiSAWOZ to 4 languages: English, French, Hindi, Korean; and a code-mixed English-Hindi language.X-RiSAWOZ has more than 18,000 human-verified dialogue utterances for each language, and unlike most multilingual prior work, is an end-to-end dataset for building fully-functioning agents. The many difficulties we encountered in creating X-RiSAWOZ led us to develop a toolset to accelerate the post-editing of a new language dataset after translation. This toolset improves machine translation with a hybrid entity alignment technique that combines neural with dictionary-based methods, along with many automated and semi-automated validation checks. We establish strong baselines for X-RiSAWOZ by training dialogue agents in the zero- and few-shot settings where limited gold data is available in the target language. Our results suggest that our translation and post-editing methodology and toolset can be used to create new high-quality multilingual dialogue agents cost-effectively. Our dataset, code, and toolkit are released open-source.",
}

Task-oriented dialogue research has mainly focused on a few popular languages like English and Chinese, due to the high dataset creation cost for a new language. To reduce the cost, we apply manual editing to automatically translated data. We create a new multilingual benchmark, X-RiSAWOZ, by translating the Chinese RiSAWOZ to 4 languages: English, French, Hindi, Korean; and a code-mixed English-Hindi language.X-RiSAWOZ has more than 18,000 human-verified dialogue utterances for each language, and unlike most multilingual prior work, is an end-to-end dataset for building fully-functioning agents. The many difficulties we encountered in creating X-RiSAWOZ led us to develop a toolset to accelerate the post-editing of a new language dataset after translation. This toolset improves machine translation with a hybrid entity alignment technique that combines neural with dictionary-based methods, along with many automated and semi-automated validation checks. We establish strong baselines for X-RiSAWOZ by training dialogue agents in the zero- and few-shot settings where limited gold data is available in the target language. Our results suggest that our translation and post-editing methodology and toolset can be used to create new high-quality multilingual dialogue agents cost-effectively. Our dataset, code, and toolkit are released open-source.
Moradshahi, Mehrad Shen, Tianhao Bali, Kalika Choudhury, Monojit de Chalendar, Gael Goel, Anmol Kim, Sungkyun Kodali, Prashant Kumaraguru, Ponnurangam Semmar, Nasredine Semnani, Sina Seo, Jiwon Seshadri, Vivek Shrivastava, Manish Sun, Michael Yadavalli, Aditya You, Chaobin Xiong, Deyi Lam, Monica
X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents