@inproceedings{heck-etal-2023-chatgpt,
    title = "{C}hat{GPT} for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?",
    author = "Heck, Michael  and
      Lubis, Nurul  and
      Ruppik, Benjamin  and
      Vukovic, Renato  and
      Feng, Shutong  and
      Geishauser, Christian  and
      Lin, Hsien-chin  and
      van Niekerk, Carel  and
      Gasic, Milica",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-short.81",
    doi = "10.18653/v1/2023.acl-short.81",
    pages = "936--950",
    abstract = "Recent research on dialog state tracking (DST) focuses on methods that allow few- and zero-shot transfer to new domains or schemas. However, performance gains heavily depend on aggressive data augmentation and fine-tuning of ever larger language model based architectures. In contrast, general purpose language models, trained on large amounts of diverse data, hold the promise of solving any kind of task without task-specific training. We present preliminary experimental results on the ChatGPT research preview, showing that ChatGPT achieves state-of-the-art performance in zero-shot DST. Despite our findings, we argue that properties inherent to general purpose models limit their ability to replace specialized systems. We further theorize that the in-context learning capabilities of such models will likely become powerful tools to support the development of dedicated dialog state trackers and enable dynamic methods.",
}

Recent research on dialog state tracking (DST) focuses on methods that allow few- and zero-shot transfer to new domains or schemas. However, performance gains heavily depend on aggressive data augmentation and fine-tuning of ever larger language model based architectures. In contrast, general purpose language models, trained on large amounts of diverse data, hold the promise of solving any kind of task without task-specific training. We present preliminary experimental results on the ChatGPT research preview, showing that ChatGPT achieves state-of-the-art performance in zero-shot DST. Despite our findings, we argue that properties inherent to general purpose models limit their ability to replace specialized systems. We further theorize that the in-context learning capabilities of such models will likely become powerful tools to support the development of dedicated dialog state trackers and enable dynamic methods.
Heck, Michael Lubis, Nurul Ruppik, Benjamin Vukovic, Renato Feng, Shutong Geishauser, Christian Lin, Hsien-chin van Niekerk, Carel Gasic, Milica
ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?