@inproceedings{mehrabi-etal-2023-resolving,
    title = "Resolving Ambiguities in Text-to-Image Generative Models",
    author = "Mehrabi, Ninareh  and
      Goyal, Palash  and
      Verma, Apurv  and
      Dhamala, Jwala  and
      Kumar, Varun  and
      Hu, Qian  and
      Chang, Kai-Wei  and
      Zemel, Richard  and
      Galstyan, Aram  and
      Gupta, Rahul",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.804",
    doi = "10.18653/v1/2023.acl-long.804",
    pages = "14367--14388",
    abstract = "Natural language often contains ambiguities that can lead to misinterpretation and miscommunication. While humans can handle ambiguities effectively by asking clarifying questions and/or relying on contextual cues and common-sense knowledge, resolving ambiguities can be notoriously hard for machines. In this work, we study ambiguities that arise in text-to-image generative models. We curate the Text-to-image Ambiguity Benchmark (TAB) dataset to study different types of ambiguities in text-to-image generative models. We then propose the Text-to-ImagE Disambiguation (TIED) framework to disambiguate the prompts given to the text-to-image generative models by soliciting clarifications from the end user. Through automatic and human evaluations, we show the effectiveness of our framework in generating more faithful images aligned with end user intention in the presence of ambiguities.",
}

Natural language often contains ambiguities that can lead to misinterpretation and miscommunication. While humans can handle ambiguities effectively by asking clarifying questions and/or relying on contextual cues and common-sense knowledge, resolving ambiguities can be notoriously hard for machines. In this work, we study ambiguities that arise in text-to-image generative models. We curate the Text-to-image Ambiguity Benchmark (TAB) dataset to study different types of ambiguities in text-to-image generative models. We then propose the Text-to-ImagE Disambiguation (TIED) framework to disambiguate the prompts given to the text-to-image generative models by soliciting clarifications from the end user. Through automatic and human evaluations, we show the effectiveness of our framework in generating more faithful images aligned with end user intention in the presence of ambiguities.
Mehrabi, Ninareh Goyal, Palash Verma, Apurv Dhamala, Jwala Kumar, Varun Hu, Qian Chang, Kai-Wei Zemel, Richard Galstyan, Aram Gupta, Rahul
Resolving Ambiguities in Text-to-Image Generative Models