@inproceedings{yu-etal-2023-regen,
    title = "{R}e{G}en: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval",
    author = "Yu, Yue  and
      Zhuang, Yuchen  and
      Zhang, Rongzhi  and
      Meng, Yu  and
      Shen, Jiaming  and
      Zhang, Chao",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.748",
    doi = "10.18653/v1/2023.findings-acl.748",
    pages = "11782--11805",
    abstract = "With the development of large language models (LLMs), zero-shot learning has attracted much attention for various NLP tasks. Different from prior works that generate training data with billion-scale natural language generation (NLG) models, we propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. To realize this, we first conduct contrastive pretraining to learn an unsupervised dense retriever for extracting the most relevant documents using class-descriptive verbalizers. We then further pro- pose two simple strategies, namely Verbalizer Augmentation with Demonstrations and Self- consistency Guided Filtering to improve the topic coverage of the dataset while removing noisy examples. Experiments on nine datasets demonstrate that ReGen achieves 4.3{\%} gain over the strongest baselines and saves around 70{\%} of the time when compared with baselines using large NLG models. Besides, REGEN can be naturally integrated with recently proposed large language models to boost performance.",
}

With the development of large language models (LLMs), zero-shot learning has attracted much attention for various NLP tasks. Different from prior works that generate training data with billion-scale natural language generation (NLG) models, we propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. To realize this, we first conduct contrastive pretraining to learn an unsupervised dense retriever for extracting the most relevant documents using class-descriptive verbalizers. We then further pro- pose two simple strategies, namely Verbalizer Augmentation with Demonstrations and Self- consistency Guided Filtering to improve the topic coverage of the dataset while removing noisy examples. Experiments on nine datasets demonstrate that ReGen achieves 4.3{\%} gain over the strongest baselines and saves around 70{\%} of the time when compared with baselines using large NLG models. Besides, REGEN can be naturally integrated with recently proposed large language models to boost performance.
Yu, Yue Zhuang, Yuchen Zhang, Rongzhi Meng, Yu Shen, Jiaming Zhang, Chao
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval