@inproceedings{luo-etal-2023-study,
    title = "A Study on the Efficiency and Generalization of Light Hybrid Retrievers",
    author = "Luo, Man  and
      Jain, Shashank  and
      Gupta, Anchit  and
      Einolghozati, Arash  and
      Oguz, Barlas  and
      Chatterjee, Debojeet  and
      Chen, Xilun  and
      Baral, Chitta  and
      Heidari, Peyman",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-short.139",
    doi = "10.18653/v1/2023.acl-short.139",
    pages = "1617--1626",
    abstract = "Hybrid retrievers can take advantage of both sparse and dense retrievers. Previous hybrid retrievers leverage indexing-heavy dense retrievers. In this work, we study {``}Is it possible to reduce the indexing memory of hybrid retrievers without sacrificing performance{''}? Driven by this question, we leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. LITE is jointly trained on contrastive learning and knowledge distillation from DrBoost. Then, we integrate BM25, a sparse retriever, with either LITE or DrBoost to form light hybrid retrievers. Our Hybrid-LITE retriever saves $13\times$ memory while maintaining 98.0{\%} performance of the hybrid retriever of BM25 and DPR. In addition, we study the generalization capacity of our light hybrid retrievers on out-of-domain dataset and a set of adversarial attacks datasets. Experiments showcase that light hybrid retrievers achieve better generalization performance than individual sparse and dense retrievers. Nevertheless, our analysis shows that there is a large room to improve the robustness of retrievers, suggesting a new research direction.",
}

Hybrid retrievers can take advantage of both sparse and dense retrievers. Previous hybrid retrievers leverage indexing-heavy dense retrievers. In this work, we study {``}Is it possible to reduce the indexing memory of hybrid retrievers without sacrificing performance{''}? Driven by this question, we leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. LITE is jointly trained on contrastive learning and knowledge distillation from DrBoost. Then, we integrate BM25, a sparse retriever, with either LITE or DrBoost to form light hybrid retrievers. Our Hybrid-LITE retriever saves $13\times$ memory while maintaining 98.0{\%} performance of the hybrid retriever of BM25 and DPR. In addition, we study the generalization capacity of our light hybrid retrievers on out-of-domain dataset and a set of adversarial attacks datasets. Experiments showcase that light hybrid retrievers achieve better generalization performance than individual sparse and dense retrievers. Nevertheless, our analysis shows that there is a large room to improve the robustness of retrievers, suggesting a new research direction.
Luo, Man Jain, Shashank Gupta, Anchit Einolghozati, Arash Oguz, Barlas Chatterjee, Debojeet Chen, Xilun Baral, Chitta Heidari, Peyman
A Study on the Efficiency and Generalization of Light Hybrid Retrievers