@inproceedings{deguchi-etal-2023-subset,
    title = "Subset Retrieval Nearest Neighbor Machine Translation",
    author = "Deguchi, Hiroyuki  and
      Watanabe, Taro  and
      Matsui, Yusuke  and
      Utiyama, Masao  and
      Tanaka, Hideki  and
      Sumita, Eiichiro",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.10",
    doi = "10.18653/v1/2023.acl-long.10",
    pages = "174--189",
    abstract = "k-nearest-neighbor machine translation (kNN-MT) (Khandelwal et al., 2021) boosts the translation performance of trained neural machine translation (NMT) models by incorporating example-search into the decoding algorithm. However, decoding is seriously time-consuming, i.e., roughly 100 to 1,000 times slower than standard NMT, because neighbor tokens are retrieved from all target tokens of parallel data in each timestep. In this paper, we propose {``}Subset kNN-MT{''}, which improves the decoding speed of kNN-MT by two methods: (1) retrieving neighbor target tokens from a subset that is the set of neighbor sentences of the input sentence, not from all sentences, and (2) efficient distance computation technique that is suitable for subset neighbor search using a look-up table. Our proposed method achieved a speed-up of up to 132.2 times and an improvement in BLEU score of up to 1.6 compared with kNN-MT in the WMT{'}19 De-En translation task and the domain adaptation tasks in De-En and En-Ja.",
}

k-nearest-neighbor machine translation (kNN-MT) (Khandelwal et al., 2021) boosts the translation performance of trained neural machine translation (NMT) models by incorporating example-search into the decoding algorithm. However, decoding is seriously time-consuming, i.e., roughly 100 to 1,000 times slower than standard NMT, because neighbor tokens are retrieved from all target tokens of parallel data in each timestep. In this paper, we propose {``}Subset kNN-MT{''}, which improves the decoding speed of kNN-MT by two methods: (1) retrieving neighbor target tokens from a subset that is the set of neighbor sentences of the input sentence, not from all sentences, and (2) efficient distance computation technique that is suitable for subset neighbor search using a look-up table. Our proposed method achieved a speed-up of up to 132.2 times and an improvement in BLEU score of up to 1.6 compared with kNN-MT in the WMT{'}19 De-En translation task and the domain adaptation tasks in De-En and En-Ja.
Deguchi, Hiroyuki Watanabe, Taro Matsui, Yusuke Utiyama, Masao Tanaka, Hideki Sumita, Eiichiro
Subset Retrieval Nearest Neighbor Machine Translation