@inproceedings{jiang-etal-2023-recall,
    title = "Recall, Expand, and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing",
    author = "Jiang, Chengyue  and
      Hui, Wenyang  and
      Jiang, Yong  and
      Wang, Xiaobin  and
      Xie, Pengjun  and
      Tu, Kewei",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.648",
    doi = "10.18653/v1/2023.acl-long.648",
    pages = "11597--11609",
    abstract = "Ultra-fine entity typing (UFET) predicts extremely free-formed types (e.g., \textit{president, politician}) of a given entity mention (e.g., \textit{Joe Biden}) in context. State-of-the-art (SOTA) methods use the cross-encoder (CE) based architecture. CE concatenates a mention (and its context) with each type and feeds the pair into a pretrained language model (PLM) to score their relevance. It brings deeper interaction between the mention and the type to reach better performance but has to perform $N$ (the type set size) forward passes to infer all the types of a single mention. CE is therefore very slow in inference when the type set is large (e.g., $N=10k$ for UFET). {\%} Cross-encoder also ignores the correlation between different types.To this end, we propose to perform entity typing in a recall-expand-filter manner. The recall and expansion stages prune the large type set and generate $K$ (typically much smaller than $N$) most relevant type candidates for each mention. At the filter stage, we use a novel model called {pasted macro {`}NAME{'}} to concurrently encode and score all these $K$ candidates in only one forward pass to obtain the final type prediction. We investigate different model options for each stage and conduct extensive experiments to compare each option, experiments show that our method reaches SOTA performance on UFET and is thousands of times faster than the CE-based architecture. We also found our method is very effective in fine-grained (130 types) and coarse-grained (9 types) entity typing. Our code is available at {pasted macro {`}CODE{'}}.",
}

Ultra-fine entity typing (UFET) predicts extremely free-formed types (e.g., \textit{president, politician}) of a given entity mention (e.g., \textit{Joe Biden}) in context. State-of-the-art (SOTA) methods use the cross-encoder (CE) based architecture. CE concatenates a mention (and its context) with each type and feeds the pair into a pretrained language model (PLM) to score their relevance. It brings deeper interaction between the mention and the type to reach better performance but has to perform $N$ (the type set size) forward passes to infer all the types of a single mention. CE is therefore very slow in inference when the type set is large (e.g., $N=10k$ for UFET). {\%} Cross-encoder also ignores the correlation between different types.To this end, we propose to perform entity typing in a recall-expand-filter manner. The recall and expansion stages prune the large type set and generate $K$ (typically much smaller than $N$) most relevant type candidates for each mention. At the filter stage, we use a novel model called {pasted macro {`}NAME{'}} to concurrently encode and score all these $K$ candidates in only one forward pass to obtain the final type prediction. We investigate different model options for each stage and conduct extensive experiments to compare each option, experiments show that our method reaches SOTA performance on UFET and is thousands of times faster than the CE-based architecture. We also found our method is very effective in fine-grained (130 types) and coarse-grained (9 types) entity typing. Our code is available at {pasted macro {`}CODE{'}}.
Jiang, Chengyue Hui, Wenyang Jiang, Yong Wang, Xiaobin Xie, Pengjun Tu, Kewei
Recall, Expand, and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing