SpEL Entity Linking Structured Prediction en dataset:CoNLL2003/AIDA dataset:Wikipedia dataset:sshavara/AIDA_testc arxiv:2310.14684 license:cc-by-4.0 region:us
---
license: cc-by-4.0
language:
- en
datasets:
- CoNLL2003/AIDA
- Wikipedia
- sshavara/AIDA_testc
tags:
- SpEL
- Entity Linking
- Structured Prediction
widget:
- text: "Leicestershire beat Somerset by an innings and 39 runs in two days."
---

## SpEL (Structured prediction for Entity Linking)
SpEL model finetuned on English Wikipedia as well as the training portion of CoNLL2003/AIDA. 
It is introduced in the paper [SPEL: Structured Prediction for Entity Linking (EMNLP 2023)](https://arxiv.org/abs/2310.14684). 
The code and data are available in [this repository](https://github.com/shavarani/SpEL).

### Usage
The following snippet demonstrates a quick way that SpEL can be used to generate subword-level, word-level, and phrase-level annotations for a sentence.

```python
# download SpEL from https://github.com/shavarani/SpEL
from transformers import AutoTokenizer
from spel.model import SpELAnnotator
from spel.configuration import device
from spel.utils import get_subword_to_word_mapping
from spel.span_annotation import WordAnnotation, PhraseAnnotation
finetuned_after_step = 4
sentence = "Grace Kelly by Mika reached the top of the UK Singles Chart in 2007."
tokenizer = AutoTokenizer.from_pretrained("roberta-base")
# ############################################# LOAD SpEL #############################################################
spel = SpELAnnotator()
spel.init_model_from_scratch(device=device)
if finetuned_after_step == 3:
    spel.shrink_classification_head_to_aida(device)
spel.load_checkpoint(None, device=device, load_from_torch_hub=True, finetuned_after_step=finetuned_after_step)
# ############################################# RUN SpEL ##############################################################
inputs = tokenizer(sentence, return_tensors="pt")
token_offsets = list(zip(inputs.encodings[0].tokens,inputs.encodings[0].offsets))
subword_annotations = spel.annotate_subword_ids(inputs.input_ids, k_for_top_k_to_keep=10, token_offsets=token_offsets)
# #################################### CREATE WORD-LEVEL ANNOTATIONS ##################################################
tokens_offsets = token_offsets[1:-1]
subword_annotations = subword_annotations[1:]
word_annotations = [WordAnnotation(subword_annotations[m[0]:m[1]], tokens_offsets[m[0]:m[1]])
                    for m in get_subword_to_word_mapping(inputs.tokens(), sentence)]
# ################################## CREATE PHRASE-LEVEL ANNOTATIONS ##################################################
phrase_annotations = []
for w in word_annotations:
    if not w.annotations:
        continue
    if phrase_annotations and phrase_annotations[-1].resolved_annotation == w.resolved_annotation:
        phrase_annotations[-1].add(w)
    else:
        phrase_annotations.append(PhraseAnnotation(w))
```





## Evaluation Results
Entity Linking evaluation results of *SpEL* compared to that of the literature over AIDA test sets:

| Approach                                                        | EL Micro-F1<br/>test-a | EL Micro-F1<br/>test-b |            #params<br/>on GPU            | speed<br/>sec/doc |
|-----------------------------------------------------------------|:----------------------:|:----------------------:|:----------------------------------------:|:-----------------:|
| Hoffart et al. (2011)                                           |          72.4          |          72.8          |                    -                     |         -         |
| Kolitsas et al. (2018)                                          |          89.4          |          82.4          |                  330.7M                  |       0.097       |
| Broscheit (2019)                                                |          86.0          |          79.3          |                  495.1M                  |       0.613       |
| Peters et al. (2019)                                            |          82.1          |          73.1          |                    -                     |         -         |
| Martins et al. (2019)                                           |          85.2          |          81.9          |                    -                     |         -         |
| van Hulst et al. (2020)                                         |          83.3          |          82.4          |                  19.0M                   |       0.337       |
| Févry et al. (2020)                                             |          79.7          |          76.7          |                    -                     |         -         |
| Poerner et al. (2020)                                           |          90.8          |          85.0          |                  131.1M                  |         -         |
| Kannan Ravi et al. (2021)                                       |           -            |          83.1          |                    -                     |         -         |
| De Cao et al. (2021b)                                           |           -            |          83.7          |                  406.3M                  |      40.969       |
| De Cao et al. (2021a)<br/>(no mention-specific candidate set)   |          61.9          |          49.4          |                  124.8M                  |       0.268       |
| De Cao et al. (2021a)<br/>(using PPRforNED candidate set)       |          90.1          |          85.5          |                  124.8M                  |       0.194       |
| Mrini et al. (2022)                                             |           -            |          85.7          |  (train) 811.5M<br/>(test) 406.2M        |         -         |
| Zhang et al. (2022)                                             |           -            |          85.8          |                 1004.3M                  |         -         |
| Feng et al. (2022)                                              |           -            |          86.3          |                  157.3M                  |         -         |
| <hr/>                                                           |         <hr/>          |         <hr/>          |                  <hr/>                   |       <hr/>       |
| **SpEL-base** (no mention-specific candidate set)               |          91.3          |          85.5          |                  128.9M                  |       0.084       |
| **SpEL-base** (KB+Yago candidate set)                           |          90.6          |          85.7          |                  128.9M                  |       0.158       |
| **SpEL-base** (PPRforNED candidate set)<br/>(context-agnostic)  |          91.7          |          86.8          |                  128.9M                  |       0.153       |
| **SpEL-base** (PPRforNED candidate set)<br/>(context-aware)     |          92.7          |          88.1          |                  128.9M                  |       0.156       |
| **SpEL-large** (no mention-specific candidate set)              |          91.6          |          85.8          |                  361.1M                  |       0.273       |
| **SpEL-large** (KB+Yago candidate set)                          |          90.8          |          85.7          |                  361.1M                  |       0.267       |
| **SpEL-large** (PPRforNED candidate set)<br/>(context-agnostic) |          92.0          |          87.3          |                  361.1M                  |       0.268       |
| **SpEL-large** (PPRforNED candidate set)<br/>(context-aware)    |          92.9          |          88.6          |                  361.1M                  |       0.267       |

----

## Citation
If you use SpEL finetuned models or data, please cite our paper:

```
@inproceedings{shavarani2023spel,
  title={Sp{EL}: Structured Prediction for Entity Linking},
  author={Shavarani, Hassan S.  and  Sarkar, Anoop},
  booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
  year={2023},
  url={https://arxiv.org/abs/2310.14684}
}
```