|
--- |
|
license: apache-2.0 |
|
tags: |
|
- token-classification |
|
datasets: |
|
- conll2003 |
|
- conllpp |
|
language: |
|
- en |
|
metrics: |
|
- f1: 92.85 |
|
- f1(valid): 96.71 |
|
- f1(CoNLLpp(2023)): 92.35 |
|
- f1(CoNLLpp(CrossWeigh)): 94.26 |
|
--- |
|
|
|
|
|
# Roberta-Base-CoNLL2003 |
|
|
|
This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the conll2003 dataset. |
|
|
|
## Model Usage |
|
|
|
We made and used the original tokenizer with [BPE-Dropout](https://aclanthology.org/2020.acl-main.170/). |
|
So, you can't use AutoTokenizer but if subword normalization is not used, original RobertaTokenizer can be substituted. |
|
|
|
Example and Tokenizer Repository: [github](https://github.com/4ldk/CoNLL2003_Choices) |
|
|
|
```python |
|
from transformers import RobertaTokenizer, AutoModelForTokenClassification |
|
from transformers import pipeline |
|
|
|
tokenizer = RobertaTokenizer.from_pretrained("4ldk/Roberta-Base-CoNLL2003") |
|
model = AutoModelForTokenClassification.from_pretrained("4ldk/Roberta-Base-CoNLL2003") |
|
|
|
nlp = pipeline("ner", model=model, tokenizer=tokenizer, grouped_entities=True) |
|
example = "My name is Philipp and live in Germany" |
|
|
|
nlp(example) |
|
|
|
``` |
|
|
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-5 |
|
- train_batch_size: 32 |
|
- eval_batch_size: 32 |
|
- seed: 42 |
|
- optimizer: AdamW with betas=(0.9,0.999), epsilon=1e-08, and weight decay=0.01 |
|
- lr_scheduler_type: linear with warmup rate = 0.1 |
|
- num_epochs: 20 |
|
- subword regularization p = 0.0 (= trained without subword regularization) |
|
|
|
And we add the sentences following the input sentence in the original dataset. Therefore, it cannot be reproduced from the dataset published on huggingface. |
|
|
|
### Training results |
|
|
|
#### CoNNL2003 |
|
|
|
It achieves the following results on the evaluation set: |
|
- Precision: 0.9707 |
|
- Recall: 0.9636 |
|
- F1: 0.9671 |
|
|
|
It achieves the following results on the test set: |
|
- Precision: 0.9352 |
|
- Recall: 0.9218 |
|
- F1: 0.9285 |
|
|
|
#### CoNNLpp(2023) |
|
|
|
[Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023]( |
|
https://aclanthology.org/2023.acl-long.459.pdf) |
|
([github](https://github.com/ShuhengL/acl2023_conllpp)) |
|
|
|
- Precision: 0.9244 |
|
- Recall: 0.9225 |
|
- F1: 0.9235 |
|
|
|
#### CoNLLpp(CrossWeigh) |
|
|
|
[CrossWeigh: Training Named Entity Tagger from Imperfect Annotations](https://aclanthology.org/D19-1519/) |
|
([github](https://github.com/ZihanWangKi/CrossWeigh)) |
|
|
|
- Precision: 0.9449 |
|
- Recall: 0.9403 |
|
- F1: 0.9426 |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.35.2 |
|
- Pytorch 2.0.1+cu117 |
|
|
