
# Fine-tuned English-Georgian NER Model with Flair

This Flair NER model was fine-tuned on the WikiANN dataset
([Rahimi et al.](https://www.aclweb.org/anthology/P19-1015) splits)
using XLM-R Large as backbone LM.

**Notice**: The dataset is very problematic, because it was automatically constructed.

We did manually inspect the development split of the Georgian data and found
a lot of bad labeled examples, e.g. DVD ( 💿 ) as `ORG`.

## Fine-Tuning

The latest
[Flair version](https://github.com/flairNLP/flair/tree/f30f5801df3f9e105ed078ec058b4e1152dd9159)
is used for fine-tuning.

We use English and Georgian training splits for fine-tuning and the
development set of Georgian for evaluation.

A hyper-parameter search over the following parameters with 5 different seeds per configuration is performed:

* Batch Sizes: [`4`]
* Learning Rates: [`5e-06`]

More details can be found in this [repository](https://github.com/stefan-it/georgian-ner).

## Results

A hyper-parameter search with 5 different seeds per configuration is performed and micro F1-score on development set
is reported:

| Configuration     | Seed 1      | Seed 2      | Seed 3      | Seed 4     | Seed 5          | Average         |
|-------------------|-------------|-------------|-------------|------------|-----------------|-----------------|
| `bs4-e10-lr5e-06` | [0.9005][1] | [0.9012][2] | [0.9069][3] | [0.905][4] | [**0.9048**][5] | 0.9037 ± 0.0027 |

[1]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-1
[2]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-2
[3]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-3
[4]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-4
[5]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-5

The result in bold shows the performance of this model.

Additionally, the Flair [training log](training.log) and [TensorBoard logs](tensorboard) are also uploaded to the model
hub.
{"language": ["en", "ka"], "license": "mit", "tags": ["flair", "token-classification", "sequence-tagger-model"], "base_model": "xlm-roberta-large", "widget": [{"text": "\u10d0\u10db\u10d8\u10d7 \u10d7\u10d0\u10d5\u10d8\u10e1\u10d8 \u10e5\u10d0\u10d3\u10d0\u10d2\u10d4\u10d1\u10d0 \u10d3\u10d0\u10d0\u10e1\u10e0\u10e3\u10da\u10d0 \u10d3\u10d0 \u10d3\u10d0\u10d1\u10e0\u10e3\u10dc\u10d3\u10d0 \u10d8\u10d4\u10e0\u10e3\u10e1\u10d0\u10da\u10d8\u10db\u10e8\u10d8 . \u10d4\u10e0\u10d7-\u10d4\u10e0\u10d7 \u10d2\u10d0\u10e0\u10d4 \u10d9\u10d4\u10d3\u10d4\u10da\u10d6\u10d4 \u10d0\u10e0\u10e1\u10d4\u10d1\u10dd\u10d1\u10e1 \u10d4\u10e0\u10dc\u10d4\u10e1\u10e2\u10dd \u10e9\u10d4 \u10d2\u10d4\u10d5\u10d0\u10e0\u10d0\u10e1 \u10de\u10dd\u10e0\u10e2\u10e0\u10d4\u10e2\u10d8 . \u10e8\u10d0\u10d9\u10dd\u10e1\u10d9\u10d0\u201c \u10d8\u10dc\u10d0\u10ee\u10d4\u10d1\u10d0 \u10d1\u10e0\u10d0\u10d6\u10d8\u10da\u10d8\u10d0\u10e8\u10d8 , \u10e1\u10d0\u10dc-\u10de\u10d0\u10e3\u10da\u10e3\u10e1 \u10ee\u10d4\u10da\u10dd\u10d5\u10dc\u10d4\u10d1\u10d8\u10e1 \u10db\u10e3\u10d6\u10d4\u10e3\u10db\u10e8\u10d8 ."}]}
flair pytorch tensorboard token-classification sequence-tagger-model en ka base_model:xlm-roberta-large license:mit region:us