
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# BioNLP13CG_bioBERT_NER

This model is a fine-tuned version of [dmis-lab/biobert-v1.1](https://huggingface.co/dmis-lab/biobert-v1.1) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1928
- Seqeval classification report:                                  precision    recall  f1-score   support

                     Amino_acid       0.89      0.88      0.88       576
              Anatomical_system       0.96      0.82      0.89       317
                         Cancer       0.92      0.91      0.91      1649
                           Cell       0.00      0.00      0.00        25
             Cellular_component       0.00      0.00      0.00        12
Developing_anatomical_structure       0.75      0.85      0.80       438
           Gene_or_gene_product       0.87      0.18      0.29        74
   Immaterial_anatomical_entity       0.84      0.84      0.84      4142
         Multi-tissue_structure       0.85      0.84      0.84       451
                          Organ       0.51      0.23      0.31        80
                       Organism       0.52      0.66      0.58       182
           Organism_subdivision       0.81      0.80      0.81       314
             Organism_substance       0.73      0.66      0.69        96
         Pathological_formation       0.75      0.68      0.71       262
                Simple_chemical       0.55      0.44      0.49       112
                         Tissue       0.82      0.91      0.87       300

                      micro avg       0.84      0.82      0.83      9030
                      macro avg       0.67      0.60      0.62      9030
                   weighted avg       0.84      0.82      0.83      9030


## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | Seqeval classification report                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
|:-------------:|:-----:|:----:|:---------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| No log        | 0.99  | 95   | 0.2929          |                                  precision    recall  f1-score   support

                     Amino_acid       0.68      0.81      0.73       576
              Anatomical_system       0.93      0.74      0.82       317
                         Cancer       0.89      0.89      0.89      1649
                           Cell       0.00      0.00      0.00        25
             Cellular_component       0.00      0.00      0.00        12
Developing_anatomical_structure       0.56      0.79      0.65       438
           Gene_or_gene_product       0.00      0.00      0.00        74
   Immaterial_anatomical_entity       0.79      0.76      0.77      4142
         Multi-tissue_structure       0.84      0.75      0.79       451
                          Organ       0.00      0.00      0.00        80
                       Organism       0.62      0.08      0.15       182
           Organism_subdivision       0.64      0.78      0.70       314
             Organism_substance       0.00      0.00      0.00        96
         Pathological_formation       0.63      0.44      0.52       262
                Simple_chemical       0.79      0.13      0.23       112
                         Tissue       0.82      0.45      0.58       300

                      micro avg       0.78      0.72      0.75      9030
                      macro avg       0.51      0.41      0.43      9030
                   weighted avg       0.76      0.72      0.73      9030
 |
| No log        | 2.0   | 191  | 0.2053          |                                  precision    recall  f1-score   support

                     Amino_acid       0.87      0.87      0.87       576
              Anatomical_system       0.98      0.80      0.88       317
                         Cancer       0.89      0.92      0.91      1649
                           Cell       0.00      0.00      0.00        25
             Cellular_component       0.00      0.00      0.00        12
Developing_anatomical_structure       0.74      0.84      0.79       438
           Gene_or_gene_product       1.00      0.05      0.10        74
   Immaterial_anatomical_entity       0.83      0.83      0.83      4142
         Multi-tissue_structure       0.85      0.82      0.83       451
                          Organ       0.48      0.15      0.23        80
                       Organism       0.49      0.66      0.56       182
           Organism_subdivision       0.79      0.80      0.80       314
             Organism_substance       0.75      0.58      0.65        96
         Pathological_formation       0.76      0.66      0.71       262
                Simple_chemical       0.48      0.42      0.45       112
                         Tissue       0.80      0.90      0.85       300

                      micro avg       0.82      0.82      0.82      9030
                      macro avg       0.67      0.58      0.59      9030
                   weighted avg       0.82      0.82      0.81      9030
 |
| No log        | 2.98  | 285  | 0.1928          |                                  precision    recall  f1-score   support

                     Amino_acid       0.89      0.88      0.88       576
              Anatomical_system       0.96      0.82      0.89       317
                         Cancer       0.92      0.91      0.91      1649
                           Cell       0.00      0.00      0.00        25
             Cellular_component       0.00      0.00      0.00        12
Developing_anatomical_structure       0.75      0.85      0.80       438
           Gene_or_gene_product       0.87      0.18      0.29        74
   Immaterial_anatomical_entity       0.84      0.84      0.84      4142
         Multi-tissue_structure       0.85      0.84      0.84       451
                          Organ       0.51      0.23      0.31        80
                       Organism       0.52      0.66      0.58       182
           Organism_subdivision       0.81      0.80      0.81       314
             Organism_substance       0.73      0.66      0.69        96
         Pathological_formation       0.75      0.68      0.71       262
                Simple_chemical       0.55      0.44      0.49       112
                         Tissue       0.82      0.91      0.87       300

                      micro avg       0.84      0.82      0.83      9030
                      macro avg       0.67      0.60      0.62      9030
                   weighted avg       0.84      0.82      0.83      9030
 |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0

{"tags": ["generated_from_trainer"], "base_model": "dmis-lab/biobert-v1.1", "model-index": [{"name": "BioNLP13CG_bioBERT_NER", "results": []}]}
transformers tensorboard safetensors bert token-classification generated_from_trainer base_model:dmis-lab/biobert-v1.1 autotrain_compatible endpoints_compatible region:us