
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# SciBERT_100K_steps

This model is a fine-tuned version of [allenai/scibert_scivocab_uncased](https://huggingface.co/allenai/scibert_scivocab_uncased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0144
- Accuracy: 0.9947
- Precision: 0.7850
- Recall: 0.6355
- F1: 0.7024
- Hamming: 0.0053

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- training_steps: 100000

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Accuracy | Precision | Recall | F1     | Hamming |
|:-------------:|:-----:|:------:|:---------------:|:--------:|:---------:|:------:|:------:|:-------:|
| 0.1681        | 0.08  | 5000   | 0.0487          | 0.9902   | 0.0       | 0.0    | 0.0    | 0.0098  |
| 0.032         | 0.16  | 10000  | 0.0223          | 0.9930   | 0.8068    | 0.3728 | 0.5100 | 0.0070  |
| 0.0201        | 0.24  | 15000  | 0.0186          | 0.9937   | 0.7815    | 0.4970 | 0.6076 | 0.0063  |
| 0.018         | 0.32  | 20000  | 0.0172          | 0.9941   | 0.7763    | 0.5550 | 0.6472 | 0.0059  |
| 0.017         | 0.4   | 25000  | 0.0166          | 0.9942   | 0.7864    | 0.5624 | 0.6558 | 0.0058  |
| 0.0166        | 0.47  | 30000  | 0.0163          | 0.9943   | 0.7707    | 0.5880 | 0.6671 | 0.0057  |
| 0.0163        | 0.55  | 35000  | 0.0160          | 0.9943   | 0.7802    | 0.5809 | 0.6659 | 0.0057  |
| 0.0159        | 0.63  | 40000  | 0.0158          | 0.9944   | 0.7719    | 0.6012 | 0.6759 | 0.0056  |
| 0.0157        | 0.71  | 45000  | 0.0155          | 0.9945   | 0.7750    | 0.6104 | 0.6829 | 0.0055  |
| 0.0154        | 0.79  | 50000  | 0.0153          | 0.9945   | 0.7734    | 0.6202 | 0.6884 | 0.0055  |
| 0.0153        | 0.87  | 55000  | 0.0151          | 0.9945   | 0.7823    | 0.6072 | 0.6837 | 0.0055  |
| 0.0152        | 0.95  | 60000  | 0.0151          | 0.9945   | 0.7813    | 0.6124 | 0.6866 | 0.0055  |
| 0.0148        | 1.03  | 65000  | 0.0149          | 0.9946   | 0.7843    | 0.6208 | 0.6930 | 0.0054  |
| 0.0143        | 1.11  | 70000  | 0.0148          | 0.9946   | 0.7802    | 0.6231 | 0.6929 | 0.0054  |
| 0.0142        | 1.19  | 75000  | 0.0148          | 0.9946   | 0.7714    | 0.6377 | 0.6982 | 0.0054  |
| 0.0141        | 1.27  | 80000  | 0.0146          | 0.9947   | 0.7837    | 0.6281 | 0.6973 | 0.0053  |
| 0.0141        | 1.34  | 85000  | 0.0146          | 0.9947   | 0.7836    | 0.6374 | 0.7030 | 0.0053  |
| 0.014         | 1.42  | 90000  | 0.0145          | 0.9947   | 0.7859    | 0.6326 | 0.7010 | 0.0053  |
| 0.0139        | 1.5   | 95000  | 0.0145          | 0.9947   | 0.7875    | 0.6317 | 0.7010 | 0.0053  |
| 0.0139        | 1.58  | 100000 | 0.0144          | 0.9947   | 0.7850    | 0.6355 | 0.7024 | 0.0053  |


### Framework versions

- Transformers 4.35.0.dev0
- Pytorch 2.0.1+cu118
- Datasets 2.7.1
- Tokenizers 0.14.1

{"tags": ["generated_from_trainer"], "metrics": ["accuracy", "precision", "recall", "f1"], "base_model": "allenai/scibert_scivocab_uncased", "model-index": [{"name": "SciBERT_100K_steps", "results": []}]}
transformers pytorch bert text-classification generated_from_trainer base_model:allenai/scibert_scivocab_uncased autotrain_compatible endpoints_compatible region:us