
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# roberta-large-lora-token-classification

This model is a fine-tuned version of [roberta-large](https://huggingface.co/roberta-large) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2546
- Precision: 0.9604
- Recall: 0.9600
- F1-score: 0.9601
- Accuracy: 0.9598

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1-score | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:--------:|:--------:|
| No log        | 0.11  | 25   | 0.9686          | 0.6063    | 0.5640 | 0.5392   | 0.5617   |
| No log        | 0.23  | 50   | 0.9781          | 0.5396    | 0.5383 | 0.4969   | 0.5362   |
| No log        | 0.34  | 75   | 0.9558          | 0.6942    | 0.4627 | 0.3789   | 0.4651   |
| No log        | 0.46  | 100  | 0.8439          | 0.6372    | 0.5855 | 0.5856   | 0.5871   |
| No log        | 0.57  | 125  | 0.6714          | 0.7624    | 0.7533 | 0.7539   | 0.7520   |
| No log        | 0.69  | 150  | 0.6038          | 0.7961    | 0.7635 | 0.7693   | 0.7641   |
| No log        | 0.8   | 175  | 0.3989          | 0.8356    | 0.8362 | 0.8358   | 0.8351   |
| No log        | 0.92  | 200  | 0.3601          | 0.8801    | 0.8600 | 0.8523   | 0.8579   |
| No log        | 1.03  | 225  | 0.3550          | 0.8900    | 0.8797 | 0.8752   | 0.8780   |
| No log        | 1.15  | 250  | 0.3047          | 0.8921    | 0.8894 | 0.8897   | 0.8887   |
| No log        | 1.26  | 275  | 0.3499          | 0.9155    | 0.9162 | 0.9150   | 0.9155   |
| No log        | 1.38  | 300  | 0.3978          | 0.8969    | 0.8890 | 0.8885   | 0.8887   |
| No log        | 1.49  | 325  | 0.2146          | 0.9293    | 0.9220 | 0.9206   | 0.9209   |
| No log        | 1.61  | 350  | 0.2954          | 0.9274    | 0.9283 | 0.9273   | 0.9276   |
| No log        | 1.72  | 375  | 0.4710          | 0.9100    | 0.9005 | 0.9022   | 0.9008   |
| No log        | 1.83  | 400  | 0.2987          | 0.9193    | 0.9174 | 0.9160   | 0.9169   |
| No log        | 1.95  | 425  | 0.2549          | 0.9442    | 0.9403 | 0.9401   | 0.9397   |
| No log        | 2.06  | 450  | 0.3173          | 0.9396    | 0.9401 | 0.9397   | 0.9397   |
| No log        | 2.18  | 475  | 0.3349          | 0.9491    | 0.9484 | 0.9477   | 0.9477   |
| 0.5562        | 2.29  | 500  | 0.3155          | 0.9440    | 0.9442 | 0.9438   | 0.9437   |
| 0.5562        | 2.41  | 525  | 0.2871          | 0.9440    | 0.9442 | 0.9438   | 0.9437   |
| 0.5562        | 2.52  | 550  | 0.2983          | 0.9499    | 0.9496 | 0.9491   | 0.9491   |
| 0.5562        | 2.64  | 575  | 0.2736          | 0.9504    | 0.9482 | 0.9481   | 0.9477   |
| 0.5562        | 2.75  | 600  | 0.3900          | 0.9397    | 0.9386 | 0.9390   | 0.9383   |
| 0.5562        | 2.87  | 625  | 0.3460          | 0.9511    | 0.9510 | 0.9503   | 0.9504   |
| 0.5562        | 2.98  | 650  | 0.3509          | 0.9477    | 0.9445 | 0.9437   | 0.9437   |
| 0.5562        | 3.1   | 675  | 0.2368          | 0.9558    | 0.9551 | 0.9545   | 0.9544   |
| 0.5562        | 3.21  | 700  | 0.2279          | 0.9601    | 0.9591 | 0.9585   | 0.9584   |
| 0.5562        | 3.33  | 725  | 0.2538          | 0.9539    | 0.9537 | 0.9531   | 0.9531   |
| 0.5562        | 3.44  | 750  | 0.2577          | 0.9504    | 0.9509 | 0.9506   | 0.9504   |
| 0.5562        | 3.56  | 775  | 0.1994          | 0.9548    | 0.9548 | 0.9546   | 0.9544   |
| 0.5562        | 3.67  | 800  | 0.2247          | 0.9625    | 0.9628 | 0.9626   | 0.9625   |
| 0.5562        | 3.78  | 825  | 0.2145          | 0.9577    | 0.9576 | 0.9572   | 0.9571   |
| 0.5562        | 3.9   | 850  | 0.2466          | 0.9603    | 0.9591 | 0.9584   | 0.9584   |
| 0.5562        | 4.01  | 875  | 0.2373          | 0.9647    | 0.9643 | 0.9639   | 0.9638   |
| 0.5562        | 4.13  | 900  | 0.2058          | 0.9653    | 0.9655 | 0.9653   | 0.9651   |
| 0.5562        | 4.24  | 925  | 0.2640          | 0.9641    | 0.9642 | 0.9640   | 0.9638   |
| 0.5562        | 4.36  | 950  | 0.3120          | 0.9496    | 0.9459 | 0.9447   | 0.9450   |
| 0.5562        | 4.47  | 975  | 0.1946          | 0.9587    | 0.9590 | 0.9585   | 0.9584   |
| 0.2573        | 4.59  | 1000 | 0.2094          | 0.9652    | 0.9655 | 0.9653   | 0.9651   |
| 0.2573        | 4.7   | 1025 | 0.2405          | 0.9594    | 0.9577 | 0.9572   | 0.9571   |
| 0.2573        | 4.82  | 1050 | 0.2205          | 0.9628    | 0.9627 | 0.9627   | 0.9625   |
| 0.2573        | 4.93  | 1075 | 0.2546          | 0.9604    | 0.9600 | 0.9601   | 0.9598   |


### Framework versions

- Transformers 4.36.0.dev0
- Pytorch 2.1.0+cu118
- Datasets 2.14.6
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "metrics": ["precision", "recall", "accuracy"], "base_model": "roberta-large", "model-index": [{"name": "roberta-large-lora-token-classification", "results": []}]}
transformers safetensors roberta text-classification generated_from_trainer base_model:roberta-large license:mit autotrain_compatible endpoints_compatible region:us