transformers safetensors bart text2text-generation generated_from_trainer base_model:checkpoint_global_step_200000 autotrain_compatible endpoints_compatible region:us
---
base_model: checkpoint_global_step_200000
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: fine-tuned-BioBART-20-epochs-test
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# fine-tuned-BioBART-20-epochs-test

This model is a fine-tuned version of [checkpoint_global_step_200000](https://huggingface.co/checkpoint_global_step_200000) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1997
- Rouge1: 0.0956
- Rouge2: 0.0145
- Rougel: 0.0591
- Rougelsum: 0.0593
- Gen Len: 217.0

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
| 0.7878        | 1.0   | 1201  | 0.2002          | 0.0888 | 0.0237 | 0.0689 | 0.0691    | 224.36  |
| 0.2064        | 2.0   | 2402  | 0.1817          | 0.02   | 0.0    | 0.02   | 0.02      | 8.0     |
| 0.1708        | 3.0   | 3603  | 0.1638          | 0.03   | 0.0    | 0.03   | 0.03      | 5.0     |
| 0.136         | 4.0   | 4804  | 0.1576          | 0.0228 | 0.0036 | 0.0232 | 0.023     | 10.0    |
| 0.1346        | 5.0   | 6005  | 0.1559          | 0.0631 | 0.018  | 0.0592 | 0.0591    | 11.0    |
| 0.097         | 6.0   | 7206  | 0.1573          | 0.0928 | 0.0177 | 0.0784 | 0.079     | 20.0    |
| 0.086         | 7.0   | 8407  | 0.1607          | 0.0638 | 0.0086 | 0.0522 | 0.0523    | 21.0    |
| 0.0638        | 8.0   | 9608  | 0.1649          | 0.0228 | 0.0036 | 0.0232 | 0.023     | 10.0    |
| 0.0425        | 9.0   | 10809 | 0.1690          | 0.064  | 0.0198 | 0.0578 | 0.0579    | 20.0    |
| 0.0359        | 10.0  | 12010 | 0.1726          | 0.1024 | 0.0157 | 0.0817 | 0.0817    | 49.0    |
| 0.0262        | 11.0  | 13211 | 0.1771          | 0.0868 | 0.0198 | 0.0787 | 0.0792    | 20.0    |
| 0.0204        | 12.0  | 14412 | 0.1819          | 0.0977 | 0.0104 | 0.0748 | 0.075     | 43.0    |
| 0.0156        | 13.0  | 15613 | 0.1852          | 0.066  | 0.0094 | 0.0509 | 0.051     | 43.0    |
| 0.0131        | 14.0  | 16814 | 0.1885          | 0.1068 | 0.018  | 0.0726 | 0.0725    | 135.0   |
| 0.0105        | 15.0  | 18015 | 0.1915          | 0.0967 | 0.0248 | 0.0784 | 0.0787    | 30.0    |
| 0.009         | 16.0  | 19216 | 0.1950          | 0.104  | 0.0221 | 0.0791 | 0.079     | 73.0    |
| 0.0081        | 17.0  | 20417 | 0.1962          | 0.0967 | 0.0248 | 0.0784 | 0.0787    | 30.0    |
| 0.0077        | 18.0  | 21618 | 0.1978          | 0.0903 | 0.0084 | 0.0567 | 0.0567    | 174.0   |
| 0.0068        | 19.0  | 22819 | 0.1991          | 0.0896 | 0.0107 | 0.055  | 0.055     | 174.0   |
| 0.0064        | 20.0  | 24020 | 0.1997          | 0.0956 | 0.0145 | 0.0591 | 0.0593    | 217.0   |


### Framework versions

- Transformers 4.36.2
- Pytorch 1.12.1+cu113
- Datasets 2.16.1
- Tokenizers 0.15.0
