
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bart-large-cnn-finetuned-Kaggle-Science-LLM

This model is a fine-tuned version of [facebook/bart-large-cnn](https://huggingface.co/facebook/bart-large-cnn) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 6.4896
- Rouge1: 29.4886
- Rouge2: 10.2696
- Rougel: 22.611
- Rougelsum: 23.6936
- Gen Len: 70.1

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| No log        | 1.0   | 90   | 2.9814          | 32.5407 | 12.8638 | 25.9593 | 28.0874   | 66.05   |
| No log        | 2.0   | 180  | 3.1081          | 33.6875 | 13.0896 | 25.2244 | 26.9945   | 68.25   |
| No log        | 3.0   | 270  | 3.4845          | 33.889  | 12.8396 | 26.2138 | 28.2817   | 70.55   |
| No log        | 4.0   | 360  | 3.8911          | 31.8492 | 12.0458 | 23.4026 | 25.8547   | 66.25   |
| No log        | 5.0   | 450  | 4.3530          | 31.2083 | 11.0996 | 23.9196 | 26.1564   | 72.25   |
| 1.4121        | 6.0   | 540  | 4.4582          | 29.7758 | 11.1798 | 22.9812 | 24.9141   | 72.2    |
| 1.4121        | 7.0   | 630  | 4.5299          | 30.3925 | 11.41   | 23.9357 | 25.4386   | 74.15   |
| 1.4121        | 8.0   | 720  | 5.0756          | 30.1282 | 10.1879 | 22.5263 | 24.3294   | 71.05   |
| 1.4121        | 9.0   | 810  | 5.2213          | 29.1958 | 11.9758 | 22.9344 | 25.3243   | 70.95   |
| 1.4121        | 10.0  | 900  | 5.0236          | 32.2902 | 12.9557 | 24.9154 | 26.9866   | 71.85   |
| 1.4121        | 11.0  | 990  | 5.2231          | 29.9105 | 11.4629 | 22.5421 | 24.7261   | 73.15   |
| 0.1808        | 12.0  | 1080 | 5.4899          | 30.6426 | 10.8586 | 23.0649 | 25.4052   | 69.35   |
| 0.1808        | 13.0  | 1170 | 5.5205          | 31.4239 | 12.4297 | 24.2742 | 25.8058   | 64.9    |
| 0.1808        | 14.0  | 1260 | 5.4710          | 31.3377 | 11.5225 | 23.4415 | 25.9487   | 68.3    |
| 0.1808        | 15.0  | 1350 | 5.3894          | 30.5681 | 11.3301 | 22.5992 | 25.0445   | 67.1    |
| 0.1808        | 16.0  | 1440 | 5.7293          | 30.7485 | 10.2947 | 23.2461 | 25.1156   | 67.8    |
| 0.0634        | 17.0  | 1530 | 5.8342          | 27.8846 | 9.4002  | 20.5223 | 22.8928   | 73.7    |
| 0.0634        | 18.0  | 1620 | 5.7280          | 31.3703 | 12.7091 | 24.947  | 27.6756   | 68.7    |
| 0.0634        | 19.0  | 1710 | 6.0204          | 29.311  | 10.8717 | 22.2206 | 23.6151   | 66.05   |
| 0.0634        | 20.0  | 1800 | 5.8662          | 30.3449 | 10.9645 | 22.7105 | 25.3131   | 75.6    |
| 0.0634        | 21.0  | 1890 | 6.0514          | 29.4108 | 10.9479 | 22.1319 | 23.8446   | 70.6    |
| 0.0634        | 22.0  | 1980 | 5.9087          | 30.1637 | 10.7748 | 21.7979 | 23.8345   | 71.6    |
| 0.0281        | 23.0  | 2070 | 6.1406          | 30.3179 | 11.0906 | 23.2057 | 24.9556   | 69.65   |
| 0.0281        | 24.0  | 2160 | 6.0541          | 29.7931 | 11.492  | 22.7251 | 24.4958   | 68.9    |
| 0.0281        | 25.0  | 2250 | 6.4349          | 29.6705 | 11.3079 | 22.1845 | 24.0782   | 68.2    |
| 0.0281        | 26.0  | 2340 | 6.2949          | 30.3573 | 9.7319  | 22.8766 | 25.5102   | 68.65   |
| 0.0281        | 27.0  | 2430 | 6.3606          | 30.2358 | 10.7457 | 22.9097 | 24.7486   | 69.8    |
| 0.0167        | 28.0  | 2520 | 6.2235          | 29.131  | 11.0196 | 23.0364 | 24.7254   | 69.0    |
| 0.0167        | 29.0  | 2610 | 6.2203          | 30.0767 | 10.4042 | 23.0845 | 24.5571   | 71.15   |
| 0.0167        | 30.0  | 2700 | 6.3899          | 29.524  | 11.0226 | 22.7426 | 24.7137   | 71.45   |
| 0.0167        | 31.0  | 2790 | 6.4216          | 29.9921 | 11.1592 | 22.7774 | 25.4653   | 70.35   |
| 0.0167        | 32.0  | 2880 | 6.4758          | 29.4138 | 10.1446 | 22.5501 | 24.4203   | 68.0    |
| 0.0167        | 33.0  | 2970 | 6.4529          | 30.7129 | 9.9512  | 23.3078 | 25.1444   | 70.1    |
| 0.0086        | 34.0  | 3060 | 6.3910          | 32.0673 | 11.8157 | 24.4371 | 26.4378   | 67.4    |
| 0.0086        | 35.0  | 3150 | 6.4725          | 31.0417 | 11.8642 | 23.9718 | 25.9358   | 65.5    |
| 0.0086        | 36.0  | 3240 | 6.5413          | 31.2471 | 11.9972 | 24.537  | 25.6679   | 66.6    |
| 0.0086        | 37.0  | 3330 | 6.6040          | 30.6614 | 11.4845 | 23.6335 | 26.3165   | 72.15   |
| 0.0086        | 38.0  | 3420 | 6.4808          | 30.1209 | 10.4855 | 22.7931 | 24.9675   | 74.75   |
| 0.0053        | 39.0  | 3510 | 6.4196          | 29.9709 | 11.1147 | 23.3882 | 25.1429   | 73.3    |
| 0.0053        | 40.0  | 3600 | 6.4798          | 32.6666 | 11.6476 | 24.0167 | 25.8167   | 67.7    |
| 0.0053        | 41.0  | 3690 | 6.4364          | 31.7081 | 11.4081 | 23.8924 | 25.3477   | 67.35   |
| 0.0053        | 42.0  | 3780 | 6.4463          | 31.371  | 11.3334 | 23.8642 | 25.5894   | 67.85   |
| 0.0053        | 43.0  | 3870 | 6.4507          | 29.6148 | 11.0601 | 22.5613 | 24.2758   | 70.95   |
| 0.0053        | 44.0  | 3960 | 6.5410          | 30.9704 | 10.054  | 22.8276 | 25.1106   | 66.25   |
| 0.0036        | 45.0  | 4050 | 6.4484          | 30.6993 | 10.2855 | 22.8241 | 25.1591   | 69.3    |
| 0.0036        | 46.0  | 4140 | 6.4579          | 29.6269 | 10.353  | 21.9677 | 23.4709   | 71.15   |
| 0.0036        | 47.0  | 4230 | 6.4931          | 29.8756 | 10.4957 | 23.039  | 24.2656   | 69.0    |
| 0.0036        | 48.0  | 4320 | 6.4831          | 29.6629 | 10.0869 | 22.8167 | 24.0125   | 70.35   |
| 0.0036        | 49.0  | 4410 | 6.4871          | 29.908  | 10.3116 | 22.9103 | 24.0365   | 71.9    |
| 0.0023        | 50.0  | 4500 | 6.4896          | 29.4886 | 10.2696 | 22.611  | 23.6936   | 70.1    |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.13.3

{"license": "mit", "tags": ["generated_from_trainer"], "metrics": ["rouge"], "base_model": "facebook/bart-large-cnn", "model-index": [{"name": "bart-large-cnn-finetuned-Kaggle-Science-LLM", "results": []}]}
transformers pytorch bart text2text-generation generated_from_trainer base_model:facebook/bart-large-cnn license:mit autotrain_compatible endpoints_compatible region:us