transformers pytorch tensorboard wav2vec2 automatic-speech-recognition generated_from_trainer license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: torgo_xlsr_finetune_M04__keep_all
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# torgo_xlsr_finetune_M04__keep_all

This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the torgo dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4844
- Wer: 0.2303

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 3.5764        | 0.55  | 1000  | 3.3931          | 1.0    |
| 2.2058        | 1.1   | 2000  | 1.8051          | 0.8589 |
| 1.0531        | 1.66  | 3000  | 1.5038          | 0.6508 |
| 0.7783        | 2.21  | 4000  | 1.2594          | 0.5208 |
| 0.6084        | 2.76  | 5000  | 1.3131          | 0.4586 |
| 0.5241        | 3.31  | 6000  | 1.3666          | 0.4339 |
| 0.4586        | 3.87  | 7000  | 1.3358          | 0.3866 |
| 0.4149        | 4.42  | 8000  | 1.2625          | 0.3332 |
| 0.3796        | 4.97  | 9000  | 1.5808          | 0.3629 |
| 0.3685        | 5.52  | 10000 | 1.2197          | 0.3298 |
| 0.3322        | 6.07  | 11000 | 1.6204          | 0.3473 |
| 0.3133        | 6.63  | 12000 | 1.6558          | 0.3446 |
| 0.2833        | 7.18  | 13000 | 1.5270          | 0.3100 |
| 0.2941        | 7.73  | 14000 | 1.4321          | 0.3134 |
| 0.2709        | 8.28  | 15000 | 1.3682          | 0.3092 |
| 0.2362        | 8.83  | 16000 | 1.2184          | 0.2787 |
| 0.2205        | 9.39  | 17000 | 1.4273          | 0.2863 |
| 0.2515        | 9.94  | 18000 | 1.3085          | 0.2665 |
| 0.2185        | 10.49 | 19000 | 1.5292          | 0.2852 |
| 0.2197        | 11.04 | 20000 | 1.4625          | 0.2817 |
| 0.2122        | 11.6  | 21000 | 1.4086          | 0.2634 |
| 0.1869        | 12.15 | 22000 | 1.6290          | 0.2791 |
| 0.1839        | 12.7  | 23000 | 1.4520          | 0.2722 |
| 0.1946        | 13.25 | 24000 | 1.5211          | 0.2653 |
| 0.1871        | 13.8  | 25000 | 1.3136          | 0.2390 |
| 0.1831        | 14.36 | 26000 | 1.4022          | 0.2581 |
| 0.1644        | 14.91 | 27000 | 1.5609          | 0.2673 |
| 0.1499        | 15.46 | 28000 | 1.3431          | 0.2429 |
| 0.1566        | 16.01 | 29000 | 1.5110          | 0.2566 |
| 0.1533        | 16.57 | 30000 | 1.4567          | 0.2345 |
| 0.1446        | 17.12 | 31000 | 1.5160          | 0.2478 |
| 0.1451        | 17.67 | 32000 | 1.4081          | 0.2379 |
| 0.1269        | 18.22 | 33000 | 1.5296          | 0.2379 |
| 0.1438        | 18.77 | 34000 | 1.5765          | 0.2406 |
| 0.1112        | 19.33 | 35000 | 1.5061          | 0.2337 |
| 0.1215        | 19.88 | 36000 | 1.4844          | 0.2303 |


### Framework versions

- Transformers 4.26.1
- Pytorch 2.2.0
- Datasets 2.16.1
- Tokenizers 0.13.3
