transformers pytorch tensorboard wav2vec2 automatic-speech-recognition generated_from_trainer license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: torgo_xlsr_finetune_F03_keep_all
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# torgo_xlsr_finetune_F03_keep_all

This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1972
- Wer: 0.0443

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 3.573         | 0.56  | 1000  | 3.3927          | 1.0    |
| 1.9872        | 1.11  | 2000  | 1.0729          | 0.7588 |
| 1.0338        | 1.67  | 3000  | 0.6272          | 0.4530 |
| 0.7363        | 2.22  | 4000  | 0.5054          | 0.3579 |
| 0.6181        | 2.78  | 5000  | 0.3583          | 0.2448 |
| 0.5327        | 3.33  | 6000  | 0.2902          | 0.1771 |
| 0.5046        | 3.89  | 7000  | 0.3202          | 0.1628 |
| 0.4227        | 4.44  | 8000  | 0.2460          | 0.1400 |
| 0.4405        | 5.0   | 9000  | 0.2750          | 0.1287 |
| 0.3927        | 5.55  | 10000 | 0.2235          | 0.0928 |
| 0.3496        | 6.11  | 11000 | 0.2621          | 0.1083 |
| 0.3294        | 6.66  | 12000 | 0.2056          | 0.0688 |
| 0.3201        | 7.22  | 13000 | 0.2155          | 0.0838 |
| 0.2973        | 7.77  | 14000 | 0.2909          | 0.0844 |
| 0.2758        | 8.33  | 15000 | 0.3096          | 0.0832 |
| 0.2688        | 8.88  | 16000 | 0.2420          | 0.0664 |
| 0.2537        | 9.44  | 17000 | 0.2438          | 0.0772 |
| 0.2642        | 9.99  | 18000 | 0.1836          | 0.0598 |
| 0.2611        | 10.55 | 19000 | 0.2562          | 0.0670 |
| 0.2019        | 11.1  | 20000 | 0.2084          | 0.0658 |
| 0.205         | 11.66 | 21000 | 0.1959          | 0.0628 |
| 0.2019        | 12.22 | 22000 | 0.2316          | 0.0700 |
| 0.1923        | 12.77 | 23000 | 0.2051          | 0.0664 |
| 0.1614        | 13.33 | 24000 | 0.2322          | 0.0724 |
| 0.1955        | 13.88 | 25000 | 0.1920          | 0.0551 |
| 0.1705        | 14.44 | 26000 | 0.2071          | 0.0521 |
| 0.1906        | 14.99 | 27000 | 0.2112          | 0.0533 |
| 0.1503        | 15.55 | 28000 | 0.1914          | 0.0491 |
| 0.1692        | 16.1  | 29000 | 0.1870          | 0.0557 |
| 0.149         | 16.66 | 30000 | 0.2172          | 0.0467 |
| 0.1413        | 17.21 | 31000 | 0.2313          | 0.0479 |
| 0.1466        | 17.77 | 32000 | 0.1989          | 0.0437 |
| 0.1533        | 18.32 | 33000 | 0.2078          | 0.0437 |
| 0.1176        | 18.88 | 34000 | 0.2034          | 0.0431 |
| 0.1189        | 19.43 | 35000 | 0.1929          | 0.0413 |
| 0.1056        | 19.99 | 36000 | 0.1972          | 0.0443 |


### Framework versions

- Transformers 4.26.1
- Pytorch 2.2.0
- Datasets 2.16.1
- Tokenizers 0.13.3
