transformers pytorch tensorboard wav2vec2 automatic-speech-recognition generated_from_trainer license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: torgo_xlsr_finetune_M05_keep_all
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# torgo_xlsr_finetune_M05_keep_all

This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6032
- Wer: 0.2230

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 3.6054        | 0.55  | 1000  | 3.4415          | 1.0    |
| 3.1884        | 1.1   | 2000  | 2.7133          | 1.0    |
| 1.25          | 1.65  | 3000  | 1.5085          | 0.7190 |
| 0.8201        | 2.2   | 4000  | 1.5053          | 0.5696 |
| 0.6758        | 2.75  | 5000  | 1.3675          | 0.5063 |
| 0.542         | 3.3   | 6000  | 1.4764          | 0.4445 |
| 0.5344        | 3.85  | 7000  | 1.4226          | 0.4293 |
| 0.4464        | 4.4   | 8000  | 1.5430          | 0.3816 |
| 0.4191        | 4.95  | 9000  | 1.5657          | 0.3835 |
| 0.402         | 5.5   | 10000 | 1.5064          | 0.3610 |
| 0.332         | 6.05  | 11000 | 1.8348          | 0.3519 |
| 0.335         | 6.6   | 12000 | 1.3122          | 0.3408 |
| 0.3045        | 7.15  | 13000 | 1.4435          | 0.3126 |
| 0.2982        | 7.7   | 14000 | 1.4392          | 0.3347 |
| 0.2706        | 8.25  | 15000 | 1.7165          | 0.3481 |
| 0.2618        | 8.8   | 16000 | 1.3418          | 0.2856 |
| 0.2422        | 9.35  | 17000 | 1.4707          | 0.3122 |
| 0.2396        | 9.9   | 18000 | 1.6035          | 0.3035 |
| 0.248         | 10.45 | 19000 | 1.3132          | 0.2375 |
| 0.2272        | 10.99 | 20000 | 1.5060          | 0.2722 |
| 0.194         | 11.54 | 21000 | 1.4504          | 0.2650 |
| 0.2096        | 12.09 | 22000 | 1.7180          | 0.2871 |
| 0.2137        | 12.64 | 23000 | 1.6053          | 0.2676 |
| 0.1929        | 13.19 | 24000 | 1.5596          | 0.2577 |
| 0.1811        | 13.74 | 25000 | 1.6539          | 0.2600 |
| 0.1749        | 14.29 | 26000 | 1.7676          | 0.2570 |
| 0.1697        | 14.84 | 27000 | 1.7116          | 0.2577 |
| 0.1659        | 15.39 | 28000 | 1.7506          | 0.2551 |
| 0.1625        | 15.94 | 29000 | 1.6863          | 0.2566 |
| 0.1603        | 16.49 | 30000 | 1.6332          | 0.2406 |
| 0.1497        | 17.04 | 31000 | 1.4480          | 0.2307 |
| 0.1503        | 17.59 | 32000 | 1.5508          | 0.2238 |
| 0.1349        | 18.14 | 33000 | 1.6107          | 0.2280 |
| 0.1342        | 18.69 | 34000 | 1.5869          | 0.2280 |
| 0.1309        | 19.24 | 35000 | 1.5790          | 0.2268 |
| 0.1319        | 19.79 | 36000 | 1.6032          | 0.2230 |


### Framework versions

- Transformers 4.26.1
- Pytorch 2.2.0
- Datasets 2.16.1
- Tokenizers 0.13.3
