transformers pytorch wav2vec2 automatic-speech-recognition generated_from_trainer license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: wav2vec2-xlsr-1b-mecita-portuguese-all-clean-04
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xlsr-1b-mecita-portuguese-all-clean-04

This model is a fine-tuned version of [jonatasgrosman/wav2vec2-xls-r-1b-portuguese](https://huggingface.co/jonatasgrosman/wav2vec2-xls-r-1b-portuguese) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1671
- Wer: 0.0812
- Cer: 0.0271

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|
| 21.5107       | 1.0   | 67   | 6.3931          | 0.9608 | 0.9566 |
| 6.0165        | 2.0   | 134  | 4.4922          | 0.9648 | 0.9784 |
| 4.0985        | 3.0   | 201  | 4.4454          | 0.9770 | 0.9791 |
| 4.0985        | 4.0   | 268  | 4.0621          | 0.9824 | 0.9753 |
| 3.471         | 5.0   | 335  | 4.0903          | 0.9797 | 0.9744 |
| 3.9535        | 6.0   | 402  | 3.1412          | 0.9770 | 0.9805 |
| 3.9535        | 7.0   | 469  | 3.7763          | 0.9743 | 0.9711 |
| 3.4618        | 8.0   | 536  | 4.0923          | 0.9642 | 0.9620 |
| 3.4186        | 9.0   | 603  | 4.2152          | 0.9753 | 0.9705 |
| 3.4186        | 10.0  | 670  | 2.6991          | 0.9811 | 0.9865 |
| 2.8947        | 11.0  | 737  | 0.4025          | 0.4569 | 0.0963 |
| 0.7229        | 12.0  | 804  | 0.2378          | 0.1302 | 0.0407 |
| 0.7229        | 13.0  | 871  | 0.2056          | 0.1106 | 0.0342 |
| 0.3162        | 14.0  | 938  | 0.1893          | 0.1099 | 0.0328 |
| 0.2601        | 15.0  | 1005 | 0.1732          | 0.1018 | 0.0316 |
| 0.2601        | 16.0  | 1072 | 0.1879          | 0.0933 | 0.0305 |
| 0.2292        | 17.0  | 1139 | 0.1710          | 0.0852 | 0.0283 |
| 0.2036        | 18.0  | 1206 | 0.1690          | 0.0937 | 0.0300 |
| 0.2036        | 19.0  | 1273 | 0.1691          | 0.0862 | 0.0282 |
| 0.174         | 20.0  | 1340 | 0.1737          | 0.0906 | 0.0288 |
| 0.1795        | 21.0  | 1407 | 0.1809          | 0.0889 | 0.0287 |
| 0.1795        | 22.0  | 1474 | 0.1689          | 0.0856 | 0.0283 |
| 0.1643        | 23.0  | 1541 | 0.1802          | 0.0845 | 0.0283 |
| 0.148         | 24.0  | 1608 | 0.1927          | 0.0944 | 0.0304 |
| 0.148         | 25.0  | 1675 | 0.1737          | 0.0839 | 0.0280 |
| 0.1482        | 26.0  | 1742 | 0.1749          | 0.0829 | 0.0275 |
| 0.1356        | 27.0  | 1809 | 0.1730          | 0.0862 | 0.0280 |
| 0.1356        | 28.0  | 1876 | 0.1936          | 0.0937 | 0.0303 |
| 0.1202        | 29.0  | 1943 | 0.1846          | 0.0896 | 0.0291 |
| 0.1175        | 30.0  | 2010 | 0.1853          | 0.0906 | 0.0290 |
| 0.1175        | 31.0  | 2077 | 0.1671          | 0.0812 | 0.0271 |
| 0.1086        | 32.0  | 2144 | 0.1884          | 0.0825 | 0.0269 |
| 0.1085        | 33.0  | 2211 | 0.2011          | 0.0869 | 0.0279 |
| 0.1085        | 34.0  | 2278 | 0.1895          | 0.0873 | 0.0282 |
| 0.0973        | 35.0  | 2345 | 0.1826          | 0.0879 | 0.0290 |
| 0.1219        | 36.0  | 2412 | 0.2073          | 0.0893 | 0.0285 |
| 0.1219        | 37.0  | 2479 | 0.2004          | 0.0852 | 0.0277 |
| 0.0912        | 38.0  | 2546 | 0.2037          | 0.0862 | 0.0284 |
| 0.0868        | 39.0  | 2613 | 0.1892          | 0.0852 | 0.0277 |
| 0.0868        | 40.0  | 2680 | 0.1854          | 0.0839 | 0.0276 |
| 0.09          | 41.0  | 2747 | 0.1875          | 0.0829 | 0.0276 |
| 0.0788        | 42.0  | 2814 | 0.1966          | 0.0801 | 0.0268 |
| 0.0788        | 43.0  | 2881 | 0.1965          | 0.0849 | 0.0275 |
| 0.0761        | 44.0  | 2948 | 0.1863          | 0.0815 | 0.0273 |
| 0.0759        | 45.0  | 3015 | 0.1922          | 0.0815 | 0.0267 |
| 0.0759        | 46.0  | 3082 | 0.2005          | 0.0805 | 0.0273 |
| 0.0736        | 47.0  | 3149 | 0.1978          | 0.0832 | 0.0276 |
| 0.0645        | 48.0  | 3216 | 0.2211          | 0.0839 | 0.0278 |
| 0.0645        | 49.0  | 3283 | 0.2108          | 0.0845 | 0.0277 |
| 0.0653        | 50.0  | 3350 | 0.1999          | 0.0822 | 0.0277 |
| 0.0673        | 51.0  | 3417 | 0.2033          | 0.0805 | 0.0276 |


### Framework versions

- Transformers 4.28.0
- Pytorch 2.2.1+cu121
- Datasets 2.17.0
- Tokenizers 0.13.3
