transformers pytorch wav2vec2 automatic-speech-recognition generated_from_trainer license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: wav2vec2-xlsr-1b-mecita-portuguese-all-04
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xlsr-1b-mecita-portuguese-all-04

This model is a fine-tuned version of [jonatasgrosman/wav2vec2-xls-r-1b-portuguese](https://huggingface.co/jonatasgrosman/wav2vec2-xls-r-1b-portuguese) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1759
- Wer: 0.0864
- Cer: 0.0277

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|
| 28.0495       | 1.0   | 86   | 0.5256          | 0.3275 | 0.0891 |
| 3.6122        | 2.0   | 172  | 0.2510          | 0.1363 | 0.0398 |
| 0.494         | 3.0   | 258  | 0.2255          | 0.1159 | 0.0359 |
| 0.3372        | 4.0   | 344  | 0.2135          | 0.1083 | 0.0328 |
| 0.2992        | 5.0   | 430  | 0.2132          | 0.0982 | 0.0304 |
| 0.2613        | 6.0   | 516  | 0.2023          | 0.0901 | 0.0276 |
| 0.2441        | 7.0   | 602  | 0.2041          | 0.0972 | 0.0303 |
| 0.2441        | 8.0   | 688  | 0.2039          | 0.0928 | 0.0292 |
| 0.2251        | 9.0   | 774  | 0.1945          | 0.0866 | 0.0268 |
| 0.2462        | 10.0  | 860  | 0.2016          | 0.0920 | 0.0290 |
| 0.1848        | 11.0  | 946  | 0.1967          | 0.0837 | 0.0265 |
| 0.1681        | 12.0  | 1032 | 0.1908          | 0.0844 | 0.0262 |
| 0.1559        | 13.0  | 1118 | 0.1954          | 0.0915 | 0.0275 |
| 0.1761        | 14.0  | 1204 | 0.1961          | 0.0849 | 0.0271 |
| 0.1761        | 15.0  | 1290 | 0.1868          | 0.0810 | 0.0259 |
| 0.1582        | 16.0  | 1376 | 0.1759          | 0.0864 | 0.0277 |
| 0.1437        | 17.0  | 1462 | 0.1827          | 0.0859 | 0.0270 |
| 0.1471        | 18.0  | 1548 | 0.1860          | 0.0817 | 0.0262 |
| 0.1283        | 19.0  | 1634 | 0.2068          | 0.0822 | 0.0267 |
| 0.1295        | 20.0  | 1720 | 0.2139          | 0.0861 | 0.0272 |
| 0.1298        | 21.0  | 1806 | 0.1837          | 0.0819 | 0.0252 |
| 0.1298        | 22.0  | 1892 | 0.1934          | 0.0864 | 0.0263 |
| 0.1054        | 23.0  | 1978 | 0.1861          | 0.0812 | 0.0249 |
| 0.1087        | 24.0  | 2064 | 0.1913          | 0.0824 | 0.0257 |
| 0.1136        | 25.0  | 2150 | 0.2102          | 0.0859 | 0.0264 |
| 0.1096        | 26.0  | 2236 | 0.2063          | 0.0797 | 0.0248 |
| 0.1083        | 27.0  | 2322 | 0.2029          | 0.0827 | 0.0259 |
| 0.1068        | 28.0  | 2408 | 0.1992          | 0.0812 | 0.0253 |
| 0.1068        | 29.0  | 2494 | 0.2134          | 0.0854 | 0.0271 |
| 0.1014        | 30.0  | 2580 | 0.1993          | 0.0859 | 0.0277 |
| 0.1219        | 31.0  | 2666 | 0.1927          | 0.0864 | 0.0265 |
| 0.0973        | 32.0  | 2752 | 0.2098          | 0.0827 | 0.0263 |
| 0.0934        | 33.0  | 2838 | 0.2106          | 0.0910 | 0.0319 |
| 0.0953        | 34.0  | 2924 | 0.2009          | 0.0842 | 0.0259 |
| 0.0984        | 35.0  | 3010 | 0.2030          | 0.0827 | 0.0252 |
| 0.0984        | 36.0  | 3096 | 0.2041          | 0.0839 | 0.0264 |


### Framework versions

- Transformers 4.28.0
- Pytorch 2.2.1+cu121
- Datasets 2.17.0
- Tokenizers 0.13.3
