transformers pytorch wav2vec2 automatic-speech-recognition generated_from_trainer license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: wav2vec2-xlsr-1b-mecita-portuguese-all-09
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xlsr-1b-mecita-portuguese-all-09

This model is a fine-tuned version of [jonatasgrosman/wav2vec2-xls-r-1b-portuguese](https://huggingface.co/jonatasgrosman/wav2vec2-xls-r-1b-portuguese) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1752
- Wer: 0.0943
- Cer: 0.0335

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|
| 26.8567       | 1.0   | 86   | 7.3154          | 0.9613 | 0.9363 |
| 8.6765        | 2.0   | 172  | 8.0168          | 0.9761 | 0.9769 |
| 5.1948        | 3.0   | 258  | 4.2978          | 0.9773 | 0.9779 |
| 3.8767        | 4.0   | 344  | 2.9105          | 1.0    | 1.0    |
| 2.9394        | 5.0   | 430  | 2.5112          | 1.0    | 0.9594 |
| 2.1143        | 6.0   | 516  | 0.4403          | 0.3456 | 0.0921 |
| 0.5533        | 7.0   | 602  | 0.2455          | 0.1310 | 0.0445 |
| 0.5533        | 8.0   | 688  | 0.2126          | 0.1084 | 0.0382 |
| 0.3176        | 9.0   | 774  | 0.2078          | 0.1069 | 0.0378 |
| 0.2628        | 10.0  | 860  | 0.2695          | 0.1091 | 0.0376 |
| 0.2098        | 11.0  | 946  | 0.1957          | 0.1024 | 0.0357 |
| 0.2483        | 12.0  | 1032 | 0.1927          | 0.0971 | 0.0351 |
| 0.2006        | 13.0  | 1118 | 0.1803          | 0.0959 | 0.0334 |
| 0.1594        | 14.0  | 1204 | 0.1849          | 0.0947 | 0.0326 |
| 0.1594        | 15.0  | 1290 | 0.1780          | 0.0914 | 0.0330 |
| 0.1753        | 16.0  | 1376 | 0.1752          | 0.0943 | 0.0335 |
| 0.1465        | 17.0  | 1462 | 0.1861          | 0.0967 | 0.0341 |
| 0.1502        | 18.0  | 1548 | 0.1810          | 0.0945 | 0.0337 |
| 0.1341        | 19.0  | 1634 | 0.1807          | 0.0888 | 0.0329 |
| 0.1481        | 20.0  | 1720 | 0.1834          | 0.0950 | 0.0335 |
| 0.1242        | 21.0  | 1806 | 0.1807          | 0.0902 | 0.0329 |
| 0.1242        | 22.0  | 1892 | 0.1830          | 0.0931 | 0.0336 |
| 0.1317        | 23.0  | 1978 | 0.1805          | 0.0890 | 0.0326 |
| 0.1143        | 24.0  | 2064 | 0.1798          | 0.0907 | 0.0337 |
| 0.1111        | 25.0  | 2150 | 0.1866          | 0.0924 | 0.0334 |
| 0.1264        | 26.0  | 2236 | 0.1829          | 0.0952 | 0.0341 |
| 0.0985        | 27.0  | 2322 | 0.1952          | 0.0947 | 0.0343 |
| 0.1106        | 28.0  | 2408 | 0.1893          | 0.0893 | 0.0312 |
| 0.1106        | 29.0  | 2494 | 0.1861          | 0.0895 | 0.0321 |
| 0.0888        | 30.0  | 2580 | 0.1886          | 0.0890 | 0.0318 |
| 0.091         | 31.0  | 2666 | 0.1876          | 0.0914 | 0.0329 |
| 0.0892        | 32.0  | 2752 | 0.1905          | 0.0907 | 0.0327 |
| 0.1009        | 33.0  | 2838 | 0.1832          | 0.0900 | 0.0320 |
| 0.0862        | 34.0  | 2924 | 0.2061          | 0.0916 | 0.0335 |
| 0.0915        | 35.0  | 3010 | 0.1915          | 0.0885 | 0.0313 |
| 0.0915        | 36.0  | 3096 | 0.1884          | 0.0881 | 0.0330 |


### Framework versions

- Transformers 4.28.0
- Pytorch 2.2.1+cu121
- Datasets 2.17.0
- Tokenizers 0.13.3
