transformers pytorch wav2vec2 automatic-speech-recognition generated_from_trainer license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: wav2vec2-xlsr-1b-mecita-portuguese-all-07
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xlsr-1b-mecita-portuguese-all-07

This model is a fine-tuned version of [jonatasgrosman/wav2vec2-xls-r-1b-portuguese](https://huggingface.co/jonatasgrosman/wav2vec2-xls-r-1b-portuguese) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1401
- Wer: 0.0899
- Cer: 0.0279

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|
| 28.8922       | 1.0   | 86   | 4.7137          | 0.9821 | 0.9792 |
| 7.417         | 2.0   | 172  | 2.8760          | 1.0    | 1.0000 |
| 3.4374        | 3.0   | 258  | 0.2950          | 0.1869 | 0.0559 |
| 0.8615        | 4.0   | 344  | 0.2054          | 0.1199 | 0.0392 |
| 0.4009        | 5.0   | 430  | 0.1741          | 0.1143 | 0.0353 |
| 0.2953        | 6.0   | 516  | 0.1669          | 0.1042 | 0.0331 |
| 0.2706        | 7.0   | 602  | 0.1588          | 0.1008 | 0.0319 |
| 0.2706        | 8.0   | 688  | 0.1656          | 0.0989 | 0.0321 |
| 0.2436        | 9.0   | 774  | 0.1644          | 0.1001 | 0.0311 |
| 0.2338        | 10.0  | 860  | 0.1701          | 0.0957 | 0.0309 |
| 0.2105        | 11.0  | 946  | 0.1607          | 0.1003 | 0.0300 |
| 0.2153        | 12.0  | 1032 | 0.1592          | 0.0948 | 0.0298 |
| 0.1895        | 13.0  | 1118 | 0.1673          | 0.0984 | 0.0318 |
| 0.1883        | 14.0  | 1204 | 0.1563          | 0.0902 | 0.0298 |
| 0.1883        | 15.0  | 1290 | 0.1543          | 0.0955 | 0.0306 |
| 0.1776        | 16.0  | 1376 | 0.1480          | 0.0885 | 0.0291 |
| 0.1699        | 17.0  | 1462 | 0.1789          | 0.0921 | 0.0303 |
| 0.1699        | 18.0  | 1548 | 0.1687          | 0.0921 | 0.0302 |
| 0.1551        | 19.0  | 1634 | 0.1645          | 0.0875 | 0.0277 |
| 0.1441        | 20.0  | 1720 | 0.1619          | 0.0870 | 0.0281 |
| 0.1448        | 21.0  | 1806 | 0.1584          | 0.0861 | 0.0282 |
| 0.1448        | 22.0  | 1892 | 0.1666          | 0.0899 | 0.0291 |
| 0.1404        | 23.0  | 1978 | 0.1552          | 0.0892 | 0.0282 |
| 0.1457        | 24.0  | 2064 | 0.1671          | 0.0919 | 0.0288 |
| 0.1299        | 25.0  | 2150 | 0.1594          | 0.0887 | 0.0285 |
| 0.1385        | 26.0  | 2236 | 0.1572          | 0.0919 | 0.0288 |
| 0.1289        | 27.0  | 2322 | 0.1546          | 0.0880 | 0.0275 |
| 0.1268        | 28.0  | 2408 | 0.1574          | 0.0856 | 0.0272 |
| 0.1268        | 29.0  | 2494 | 0.1483          | 0.0868 | 0.0277 |
| 0.1137        | 30.0  | 2580 | 0.1535          | 0.0909 | 0.0288 |
| 0.1183        | 31.0  | 2666 | 0.1459          | 0.0865 | 0.0270 |
| 0.1061        | 32.0  | 2752 | 0.1428          | 0.0902 | 0.0277 |
| 0.0994        | 33.0  | 2838 | 0.1401          | 0.0899 | 0.0279 |
| 0.0967        | 34.0  | 2924 | 0.1403          | 0.0810 | 0.0258 |
| 0.1088        | 35.0  | 3010 | 0.1490          | 0.0868 | 0.0270 |
| 0.1088        | 36.0  | 3096 | 0.1487          | 0.0834 | 0.0256 |
| 0.0894        | 37.0  | 3182 | 0.1515          | 0.0882 | 0.0274 |
| 0.0953        | 38.0  | 3268 | 0.1557          | 0.0844 | 0.0264 |
| 0.0876        | 39.0  | 3354 | 0.1608          | 0.0829 | 0.0265 |
| 0.0883        | 40.0  | 3440 | 0.1595          | 0.0807 | 0.0261 |
| 0.0866        | 41.0  | 3526 | 0.1674          | 0.0829 | 0.0259 |
| 0.0872        | 42.0  | 3612 | 0.1603          | 0.0846 | 0.0273 |
| 0.0872        | 43.0  | 3698 | 0.1589          | 0.0875 | 0.0272 |
| 0.0933        | 44.0  | 3784 | 0.1513          | 0.0778 | 0.0262 |
| 0.0888        | 45.0  | 3870 | 0.1573          | 0.0851 | 0.0270 |
| 0.0814        | 46.0  | 3956 | 0.1699          | 0.0846 | 0.0270 |
| 0.0838        | 47.0  | 4042 | 0.1745          | 0.0803 | 0.0261 |
| 0.0751        | 48.0  | 4128 | 0.1599          | 0.0822 | 0.0260 |
| 0.074         | 49.0  | 4214 | 0.1709          | 0.0848 | 0.0262 |
| 0.0693        | 50.0  | 4300 | 0.1865          | 0.0856 | 0.0272 |
| 0.0693        | 51.0  | 4386 | 0.1702          | 0.0856 | 0.0268 |
| 0.0754        | 52.0  | 4472 | 0.1750          | 0.0861 | 0.0273 |
| 0.0723        | 53.0  | 4558 | 0.1785          | 0.0810 | 0.0261 |


### Framework versions

- Transformers 4.28.0
- Pytorch 2.2.1+cu121
- Datasets 2.17.0
- Tokenizers 0.13.3
