
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# apac_5sents_XLS-R_2_e-4_unfreeze

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9903
- Wer: 0.2031

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- training_steps: 20000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step  | Validation Loss | Wer    |
|:-------------:|:-------:|:-----:|:---------------:|:------:|
| 189.2127      | 11.11   | 200   | 73.2720         | 1.0    |
| 54.2997       | 22.22   | 400   | 34.4132         | 1.0    |
| 20.0866       | 33.32   | 600   | 8.5765          | 1.0    |
| 6.0035        | 44.43   | 800   | 4.9753          | 1.0    |
| 4.7213        | 55.54   | 1000  | 4.3482          | 1.0    |
| 3.4429        | 66.65   | 1200  | 3.1173          | 1.0    |
| 1.5676        | 77.76   | 1400  | 1.7944          | 0.9353 |
| 0.7909        | 88.86   | 1600  | 1.6430          | 0.9308 |
| 0.6096        | 99.97   | 1800  | 1.5703          | 0.9330 |
| 0.5121        | 111.11  | 2000  | 1.5256          | 0.9129 |
| 0.4618        | 122.22  | 2200  | 1.6791          | 0.8638 |
| 0.3701        | 133.32  | 2400  | 1.5437          | 0.6920 |
| 0.2753        | 144.43  | 2600  | 1.3421          | 0.8304 |
| 0.1842        | 155.54  | 2800  | 1.1704          | 0.7455 |
| 0.0994        | 166.65  | 3000  | 1.0909          | 0.4710 |
| 0.0719        | 177.76  | 3200  | 1.2866          | 0.5335 |
| 0.0457        | 188.86  | 3400  | 1.1482          | 0.5580 |
| 0.0373        | 199.97  | 3600  | 0.9844          | 0.6183 |
| 0.0384        | 211.11  | 3800  | 1.1314          | 0.6451 |
| 0.0329        | 222.22  | 4000  | 1.3817          | 0.6406 |
| 0.03          | 233.32  | 4200  | 1.0517          | 0.6049 |
| 0.026         | 244.43  | 4400  | 1.0459          | 0.5022 |
| 0.0339        | 255.54  | 4600  | 1.2741          | 0.4732 |
| 0.0231        | 266.65  | 4800  | 1.5190          | 0.5424 |
| 0.0244        | 277.76  | 5000  | 1.3602          | 0.6451 |
| 0.017         | 288.86  | 5200  | 1.9874          | 0.6473 |
| 0.0204        | 299.97  | 5400  | 1.4477          | 0.6540 |
| 0.0221        | 311.11  | 5600  | 1.1082          | 0.5960 |
| 0.019         | 322.22  | 5800  | 1.5668          | 0.5737 |
| 0.0148        | 333.32  | 6000  | 0.9671          | 0.5871 |
| 0.0171        | 344.43  | 6200  | 1.0411          | 0.6406 |
| 0.0135        | 355.54  | 6400  | 1.0979          | 0.5938 |
| 0.0236        | 366.65  | 6600  | 1.2793          | 0.5670 |
| 0.0178        | 377.76  | 6800  | 1.6185          | 0.5982 |
| 0.0145        | 388.86  | 7000  | 1.1957          | 0.5491 |
| 0.015         | 399.97  | 7200  | 1.6764          | 0.6272 |
| 0.0122        | 411.11  | 7400  | 1.0150          | 0.6183 |
| 0.0112        | 422.22  | 7600  | 1.0519          | 0.6429 |
| 0.0083        | 433.32  | 7800  | 1.2191          | 0.5714 |
| 0.0086        | 444.43  | 8000  | 1.3643          | 0.6116 |
| 0.0149        | 455.54  | 8200  | 0.8175          | 0.6116 |
| 0.0113        | 466.65  | 8400  | 1.3835          | 0.6161 |
| 0.0085        | 477.76  | 8600  | 1.1749          | 0.5781 |
| 0.0074        | 488.86  | 8800  | 1.4239          | 0.4754 |
| 0.0064        | 499.97  | 9000  | 1.2462          | 0.5871 |
| 0.0089        | 511.11  | 9200  | 0.9149          | 0.4754 |
| 0.0084        | 522.22  | 9400  | 0.9334          | 0.5759 |
| 0.0056        | 533.32  | 9600  | 1.2457          | 0.6585 |
| 0.007         | 544.43  | 9800  | 1.1244          | 0.6518 |
| 0.0106        | 555.54  | 10000 | 1.2021          | 0.6272 |
| 0.0082        | 566.65  | 10200 | 1.0176          | 0.6094 |
| 0.0094        | 577.76  | 10400 | 1.0082          | 0.6161 |
| 0.0085        | 588.86  | 10600 | 1.0985          | 0.5580 |
| 0.0087        | 599.97  | 10800 | 0.8951          | 0.5067 |
| 0.0088        | 611.11  | 11000 | 1.0105          | 0.6094 |
| 0.0073        | 622.22  | 11200 | 1.1884          | 0.6071 |
| 0.0058        | 633.32  | 11400 | 1.0784          | 0.5804 |
| 0.008         | 644.43  | 11600 | 0.9484          | 0.6228 |
| 0.0044        | 655.54  | 11800 | 0.9366          | 0.6295 |
| 0.0049        | 666.65  | 12000 | 0.9724          | 0.6562 |
| 0.0055        | 677.76  | 12200 | 1.0735          | 0.6272 |
| 0.0057        | 688.86  | 12400 | 1.4276          | 0.5670 |
| 0.0069        | 699.97  | 12600 | 0.8726          | 0.6004 |
| 0.0042        | 711.11  | 12800 | 1.2411          | 0.5402 |
| 0.0031        | 722.22  | 13000 | 0.9987          | 0.5603 |
| 0.0033        | 733.32  | 13200 | 1.0106          | 0.5781 |
| 0.0046        | 744.43  | 13400 | 1.0548          | 0.4241 |
| 0.0042        | 755.54  | 13600 | 0.9983          | 0.2009 |
| 0.0025        | 766.65  | 13800 | 1.0933          | 0.2879 |
| 0.0067        | 777.76  | 14000 | 1.0665          | 0.3482 |
| 0.0061        | 788.86  | 14200 | 1.3937          | 0.4085 |
| 0.0058        | 799.97  | 14400 | 1.4818          | 0.3036 |
| 0.003         | 811.11  | 14600 | 1.3054          | 0.2723 |
| 0.0036        | 822.22  | 14800 | 1.1874          | 0.4062 |
| 0.0032        | 833.32  | 15000 | 1.4134          | 0.3571 |
| 0.0024        | 844.43  | 15200 | 1.4457          | 0.3661 |
| 0.0019        | 855.54  | 15400 | 1.2084          | 0.4821 |
| 0.0014        | 866.65  | 15600 | 1.2791          | 0.3058 |
| 0.0018        | 877.76  | 15800 | 1.1289          | 0.3616 |
| 0.0013        | 888.86  | 16000 | 1.3066          | 0.2567 |
| 0.001         | 899.97  | 16200 | 1.3964          | 0.2991 |
| 0.0016        | 911.11  | 16400 | 1.3610          | 0.2522 |
| 0.001         | 922.22  | 16600 | 1.4205          | 0.2812 |
| 0.0015        | 933.32  | 16800 | 1.2090          | 0.2946 |
| 0.0009        | 944.43  | 17000 | 1.2032          | 0.2344 |
| 0.0018        | 955.54  | 17200 | 1.2981          | 0.3013 |
| 0.0014        | 966.65  | 17400 | 1.2368          | 0.3549 |
| 0.0016        | 977.76  | 17600 | 1.3524          | 0.3683 |
| 0.0009        | 988.86  | 17800 | 1.4152          | 0.3460 |
| 0.0007        | 999.97  | 18000 | 1.3977          | 0.3125 |
| 0.0015        | 1011.11 | 18200 | 1.3225          | 0.2857 |
| 0.0009        | 1022.22 | 18400 | 1.2686          | 0.2812 |
| 0.0009        | 1033.32 | 18600 | 1.2383          | 0.3036 |
| 0.0011        | 1044.43 | 18800 | 1.2066          | 0.2902 |
| 0.0009        | 1055.54 | 19000 | 1.2048          | 0.2812 |
| 0.0007        | 1066.65 | 19200 | 1.2001          | 0.2812 |
| 0.0009        | 1077.76 | 19400 | 1.2631          | 0.3013 |
| 0.0005        | 1088.86 | 19600 | 1.2438          | 0.3036 |
| 0.0005        | 1099.97 | 19800 | 1.2146          | 0.3036 |
| 0.0005        | 1111.11 | 20000 | 1.2095          | 0.2991 |


### Framework versions

- Transformers 4.26.1
- Pytorch 2.0.1+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3

{"license": "apache-2.0", "tags": ["generated_from_trainer"], "metrics": ["wer"], "model-index": [{"name": "apac_5sents_XLS-R_2_e-4_unfreeze", "results": []}]}
transformers pytorch wav2vec2 automatic-speech-recognition generated_from_trainer license:apache-2.0 endpoints_compatible region:us