transformers safetensors wav2vec2 generated_from_trainer base_model:nguyenvulebinh/wav2vec2-base-vietnamese-250h license:cc-by-nc-4.0 endpoints_compatible region:us
---
license: cc-by-nc-4.0
base_model: nguyenvulebinh/wav2vec2-base-vietnamese-250h
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: w2v2_ablation_focal_ctc_a0.75_g0.5-best_on-ling_head-tp0.025_tl10_fp0.001_fl16
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# w2v2_ablation_focal_ctc_a0.75_g0.5-best_on-ling_head-tp0.025_tl10_fp0.001_fl16

This model is a fine-tuned version of [nguyenvulebinh/wav2vec2-base-vietnamese-250h](https://huggingface.co/nguyenvulebinh/wav2vec2-base-vietnamese-250h) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 3.0151
- Wer: 0.0928

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- total_train_batch_size: 32
- total_eval_batch_size: 64
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer     |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|
| 1337.3806     | 0.94  | 100   | 875.6539        | 18.6402 |
| 928.5113      | 1.89  | 200   | 336.9626        | 17.0922 |
| 159.8738      | 2.83  | 300   | 65.9093         | 1.0     |
| 84.4363       | 3.77  | 400   | 60.3714         | 1.0     |
| 77.6119       | 4.72  | 500   | 57.3562         | 1.0     |
| 74.61         | 5.66  | 600   | 56.1581         | 1.0     |
| 73.6043       | 6.6   | 700   | 55.2834         | 1.0     |
| 72.9912       | 7.55  | 800   | 54.6511         | 1.0     |
| 71.2358       | 8.49  | 900   | 54.5294         | 1.0     |
| 69.8267       | 9.43  | 1000  | 52.0085         | 0.9654  |
| 58.2621       | 10.38 | 1100  | 28.9713         | 0.5667  |
| 32.9698       | 11.32 | 1200  | 12.7473         | 0.2391  |
| 20.9063       | 12.26 | 1300  | 8.4371          | 0.1719  |
| 15.8491       | 13.21 | 1400  | 6.8039          | 0.1471  |
| 13.5732       | 14.15 | 1500  | 5.7138          | 0.1358  |
| 11.2257       | 15.09 | 1600  | 5.0809          | 0.1287  |
| 10.3752       | 16.04 | 1700  | 4.7094          | 0.1257  |
| 9.4428        | 16.98 | 1800  | 4.4958          | 0.1240  |
| 8.7466        | 17.92 | 1900  | 4.2252          | 0.1121  |
| 8.8036        | 18.87 | 2000  | 4.0767          | 0.1063  |
| 7.8423        | 19.81 | 2100  | 4.0642          | 0.1169  |
| 7.7711        | 20.75 | 2200  | 3.7993          | 0.1062  |
| 7.3128        | 21.7  | 2300  | 3.6860          | 0.1018  |
| 7.0561        | 22.64 | 2400  | 3.5755          | 0.1031  |
| 7.0214        | 23.58 | 2500  | 3.5520          | 0.0994  |
| 6.3277        | 24.53 | 2600  | 3.5347          | 0.1000  |
| 6.4895        | 25.47 | 2700  | 3.4519          | 0.1090  |
| 6.0361        | 26.42 | 2800  | 3.4117          | 0.1075  |
| 5.6113        | 27.36 | 2900  | 3.4126          | 0.1044  |
| 5.4031        | 28.3  | 3000  | 3.4020          | 0.0991  |
| 5.408         | 29.25 | 3100  | 3.3015          | 0.0937  |
| 5.5346        | 30.19 | 3200  | 3.4108          | 0.0962  |
| 5.5502        | 31.13 | 3300  | 3.3039          | 0.0929  |
| 4.7607        | 32.08 | 3400  | 3.3695          | 0.1028  |
| 5.3438        | 33.02 | 3500  | 3.4625          | 0.1061  |
| 5.3239        | 33.96 | 3600  | 3.4865          | 0.1063  |
| 5.156         | 34.91 | 3700  | 3.3536          | 0.1001  |
| 4.7838        | 35.85 | 3800  | 3.3247          | 0.0999  |
| 4.7075        | 36.79 | 3900  | 3.2070          | 0.1022  |
| 4.8445        | 37.74 | 4000  | 3.1779          | 0.0963  |
| 4.7855        | 38.68 | 4100  | 3.2078          | 0.0973  |
| 4.3254        | 39.62 | 4200  | 3.2060          | 0.0984  |
| 4.4259        | 40.57 | 4300  | 3.2057          | 0.0967  |
| 4.4873        | 41.51 | 4400  | 3.0877          | 0.0931  |
| 4.6976        | 42.45 | 4500  | 3.0714          | 0.0963  |
| 4.0921        | 43.4  | 4600  | 3.0722          | 0.0888  |
| 3.6267        | 44.34 | 4700  | 3.1064          | 0.0943  |
| 3.8833        | 45.28 | 4800  | 3.0917          | 0.0874  |
| 3.8643        | 46.23 | 4900  | 3.1006          | 0.0881  |
| 3.7386        | 47.17 | 5000  | 3.0927          | 0.0873  |
| 3.4363        | 48.11 | 5100  | 3.0982          | 0.0891  |
| 3.5792        | 49.06 | 5200  | 3.0596          | 0.0906  |
| 3.3444        | 50.0  | 5300  | 3.0289          | 0.0951  |
| 3.3686        | 50.94 | 5400  | 3.0119          | 0.0858  |
| 3.6072        | 51.89 | 5500  | 3.0416          | 0.0986  |
| 3.7266        | 52.83 | 5600  | 3.0389          | 0.0950  |
| 3.6465        | 53.77 | 5700  | 3.0102          | 0.0945  |
| 3.2426        | 54.72 | 5800  | 3.0769          | 0.1012  |
| 3.1878        | 55.66 | 5900  | 2.9749          | 0.0956  |
| 3.1891        | 56.6  | 6000  | 3.0639          | 0.0912  |
| 3.2342        | 57.55 | 6100  | 3.0031          | 0.0958  |
| 2.9652        | 58.49 | 6200  | 2.9965          | 0.0993  |
| 3.1089        | 59.43 | 6300  | 3.0358          | 0.0914  |
| 2.9434        | 60.38 | 6400  | 3.0805          | 0.0948  |
| 3.2816        | 61.32 | 6500  | 3.0516          | 0.0944  |
| 3.1317        | 62.26 | 6600  | 3.0206          | 0.0902  |
| 3.1278        | 63.21 | 6700  | 3.0254          | 0.0973  |
| 3.1522        | 64.15 | 6800  | 3.0528          | 0.0970  |
| 3.0941        | 65.09 | 6900  | 3.0627          | 0.0970  |
| 3.1021        | 66.04 | 7000  | 3.0484          | 0.0992  |
| 2.8751        | 66.98 | 7100  | 3.0559          | 0.0953  |
| 2.8807        | 67.92 | 7200  | 3.0577          | 0.0982  |
| 3.2996        | 68.87 | 7300  | 3.0628          | 0.0944  |
| 2.9746        | 69.81 | 7400  | 3.0304          | 0.0948  |
| 2.7453        | 70.75 | 7500  | 3.0483          | 0.0936  |
| 2.7083        | 71.7  | 7600  | 3.0759          | 0.0958  |
| 2.531         | 72.64 | 7700  | 3.0622          | 0.0962  |
| 2.6315        | 73.58 | 7800  | 3.0232          | 0.0921  |
| 2.4475        | 74.53 | 7900  | 3.0046          | 0.0918  |
| 2.6836        | 75.47 | 8000  | 3.0124          | 0.0924  |
| 2.7316        | 76.42 | 8100  | 3.0200          | 0.0896  |
| 2.7433        | 77.36 | 8200  | 3.0580          | 0.0936  |
| 2.5052        | 78.3  | 8300  | 3.0516          | 0.0934  |
| 3.1428        | 79.25 | 8400  | 3.0461          | 0.0936  |
| 2.7542        | 80.19 | 8500  | 3.0198          | 0.0947  |
| 2.7269        | 81.13 | 8600  | 3.0262          | 0.0945  |
| 2.7809        | 82.08 | 8700  | 3.0139          | 0.0897  |
| 2.3545        | 83.02 | 8800  | 3.0183          | 0.0924  |
| 2.4138        | 83.96 | 8900  | 3.0209          | 0.0921  |
| 2.4908        | 84.91 | 9000  | 3.0268          | 0.0924  |
| 2.6911        | 85.85 | 9100  | 3.0228          | 0.0948  |
| 2.4881        | 86.79 | 9200  | 3.0194          | 0.0922  |
| 2.6499        | 87.74 | 9300  | 3.0090          | 0.0908  |
| 2.5886        | 88.68 | 9400  | 3.0162          | 0.0917  |
| 2.6444        | 89.62 | 9500  | 3.0180          | 0.0909  |
| 2.5907        | 90.57 | 9600  | 3.0199          | 0.0908  |
| 2.6175        | 91.51 | 9700  | 3.0198          | 0.0923  |
| 2.8366        | 92.45 | 9800  | 3.0164          | 0.0915  |
| 2.5604        | 93.4  | 9900  | 3.0118          | 0.0912  |
| 2.4371        | 94.34 | 10000 | 3.0124          | 0.0908  |
| 2.6646        | 95.28 | 10100 | 3.0187          | 0.0920  |
| 2.5563        | 96.23 | 10200 | 3.0140          | 0.0919  |
| 2.8501        | 97.17 | 10300 | 3.0144          | 0.0919  |
| 2.6802        | 98.11 | 10400 | 3.0150          | 0.0923  |
| 2.3091        | 99.06 | 10500 | 3.0150          | 0.0926  |
| 2.6642        | 100.0 | 10600 | 3.0151          | 0.0928  |


### Framework versions

- Transformers 4.35.2
- Pytorch 1.13.1+cu117
- Datasets 2.12.0
- Tokenizers 0.14.1
