transformers safetensors wav2vec2 automatic-speech-recognition generated_from_trainer base_model:facebook/wav2vec2-xls-r-300m license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
base_model: facebook/wav2vec2-xls-r-300m
tags:
- generated_from_trainer
model-index:
- name: ft_0123_korean_1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ft_0123_korean_1

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7160
- Cer: 0.1832

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 8
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Cer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 60.2779       | 0.09  | 100  | 120.0675        | 0.9556 |
| 45.2315       | 0.18  | 200  | 66.7612         | 1.0    |
| 30.7852       | 0.27  | 300  | 50.1041         | 1.0    |
| 22.2754       | 0.35  | 400  | 31.5537         | 1.0    |
| 13.4785       | 0.44  | 500  | 13.3387         | 1.0    |
| 6.4901        | 0.53  | 600  | 5.4424          | 1.0    |
| 4.8734        | 0.62  | 700  | 5.0449          | 1.0    |
| 4.7501        | 0.71  | 800  | 4.9923          | 1.0    |
| 4.7477        | 0.8   | 900  | 4.9037          | 1.0    |
| 4.7511        | 0.88  | 1000 | 4.9181          | 1.0    |
| 4.7076        | 0.97  | 1100 | 4.8488          | 1.0    |
| 4.6829        | 1.06  | 1200 | 4.7397          | 1.0    |
| 4.6768        | 1.15  | 1300 | 4.7180          | 1.0    |
| 4.6089        | 1.24  | 1400 | 4.6611          | 1.0    |
| 4.56          | 1.33  | 1500 | 4.6175          | 1.0    |
| 4.499         | 1.41  | 1600 | 4.5056          | 0.9999 |
| 4.38          | 1.5   | 1700 | 4.2237          | 0.9857 |
| 4.1179        | 1.59  | 1800 | 3.8646          | 0.9816 |
| 3.7534        | 1.68  | 1900 | 3.2313          | 0.6453 |
| 3.3149        | 1.77  | 2000 | 2.8237          | 0.5349 |
| 3.0137        | 1.86  | 2100 | 2.5717          | 0.5091 |
| 2.8094        | 1.94  | 2200 | 2.3614          | 0.4624 |
| 2.5519        | 2.03  | 2300 | 2.2160          | 0.4501 |
| 2.4768        | 2.12  | 2400 | 2.0897          | 0.4274 |
| 2.3625        | 2.21  | 2500 | 1.9856          | 0.4168 |
| 2.2876        | 2.3   | 2600 | 1.9065          | 0.4016 |
| 2.2289        | 2.39  | 2700 | 1.8137          | 0.3930 |
| 2.0996        | 2.47  | 2800 | 1.7548          | 0.3755 |
| 2.0683        | 2.56  | 2900 | 1.6713          | 0.3590 |
| 2.0371        | 2.65  | 3000 | 1.6082          | 0.3528 |
| 1.8858        | 2.74  | 3100 | 1.5596          | 0.3453 |
| 1.8692        | 2.83  | 3200 | 1.5088          | 0.3346 |
| 1.8373        | 2.92  | 3300 | 1.4700          | 0.3299 |
| 1.8318        | 3.0   | 3400 | 1.4286          | 0.3208 |
| 1.7482        | 3.09  | 3500 | 1.4024          | 0.3151 |
| 1.6887        | 3.18  | 3600 | 1.3581          | 0.3126 |
| 1.6574        | 3.27  | 3700 | 1.3161          | 0.2998 |
| 1.6119        | 3.36  | 3800 | 1.2720          | 0.2924 |
| 1.6103        | 3.45  | 3900 | 1.2531          | 0.2921 |
| 1.5362        | 3.53  | 4000 | 1.2326          | 0.2917 |
| 1.4972        | 3.62  | 4100 | 1.1904          | 0.2816 |
| 1.5005        | 3.71  | 4200 | 1.1757          | 0.2784 |
| 1.4586        | 3.8   | 4300 | 1.1463          | 0.2737 |
| 1.4483        | 3.89  | 4400 | 1.1246          | 0.2694 |
| 1.4354        | 3.98  | 4500 | 1.0976          | 0.2641 |
| 1.3648        | 4.06  | 4600 | 1.0730          | 0.2606 |
| 1.3194        | 4.15  | 4700 | 1.0460          | 0.2579 |
| 1.3316        | 4.24  | 4800 | 1.0362          | 0.2516 |
| 1.3138        | 4.33  | 4900 | 1.0166          | 0.2475 |
| 1.3217        | 4.42  | 5000 | 0.9917          | 0.2456 |
| 1.2914        | 4.51  | 5100 | 0.9835          | 0.2411 |
| 1.2364        | 4.59  | 5200 | 0.9647          | 0.2409 |
| 1.2034        | 4.68  | 5300 | 0.9621          | 0.2368 |
| 1.2028        | 4.77  | 5400 | 0.9255          | 0.2311 |
| 1.2354        | 4.86  | 5500 | 0.9119          | 0.2280 |
| 1.2295        | 4.95  | 5600 | 0.9113          | 0.2287 |
| 1.2007        | 5.04  | 5700 | 0.8934          | 0.2229 |
| 1.1637        | 5.12  | 5800 | 0.8867          | 0.2256 |
| 1.1221        | 5.21  | 5900 | 0.8787          | 0.2213 |
| 1.171         | 5.3   | 6000 | 0.8607          | 0.2176 |
| 1.1042        | 5.39  | 6100 | 0.8514          | 0.2171 |
| 1.063         | 5.48  | 6200 | 0.8510          | 0.2175 |
| 1.0965        | 5.57  | 6300 | 0.8355          | 0.2107 |
| 1.0611        | 5.65  | 6400 | 0.8298          | 0.2096 |
| 1.0697        | 5.74  | 6500 | 0.8149          | 0.2074 |
| 1.0342        | 5.83  | 6600 | 0.8043          | 0.2037 |
| 1.0586        | 5.92  | 6700 | 0.8060          | 0.2028 |
| 1.0553        | 6.01  | 6800 | 0.8017          | 0.2029 |
| 1.0369        | 6.1   | 6900 | 0.7906          | 0.2015 |
| 0.9646        | 6.18  | 7000 | 0.7870          | 0.1987 |
| 0.9747        | 6.27  | 7100 | 0.7836          | 0.1970 |
| 0.9933        | 6.36  | 7200 | 0.7708          | 0.1963 |
| 0.9793        | 6.45  | 7300 | 0.7740          | 0.1957 |
| 0.9642        | 6.54  | 7400 | 0.7618          | 0.1934 |
| 0.9936        | 6.63  | 7500 | 0.7554          | 0.1919 |
| 0.9466        | 6.71  | 7600 | 0.7438          | 0.1891 |
| 0.9597        | 6.8   | 7700 | 0.7437          | 0.1900 |
| 0.9374        | 6.89  | 7800 | 0.7415          | 0.1909 |
| 0.9719        | 6.98  | 7900 | 0.7352          | 0.1908 |
| 0.9067        | 7.07  | 8000 | 0.7358          | 0.1880 |
| 0.8998        | 7.16  | 8100 | 0.7329          | 0.1879 |
| 0.9271        | 7.24  | 8200 | 0.7262          | 0.1864 |
| 0.8951        | 7.33  | 8300 | 0.7217          | 0.1860 |
| 0.9136        | 7.42  | 8400 | 0.7239          | 0.1854 |
| 0.9446        | 7.51  | 8500 | 0.7214          | 0.1844 |
| 0.8978        | 7.6   | 8600 | 0.7220          | 0.1837 |
| 0.8923        | 7.69  | 8700 | 0.7174          | 0.1838 |
| 0.9406        | 7.77  | 8800 | 0.7187          | 0.1836 |
| 0.9242        | 7.86  | 8900 | 0.7159          | 0.1836 |
| 0.8994        | 7.95  | 9000 | 0.7160          | 0.1832 |


### Framework versions

- Transformers 4.36.2
- Pytorch 2.1.2+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0
