transformers tensorboard safetensors wav2vec2 automatic-speech-recognition generated_from_trainer dataset:common_voice_11_0 base_model:facebook/mms-1b-all license:cc-by-nc-4.0 model-index endpoints_compatible region:us
---
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
- generated_from_trainer
datasets:
- common_voice_11_0
metrics:
- wer
model-index:
- name: wav2vec2-large-mms-1b-kyrgyz-colab
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: common_voice_11_0
      type: common_voice_11_0
      config: ky
      split: test
      args: ky
    metrics:
    - name: Wer
      type: wer
      value: 1.002755073927817
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-mms-1b-kyrgyz-colab

This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the common_voice_11_0 dataset.
It achieves the following results on the evaluation set:
- Loss: 3.6994
- Wer: 1.0028

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 15
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 8.657         | 0.59  | 1000  | 5.3559          | 1.0    |
| 3.7087        | 1.18  | 2000  | 4.2908          | 1.0    |
| 3.4394        | 1.76  | 3000  | 4.1226          | 1.0    |
| 3.3918        | 2.35  | 4000  | 4.0324          | 1.0    |
| 3.3357        | 2.94  | 5000  | 3.9328          | 1.0    |
| 3.2915        | 3.53  | 6000  | 3.8575          | 0.9998 |
| 3.245         | 4.12  | 7000  | 3.7375          | 1.0001 |
| 3.219         | 4.71  | 8000  | 3.7220          | 1.0003 |
| 3.1878        | 5.29  | 9000  | 3.7172          | 1.0008 |
| 3.1826        | 5.88  | 10000 | 3.7064          | 1.0012 |
| 3.1613        | 6.47  | 11000 | 3.7029          | 1.0014 |
| 3.1548        | 7.06  | 12000 | 3.6923          | 1.0011 |
| 3.1403        | 7.65  | 13000 | 3.7013          | 1.0010 |
| 3.1286        | 8.24  | 14000 | 3.6862          | 1.0017 |
| 3.1184        | 8.82  | 15000 | 3.7129          | 1.0028 |
| 3.1198        | 9.41  | 16000 | 3.7063          | 1.0022 |
| 3.1051        | 10.0  | 17000 | 3.7093          | 1.0018 |
| 3.098         | 10.59 | 18000 | 3.6952          | 1.0031 |
| 3.1001        | 11.18 | 19000 | 3.7042          | 1.0031 |
| 3.0984        | 11.76 | 20000 | 3.6972          | 1.0020 |
| 3.0873        | 12.35 | 21000 | 3.7018          | 1.0026 |
| 3.0852        | 12.94 | 22000 | 3.7005          | 1.0025 |
| 3.0902        | 13.53 | 23000 | 3.6972          | 1.0026 |
| 3.081         | 14.12 | 24000 | 3.6984          | 1.0024 |
| 3.0777        | 14.71 | 25000 | 3.6994          | 1.0028 |


### Framework versions

- Transformers 4.37.0.dev0
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0
