
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xls-r-300m-et-children

This model is a fine-tuned version of [TalTechNLP/xls-r-300m-et](https://huggingface.co/TalTechNLP/xls-r-300m-et) on an Estonian children's speech dataset.

More information about the model's performance and the data used for evaluation and training:

Luhtaru, Agnes; Jaaska, Rauno; Kruusamäe, Karl; Fishel, Mark (2023). Automatic Transcription for Estonian Children’s Speech. In: Proceedings of the 24th Nordic Conference on Computational Linguistics. [https://openreview.net/forum?id=xbPTfBIUby](https://openreview.net/forum?id=xbPTfBIUby)

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 60

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.3485        | 1.61  | 100  | 0.2034          | 0.1782 |
| 0.1914        | 3.22  | 200  | 0.1818          | 0.1606 |
| 0.1641        | 4.83  | 300  | 0.1770          | 0.1572 |
| 0.1479        | 6.45  | 400  | 0.1745          | 0.1546 |
| 0.1374        | 8.06  | 500  | 0.1751          | 0.1538 |
| 0.1244        | 9.67  | 600  | 0.1734          | 0.1518 |
| 0.1211        | 11.29 | 700  | 0.1753          | 0.1508 |
| 0.1096        | 12.9  | 800  | 0.1758          | 0.1483 |
| 0.1059        | 14.51 | 900  | 0.1771          | 0.1469 |
| 0.0991        | 16.13 | 1000 | 0.1776          | 0.1469 |
| 0.0965        | 17.74 | 1100 | 0.1759          | 0.1469 |
| 0.0944        | 19.35 | 1200 | 0.1784          | 0.1459 |
| 0.0902        | 20.96 | 1300 | 0.1799          | 0.1469 |
| 0.0867        | 22.58 | 1400 | 0.1814          | 0.1440 |
| 0.0817        | 24.19 | 1500 | 0.1828          | 0.1438 |
| 0.0802        | 25.8  | 1600 | 0.1845          | 0.1438 |
| 0.0762        | 27.42 | 1700 | 0.1843          | 0.1431 |
| 0.0774        | 29.03 | 1800 | 0.1839          | 0.1432 |
| 0.0741        | 30.64 | 1900 | 0.1843          | 0.1442 |
| 0.0718        | 32.26 | 2000 | 0.1846          | 0.1429 |
| 0.07          | 33.86 | 2100 | 0.1852          | 0.1429 |
| 0.0692        | 35.48 | 2200 | 0.1872          | 0.1435 |
| 0.0671        | 37.1  | 2300 | 0.1874          | 0.1433 |
| 0.0667        | 38.7  | 2400 | 0.1887          | 0.1435 |
| 0.066         | 40.32 | 2500 | 0.1880          | 0.1422 |
| 0.0628        | 41.93 | 2600 | 0.1897          | 0.1426 |
| 0.0643        | 43.54 | 2700 | 0.1910          | 0.1428 |
| 0.0643        | 45.16 | 2800 | 0.1900          | 0.1431 |
| 0.0638        | 46.77 | 2900 | 0.1900          | 0.1427 |
| 0.0601        | 48.38 | 3000 | 0.1911          | 0.1431 |
| 0.0593        | 49.99 | 3100 | 0.1914          | 0.1432 |
| 0.0606        | 51.61 | 3200 | 0.1912          | 0.1433 |
| 0.0609        | 53.22 | 3300 | 0.1912          | 0.1431 |
| 0.0587        | 54.83 | 3400 | 0.1921          | 0.1429 |
| 0.0567        | 56.45 | 3500 | 0.1924          | 0.1430 |
| 0.0611        | 58.06 | 3600 | 0.1927          | 0.1431 |
| 0.0581        | 59.67 | 3700 | 0.1926          | 0.1430 |


### Framework versions

- Transformers 4.26.0.dev0
- Pytorch 1.12.1+rocm5.1.1
- Datasets 2.7.1.dev0
- Tokenizers 0.13.2
{"language": ["et"], "license": "cc-by-4.0", "library_name": "transformers", "tags": ["generated_from_trainer"], "metrics": ["wer"], "model-index": [{"name": "xls-r-300m-et-children", "results": []}]}
transformers pytorch wav2vec2 automatic-speech-recognition generated_from_trainer et license:cc-by-4.0 endpoints_compatible region:us
TAGS
#transformers #pytorch #wav2vec2 #automatic-speech-recognition #generated_from_trainer #et #license-cc-by-4.0 #endpoints_compatible #region-us 

xls-r-300m-et-children
======================


This model is a fine-tuned version of TalTechNLP/xls-r-300m-et on an Estonian children's speech dataset.


More information about the model's performance and the data used for evaluation and training:


Luhtaru, Agnes; Jaaska, Rauno; Kruusamäe, Karl; Fishel, Mark (2023). Automatic Transcription for Estonian Children’s Speech. In: Proceedings of the 24th Nordic Conference on Computational Linguistics. URL


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 32
* eval\_batch\_size: 8
* seed: 42
* gradient\_accumulation\_steps: 2
* total\_train\_batch\_size: 64
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* lr\_scheduler\_warmup\_steps: 200
* num\_epochs: 60


### Training results



### Framework versions


* Transformers 4.26.0.dev0
* Pytorch 1.12.1+rocm5.1.1
* Datasets 2.7.1.dev0
* Tokenizers 0.13.2



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 32
* eval\_batch\_size: 8
* seed: 42
* gradient\_accumulation\_steps: 2
* total\_train\_batch\_size: 64
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* lr\_scheduler\_warmup\_steps: 200
* num\_epochs: 60 ### Training results ### Framework versions


* Transformers 4.26.0.dev0
* Pytorch 1.12.1+rocm5.1.1
* Datasets 2.7.1.dev0
* Tokenizers 0.13.2
TAGS
#transformers #pytorch #wav2vec2 #automatic-speech-recognition #generated_from_trainer #et #license-cc-by-4.0 #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 32
* eval\_batch\_size: 8
* seed: 42
* gradient\_accumulation\_steps: 2
* total\_train\_batch\_size: 64
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* lr\_scheduler\_warmup\_steps: 200
* num\_epochs: 60 ### Training results ### Framework versions


* Transformers 4.26.0.dev0
* Pytorch 1.12.1+rocm5.1.1
* Datasets 2.7.1.dev0
* Tokenizers 0.13.2
TAGS
#transformers #pytorch #wav2vec2 #automatic-speech-recognition #generated_from_trainer #et #license-cc-by-4.0 #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 32
* eval\_batch\_size: 8
* seed: 42
* gradient\_accumulation\_steps: 2
* total\_train\_batch\_size: 64
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* lr\_scheduler\_warmup\_steps: 200
* num\_epochs: 60### Training results### Framework versions


* Transformers 4.26.0.dev0
* Pytorch 1.12.1+rocm5.1.1
* Datasets 2.7.1.dev0
* Tokenizers 0.13.2