Initial model

.ipynb_checkpoints/README-checkpoint.md

@@ -0,0 +1,272 @@
+---
+language: et
+datasets:
+- common_voice
+tags:
+- audio
+- automatic-speech-recognition
+- speech
+- xlsr-fine-tuning-week
+license: apache-2.0
+widget:
+- label: Common Voice sample 1123
+  src: https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-estonian/resolve/main/sample1123.flac
+- label: Common Voice sample 910
+  src: https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-estonian/resolve/main/sample910.flac
+model-index:
+- name: XLSR Wav2Vec2 Estonian by Mehrdad Farahani
+  results:
+  - task: 
+      name: Speech Recognition
+      type: automatic-speech-recognition
+    dataset:
+      name: Common Voice et
+      type: common_voice
+      args: et
+    metrics:
+       - name: Test WER
+         type: wer
+         value: 33.73
+        
+---
+
+# Wav2Vec2-Large-XLSR-53-Estonian
+
+Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Estonian using [Common Voice](https://huggingface.co/datasets/common_voice). When using this model, make sure that your speech input is sampled at 16kHz.
+
+## Usage
+The model can be used directly (without a language model) as follows:
+
+**Requirements**
+```bash
+# requirement packages
+!pip install git+https://github.com/huggingface/datasets.git
+!pip install git+https://github.com/huggingface/transformers.git
+!pip install torchaudio
+!pip install librosa
+!pip install jiwer
+```
+
+
+**Prediction**
+```python
+import librosa
+import torch
+import torchaudio
+from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
+from datasets import load_dataset
+
+import numpy as np
+import re
+import string
+
+import IPython.display as ipd
+
+chars_to_ignore = [
+    ",", "?", ".", "!", "-", ";", ":", '""', "%", "'", '"', "�",
+    "#", "!", "?", "«", "»", "(", ")", "؛", ",", "?", ".", "!", "-", ";", ":", '"', 
+    "“", "%", "‘", "�", "–", "…", "_", "”", '“', '„'
+]
+chars_to_mapping = {
+"\u200c": " ", "\u200d": " ", "\u200e": " ", "\u200f": " ", "\ufeff": " ",
+}
+
+def multiple_replace(text, chars_to_mapping):
+    pattern = "|".join(map(re.escape, chars_to_mapping.keys()))
+    return re.sub(pattern, lambda m: chars_to_mapping[m.group()], str(text))
+
+def remove_special_characters(text, chars_to_ignore_regex):
+    text = re.sub(chars_to_ignore_regex, '', text).lower() + " "
+    return text
+
+def normalizer(batch, chars_to_ignore, chars_to_mapping):
+    chars_to_ignore_regex = f"""[{"".join(chars_to_ignore)}]"""
+    text = batch["sentence"].lower().strip()
+    
+    text = text.replace("\u0307", " ").strip()
+    text = multiple_replace(text, chars_to_mapping)
+    text = remove_special_characters(text, chars_to_ignore_regex)
+
+    batch["sentence"] = text
+    return batch
+
+
+def speech_file_to_array_fn(batch):
+    speech_array, sampling_rate = torchaudio.load(batch["path"])
+    speech_array = speech_array.squeeze().numpy()
+    speech_array = librosa.resample(np.asarray(speech_array), sampling_rate, 16_000)
+
+    batch["speech"] = speech_array
+    return batch
+
+
+def predict(batch):
+    features = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
+
+    input_values = features.input_values.to(device)
+    attention_mask = features.attention_mask.to(device)
+
+    with torch.no_grad():
+        logits = model(input_values, attention_mask=attention_mask).logits 
+        
+    pred_ids = torch.argmax(logits, dim=-1)
+
+    batch["predicted"] = processor.batch_decode(pred_ids)[0]
+    return batch
+
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+processor = Wav2Vec2Processor.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-estonian")
+model = Wav2Vec2ForCTC.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-estonian").to(device)
+
+dataset = load_dataset("common_voice", "et", split="test[:1%]")
+dataset = dataset.map(
+    normalizer, 
+    fn_kwargs={"chars_to_ignore": chars_to_ignore, "chars_to_mapping": chars_to_mapping},
+    remove_columns=list(set(dataset.column_names) - set(['sentence', 'path']))
+)
+
+dataset = dataset.map(speech_file_to_array_fn)
+result = dataset.map(predict)
+
+max_items = np.random.randint(0, len(result), 10).tolist()
+for i in max_items:
+    reference, predicted =  result["sentence"][i], result["predicted"][i]
+    print("reference:", reference)
+    print("predicted:", predicted)
+    print('---')
+```
+
+**Output:**
+```text
+reference: õhulossid lagunevad ning ees ootab maapind 
+predicted: õhulassid lagunevad ning ees ootab maapind
+---
+reference: milliseks kiievisse pääsemise nimel võistlev muusik soome muusikamaastiku hetkeseisu hindab ning kas ta ka ennast sellel tulevikus tegutsemas näeb kuuled videost 
+predicted: milliseks gievisse pääsemise nimel võitlev muusiks soome muusikama aastiku hetke seisu hindab ning kas ta ennast selle tulevikus tegutsemast näeb kuulad videost
+---
+reference: näiteks kui pool seina on tehtud tekib tunne et tahaks tegelikult natuke teistsugust ja hakkame otsast peale 
+predicted: näiteks kui pool seine on tehtud tekib tunnetahaks tegelikult matuka teistsugust jahappanna otsast peane
+---
+reference: neuroesteetilised katsed näitavad et just nägude vaatlemine aktiveerib inimese aju esteetilist keskust 
+predicted: neuroaisteetiliselt katsed näitaval et just nägude vaatlemine aptiveerid inimese aju est eedilist keskust
+---
+reference: paljud inimesed kindlasti kadestavad teid kuid ei julge samamoodi vabalt võtta 
+predicted: paljud inimesed kindlasti kadestavadteid kuid ei julge sama moodi vabalt võtta
+---
+reference: parem on otsida pileteid inkognito veebi kaudu 
+predicted: parem on otsida pileteid ning kognitu veebikaudu
+---
+reference: ja vot siin ma jäin vaikseks 
+predicted: ja vat siisma ja invaikseks
+---
+reference: mida sa iseendale juubeli puhul soovid 
+predicted: mida saise endale jubeli puhul soovid
+---
+reference: kuumuse ja kõrge temperatuuri tõttu kuivas tühjadel karjamaadel rohi mis muutus kergesti süttivaks 
+predicted: kuumuse ja kõrge temperatuuri tõttu kuivast ühjadal karjamaadel rohi mis muutus kergesti süttivaks
+---
+reference: ilmselt on inimesi kelle jaoks on see hea lahendus 
+predicted: ilmselt on inimesi kelle jaoks on see hea lahendus
+---
+```
+
+
+## Evaluation
+
+The model can be evaluated as follows on the Estonian test data of Common Voice.
+
+```python
+import librosa
+import torch
+import torchaudio
+from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
+from datasets import load_dataset, load_metric
+
+import numpy as np
+import re
+import string
+
+
+chars_to_ignore = [
+    ",", "?", ".", "!", "-", ";", ":", '""', "%", "'", '"', "�",
+    "#", "!", "?", "«", "»", "(", ")", "؛", ",", "?", ".", "!", "-", ";", ":", '"', 
+    "“", "%", "‘", "�", "–", "…", "_", "”", '“', '„'
+]
+chars_to_mapping = {
+    "\u200c": " ", "\u200d": " ", "\u200e": " ", "\u200f": " ", "\ufeff": " ",
+}
+
+def multiple_replace(text, chars_to_mapping):
+    pattern = "|".join(map(re.escape, chars_to_mapping.keys()))
+    return re.sub(pattern, lambda m: chars_to_mapping[m.group()], str(text))
+
+def remove_special_characters(text, chars_to_ignore_regex):
+    text = re.sub(chars_to_ignore_regex, '', text).lower() + " "
+    return text
+
+def normalizer(batch, chars_to_ignore, chars_to_mapping):
+    chars_to_ignore_regex = f"""[{"".join(chars_to_ignore)}]"""
+    text = batch["sentence"].lower().strip()
+    
+    text = multiple_replace(text, chars_to_mapping)
+    text = remove_special_characters(text, chars_to_ignore_regex)
+
+    batch["sentence"] = text
+    return batch
+
+
+def speech_file_to_array_fn(batch):
+    speech_array, sampling_rate = torchaudio.load(batch["path"])
+    speech_array = speech_array.squeeze().numpy()
+    speech_array = librosa.resample(np.asarray(speech_array), sampling_rate, 16_000)
+
+    batch["speech"] = speech_array
+    return batch
+
+
+def predict(batch):
+    features = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
+
+    input_values = features.input_values.to(device)
+    attention_mask = features.attention_mask.to(device)
+
+    with torch.no_grad():
+        logits = model(input_values, attention_mask=attention_mask).logits 
+        
+    pred_ids = torch.argmax(logits, dim=-1)
+
+    batch["predicted"] = processor.batch_decode(pred_ids)[0]
+    return batch
+
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+processor = Wav2Vec2Processor.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-estonian")
+model = Wav2Vec2ForCTC.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-estonian").to(device)
+
+dataset = load_dataset("common_voice", "et", split="test")
+dataset = dataset.map(
+    normalizer, 
+    fn_kwargs={"chars_to_ignore": chars_to_ignore, "chars_to_mapping": chars_to_mapping},
+    remove_columns=list(set(dataset.column_names) - set(['sentence', 'path']))
+)
+
+dataset = dataset.map(speech_file_to_array_fn)
+result = dataset.map(predict)
+
+wer = load_metric("wer")
+
+print("WER: {:.2f}".format(100 * wer.compute(predictions=result["predicted"], references=result["sentence"])))
+```
+]
+
+**Test Result**: 
+- WER: 33.73%
+
+
+## Training & Report
+The Common Voice `train`, `validation` datasets were used for training.
+
+You can see the training states [here](https://wandb.ai/m3hrdadfi/finetuned_wav2vec_xlsr_estonian/reports/Fine-Tuning-for-Wav2Vec2-Large-XLSR-53-Estonian--Vmlldzo1NjA1MTI?accessToken=k2b2g3a2i12m1sdwf13q8b226pplmmyw12joxo6vk38eb4djellfzmn9fp2725fw)
+
+The script used for training can be found [here](https://colab.research.google.com/github/m3hrdadfi/notebooks/blob/main/Fine_Tune_XLSR_Wav2Vec2_on_Estonian_ASR_with_%F0%9F%A4%97_Transformers_ipynb.ipynb)

README.md

@@ -0,0 +1,272 @@
+---
+language: et
+datasets:
+- common_voice
+tags:
+- audio
+- automatic-speech-recognition
+- speech
+- xlsr-fine-tuning-week
+license: apache-2.0
+widget:
+- label: Common Voice sample 1123
+  src: https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-estonian/resolve/main/sample1123.flac
+- label: Common Voice sample 910
+  src: https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-estonian/resolve/main/sample910.flac
+model-index:
+- name: XLSR Wav2Vec2 Estonian by Mehrdad Farahani
+  results:
+  - task: 
+      name: Speech Recognition
+      type: automatic-speech-recognition
+    dataset:
+      name: Common Voice et
+      type: common_voice
+      args: et
+    metrics:
+       - name: Test WER
+         type: wer
+         value: 33.73
+        
+---
+
+# Wav2Vec2-Large-XLSR-53-Estonian
+
+Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Estonian using [Common Voice](https://huggingface.co/datasets/common_voice). When using this model, make sure that your speech input is sampled at 16kHz.
+
+## Usage
+The model can be used directly (without a language model) as follows:
+
+**Requirements**
+```bash
+# requirement packages
+!pip install git+https://github.com/huggingface/datasets.git
+!pip install git+https://github.com/huggingface/transformers.git
+!pip install torchaudio
+!pip install librosa
+!pip install jiwer
+```
+
+
+**Prediction**
+```python
+import librosa
+import torch
+import torchaudio
+from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
+from datasets import load_dataset
+
+import numpy as np
+import re
+import string
+
+import IPython.display as ipd
+
+chars_to_ignore = [
+    ",", "?", ".", "!", "-", ";", ":", '""', "%", "'", '"', "�",
+    "#", "!", "?", "«", "»", "(", ")", "؛", ",", "?", ".", "!", "-", ";", ":", '"', 
+    "“", "%", "‘", "�", "–", "…", "_", "”", '“', '„'
+]
+chars_to_mapping = {
+"\u200c": " ", "\u200d": " ", "\u200e": " ", "\u200f": " ", "\ufeff": " ",
+}
+
+def multiple_replace(text, chars_to_mapping):
+    pattern = "|".join(map(re.escape, chars_to_mapping.keys()))
+    return re.sub(pattern, lambda m: chars_to_mapping[m.group()], str(text))
+
+def remove_special_characters(text, chars_to_ignore_regex):
+    text = re.sub(chars_to_ignore_regex, '', text).lower() + " "
+    return text
+
+def normalizer(batch, chars_to_ignore, chars_to_mapping):
+    chars_to_ignore_regex = f"""[{"".join(chars_to_ignore)}]"""
+    text = batch["sentence"].lower().strip()
+    
+    text = text.replace("\u0307", " ").strip()
+    text = multiple_replace(text, chars_to_mapping)
+    text = remove_special_characters(text, chars_to_ignore_regex)
+
+    batch["sentence"] = text
+    return batch
+
+
+def speech_file_to_array_fn(batch):
+    speech_array, sampling_rate = torchaudio.load(batch["path"])
+    speech_array = speech_array.squeeze().numpy()
+    speech_array = librosa.resample(np.asarray(speech_array), sampling_rate, 16_000)
+
+    batch["speech"] = speech_array
+    return batch
+
+
+def predict(batch):
+    features = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
+
+    input_values = features.input_values.to(device)
+    attention_mask = features.attention_mask.to(device)
+
+    with torch.no_grad():
+        logits = model(input_values, attention_mask=attention_mask).logits 
+        
+    pred_ids = torch.argmax(logits, dim=-1)
+
+    batch["predicted"] = processor.batch_decode(pred_ids)[0]
+    return batch
+
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+processor = Wav2Vec2Processor.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-estonian")
+model = Wav2Vec2ForCTC.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-estonian").to(device)
+
+dataset = load_dataset("common_voice", "et", split="test[:1%]")
+dataset = dataset.map(
+    normalizer, 
+    fn_kwargs={"chars_to_ignore": chars_to_ignore, "chars_to_mapping": chars_to_mapping},
+    remove_columns=list(set(dataset.column_names) - set(['sentence', 'path']))
+)
+
+dataset = dataset.map(speech_file_to_array_fn)
+result = dataset.map(predict)
+
+max_items = np.random.randint(0, len(result), 10).tolist()
+for i in max_items:
+    reference, predicted =  result["sentence"][i], result["predicted"][i]
+    print("reference:", reference)
+    print("predicted:", predicted)
+    print('---')
+```
+
+**Output:**
+```text
+reference: õhulossid lagunevad ning ees ootab maapind 
+predicted: õhulassid lagunevad ning ees ootab maapind
+---
+reference: milliseks kiievisse pääsemise nimel võistlev muusik soome muusikamaastiku hetkeseisu hindab ning kas ta ka ennast sellel tulevikus tegutsemas näeb kuuled videost 
+predicted: milliseks gievisse pääsemise nimel võitlev muusiks soome muusikama aastiku hetke seisu hindab ning kas ta ennast selle tulevikus tegutsemast näeb kuulad videost
+---
+reference: näiteks kui pool seina on tehtud tekib tunne et tahaks tegelikult natuke teistsugust ja hakkame otsast peale 
+predicted: näiteks kui pool seine on tehtud tekib tunnetahaks tegelikult matuka teistsugust jahappanna otsast peane
+---
+reference: neuroesteetilised katsed näitavad et just nägude vaatlemine aktiveerib inimese aju esteetilist keskust 
+predicted: neuroaisteetiliselt katsed näitaval et just nägude vaatlemine aptiveerid inimese aju est eedilist keskust
+---
+reference: paljud inimesed kindlasti kadestavad teid kuid ei julge samamoodi vabalt võtta 
+predicted: paljud inimesed kindlasti kadestavadteid kuid ei julge sama moodi vabalt võtta
+---
+reference: parem on otsida pileteid inkognito veebi kaudu 
+predicted: parem on otsida pileteid ning kognitu veebikaudu
+---
+reference: ja vot siin ma jäin vaikseks 
+predicted: ja vat siisma ja invaikseks
+---
+reference: mida sa iseendale juubeli puhul soovid 
+predicted: mida saise endale jubeli puhul soovid
+---
+reference: kuumuse ja kõrge temperatuuri tõttu kuivas tühjadel karjamaadel rohi mis muutus kergesti süttivaks 
+predicted: kuumuse ja kõrge temperatuuri tõttu kuivast ühjadal karjamaadel rohi mis muutus kergesti süttivaks
+---
+reference: ilmselt on inimesi kelle jaoks on see hea lahendus 
+predicted: ilmselt on inimesi kelle jaoks on see hea lahendus
+---
+```
+
+
+## Evaluation
+
+The model can be evaluated as follows on the Estonian test data of Common Voice.
+
+```python
+import librosa
+import torch
+import torchaudio
+from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
+from datasets import load_dataset, load_metric
+
+import numpy as np
+import re
+import string
+
+
+chars_to_ignore = [
+    ",", "?", ".", "!", "-", ";", ":", '""', "%", "'", '"', "�",
+    "#", "!", "?", "«", "»", "(", ")", "؛", ",", "?", ".", "!", "-", ";", ":", '"', 
+    "“", "%", "‘", "�", "–", "…", "_", "”", '“', '„'
+]
+chars_to_mapping = {
+    "\u200c": " ", "\u200d": " ", "\u200e": " ", "\u200f": " ", "\ufeff": " ",
+}
+
+def multiple_replace(text, chars_to_mapping):
+    pattern = "|".join(map(re.escape, chars_to_mapping.keys()))
+    return re.sub(pattern, lambda m: chars_to_mapping[m.group()], str(text))
+
+def remove_special_characters(text, chars_to_ignore_regex):
+    text = re.sub(chars_to_ignore_regex, '', text).lower() + " "
+    return text
+
+def normalizer(batch, chars_to_ignore, chars_to_mapping):
+    chars_to_ignore_regex = f"""[{"".join(chars_to_ignore)}]"""
+    text = batch["sentence"].lower().strip()
+    
+    text = multiple_replace(text, chars_to_mapping)
+    text = remove_special_characters(text, chars_to_ignore_regex)
+
+    batch["sentence"] = text
+    return batch
+
+
+def speech_file_to_array_fn(batch):
+    speech_array, sampling_rate = torchaudio.load(batch["path"])
+    speech_array = speech_array.squeeze().numpy()
+    speech_array = librosa.resample(np.asarray(speech_array), sampling_rate, 16_000)
+
+    batch["speech"] = speech_array
+    return batch
+
+
+def predict(batch):
+    features = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
+
+    input_values = features.input_values.to(device)
+    attention_mask = features.attention_mask.to(device)
+
+    with torch.no_grad():
+        logits = model(input_values, attention_mask=attention_mask).logits 
+        
+    pred_ids = torch.argmax(logits, dim=-1)
+
+    batch["predicted"] = processor.batch_decode(pred_ids)[0]
+    return batch
+
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+processor = Wav2Vec2Processor.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-estonian")
+model = Wav2Vec2ForCTC.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-estonian").to(device)
+
+dataset = load_dataset("common_voice", "et", split="test")
+dataset = dataset.map(
+    normalizer, 
+    fn_kwargs={"chars_to_ignore": chars_to_ignore, "chars_to_mapping": chars_to_mapping},
+    remove_columns=list(set(dataset.column_names) - set(['sentence', 'path']))
+)
+
+dataset = dataset.map(speech_file_to_array_fn)
+result = dataset.map(predict)
+
+wer = load_metric("wer")
+
+print("WER: {:.2f}".format(100 * wer.compute(predictions=result["predicted"], references=result["sentence"])))
+```
+]
+
+**Test Result**: 
+- WER: 33.73%
+
+
+## Training & Report
+The Common Voice `train`, `validation` datasets were used for training.
+
+You can see the training states [here](https://wandb.ai/m3hrdadfi/finetuned_wav2vec_xlsr_estonian/reports/Fine-Tuning-for-Wav2Vec2-Large-XLSR-53-Estonian--Vmlldzo1NjA1MTI?accessToken=k2b2g3a2i12m1sdwf13q8b226pplmmyw12joxo6vk38eb4djellfzmn9fp2725fw)
+
+The script used for training can be found [here](https://colab.research.google.com/github/m3hrdadfi/notebooks/blob/main/Fine_Tune_XLSR_Wav2Vec2_on_Estonian_ASR_with_%F0%9F%A4%97_Transformers_ipynb.ipynb)

all_results.json

@@ -0,0 +1,24 @@
+{
+    "epoch": 30.0,
+    "eval_loss": 0.35807397961616516,
+    "eval_mem_cpu_alloc_delta": 204106932,
+    "eval_mem_cpu_peaked_delta": 25299703,
+    "eval_mem_gpu_alloc_delta": 0,
+    "eval_mem_gpu_peaked_delta": 6155822592,
+    "eval_runtime": 345.8016,
+    "eval_samples": 2509,
+    "eval_samples_per_second": 7.256,
+    "eval_wer": 0.35535428875865743,
+    "init_mem_cpu_alloc_delta": 9477734,
+    "init_mem_cpu_peaked_delta": 18306,
+    "init_mem_gpu_alloc_delta": 1261906944,
+    "init_mem_gpu_peaked_delta": 0,
+    "total_flos": 3.401354569939134e+19,
+    "train_mem_cpu_alloc_delta": 16811478,
+    "train_mem_cpu_peaked_delta": 230270329,
+    "train_mem_gpu_alloc_delta": 3786449408,
+    "train_mem_gpu_peaked_delta": 6820308992,
+    "train_runtime": 32379.322,
+    "train_samples": 5473,
+    "train_samples_per_second": 0.091
+}

config.json

@@ -0,0 +1,76 @@
+{
+  "_name_or_path": "facebook/wav2vec2-large-xlsr-53",
+  "activation_dropout": 0.0,
+  "apply_spec_augment": true,
+  "architectures": [
+    "Wav2Vec2ForCTC"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": 1,
+  "conv_bias": true,
+  "conv_dim": [
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512
+  ],
+  "conv_kernel": [
+    10,
+    3,
+    3,
+    3,
+    3,
+    2,
+    2
+  ],
+  "conv_stride": [
+    5,
+    2,
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "ctc_loss_reduction": "mean",
+  "ctc_zero_infinity": true,
+  "do_stable_layer_norm": true,
+  "eos_token_id": 2,
+  "feat_extract_activation": "gelu",
+  "feat_extract_dropout": 0.0,
+  "feat_extract_norm": "layer",
+  "feat_proj_dropout": 0.0,
+  "final_dropout": 0.0,
+  "gradient_checkpointing": true,
+  "hidden_act": "gelu",
+  "hidden_dropout": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "layerdrop": 0.1,
+  "mask_channel_length": 10,
+  "mask_channel_min_space": 1,
+  "mask_channel_other": 0.0,
+  "mask_channel_prob": 0.0,
+  "mask_channel_selection": "static",
+  "mask_feature_length": 10,
+  "mask_feature_prob": 0.0,
+  "mask_time_length": 10,
+  "mask_time_min_space": 1,
+  "mask_time_other": 0.0,
+  "mask_time_prob": 0.05,
+  "mask_time_selection": "static",
+  "model_type": "wav2vec2",
+  "num_attention_heads": 16,
+  "num_conv_pos_embedding_groups": 16,
+  "num_conv_pos_embeddings": 128,
+  "num_feat_extract_layers": 7,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "transformers_version": "4.5.0.dev0",
+  "vocab_size": 37
+}

eval_results.json

@@ -0,0 +1,12 @@
+{
+    "epoch": 30.0,
+    "eval_loss": 0.35807397961616516,
+    "eval_mem_cpu_alloc_delta": 204106932,
+    "eval_mem_cpu_peaked_delta": 25299703,
+    "eval_mem_gpu_alloc_delta": 0,
+    "eval_mem_gpu_peaked_delta": 6155822592,
+    "eval_runtime": 345.8016,
+    "eval_samples": 2509,
+    "eval_samples_per_second": 7.256,
+    "eval_wer": 0.35535428875865743
+}

preprocessor_config.json

@@ -0,0 +1,8 @@
+{
+  "do_normalize": true,
+  "feature_size": 1,
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "return_attention_mask": true,
+  "sampling_rate": 16000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:797b6155a39ac92af7588b2fead1fccce9892a261e97d5ac5c2db9d547031b1d
+size 1262085527

result.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4a7896012604abad91cc051fc09c44ff46c874582eebd44fabc6447b12a83fe
+size 3183

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"unk_token": "<unk>", "bos_token": "<s>", "eos_token": "</s>", "pad_token": "<pad>", "do_lower_case": false, "word_delimiter_token": "|"}

train_results.json

@@ -0,0 +1,15 @@
+{
+    "epoch": 30.0,
+    "init_mem_cpu_alloc_delta": 9477734,
+    "init_mem_cpu_peaked_delta": 18306,
+    "init_mem_gpu_alloc_delta": 1261906944,
+    "init_mem_gpu_peaked_delta": 0,
+    "total_flos": 3.401354569939134e+19,
+    "train_mem_cpu_alloc_delta": 16811478,
+    "train_mem_cpu_peaked_delta": 230270329,
+    "train_mem_gpu_alloc_delta": 3786449408,
+    "train_mem_gpu_peaked_delta": 6820308992,
+    "train_runtime": 32379.322,
+    "train_samples": 5473,
+    "train_samples_per_second": 0.091
+}

trainer_state.json

@@ -0,0 +1,149 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 30.0,
+  "global_step": 2940,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 3.06,
+      "learning_rate": 0.00017999999999999998,
+      "loss": 5.039,
+      "step": 300
+    },
+    {
+      "epoch": 3.06,
+      "eval_loss": 2.9455764293670654,
+      "eval_runtime": 358.486,
+      "eval_samples_per_second": 6.999,
+      "eval_wer": 1.0,
+      "step": 300
+    },
+    {
+      "epoch": 6.12,
+      "learning_rate": 0.00028770491803278684,
+      "loss": 2.9411,
+      "step": 600
+    },
+    {
+      "epoch": 6.12,
+      "eval_loss": 2.8372132778167725,
+      "eval_runtime": 362.2267,
+      "eval_samples_per_second": 6.927,
+      "eval_wer": 1.0,
+      "step": 600
+    },
+    {
+      "epoch": 9.18,
+      "learning_rate": 0.00025081967213114756,
+      "loss": 0.9286,
+      "step": 900
+    },
+    {
+      "epoch": 9.18,
+      "eval_loss": 0.3580135107040405,
+      "eval_runtime": 372.9874,
+      "eval_samples_per_second": 6.727,
+      "eval_wer": 0.514917421417155,
+      "step": 900
+    },
+    {
+      "epoch": 12.24,
+      "learning_rate": 0.0002139344262295082,
+      "loss": 0.3039,
+      "step": 1200
+    },
+    {
+      "epoch": 12.24,
+      "eval_loss": 0.33028751611709595,
+      "eval_runtime": 377.8898,
+      "eval_samples_per_second": 6.64,
+      "eval_wer": 0.44296084176878,
+      "step": 1200
+    },
+    {
+      "epoch": 15.31,
+      "learning_rate": 0.00017704918032786883,
+      "loss": 0.2022,
+      "step": 1500
+    },
+    {
+      "epoch": 15.31,
+      "eval_loss": 0.30936819314956665,
+      "eval_runtime": 374.6431,
+      "eval_samples_per_second": 6.697,
+      "eval_wer": 0.4007059136920618,
+      "step": 1500
+    },
+    {
+      "epoch": 18.37,
+      "learning_rate": 0.00014016393442622951,
+      "loss": 0.1529,
+      "step": 1800
+    },
+    {
+      "epoch": 18.37,
+      "eval_loss": 0.3163447976112366,
+      "eval_runtime": 381.4751,
+      "eval_samples_per_second": 6.577,
+      "eval_wer": 0.38395711241342567,
+      "step": 1800
+    },
+    {
+      "epoch": 21.43,
+      "learning_rate": 0.00010327868852459015,
+      "loss": 0.1165,
+      "step": 2100
+    },
+    {
+      "epoch": 21.43,
+      "eval_loss": 0.3391939103603363,
+      "eval_runtime": 380.5289,
+      "eval_samples_per_second": 6.593,
+      "eval_wer": 0.3692061800745871,
+      "step": 2100
+    },
+    {
+      "epoch": 24.49,
+      "learning_rate": 6.639344262295081e-05,
+      "loss": 0.1058,
+      "step": 2400
+    },
+    {
+      "epoch": 24.49,
+      "eval_loss": 0.35883328318595886,
+      "eval_runtime": 376.435,
+      "eval_samples_per_second": 6.665,
+      "eval_wer": 0.3648774640383591,
+      "step": 2400
+    },
+    {
+      "epoch": 27.55,
+      "learning_rate": 2.950819672131147e-05,
+      "loss": 0.086,
+      "step": 2700
+    },
+    {
+      "epoch": 27.55,
+      "eval_loss": 0.35711735486984253,
+      "eval_runtime": 376.6441,
+      "eval_samples_per_second": 6.661,
+      "eval_wer": 0.3572522642514651,
+      "step": 2700
+    },
+    {
+      "epoch": 30.0,
+      "step": 2940,
+      "total_flos": 3.401354569939134e+19,
+      "train_runtime": 32379.322,
+      "train_samples_per_second": 0.091
+    }
+  ],
+  "max_steps": 2940,
+  "num_train_epochs": 30,
+  "total_flos": 3.401354569939134e+19,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f2d423da1472e41b342fc1cc8bc8b3c4aa0672544957d802085dbaadee8c2de
+size 2351

vocab.json

@@ -0,0 +1 @@
+{"<pad>": 0, "<s>": 1, "</s>": 2, "<unk>": 3, "|": 4, "a": 5, "b": 6, "c": 7, "d": 8, "e": 9, "f": 10, "g": 11, "h": 12, "i": 13, "j": 14, "k": 15, "l": 16, "m": 17, "n": 18, "o": 19, "p": 20, "q": 21, "r": 22, "s": 23, "t": 24, "u": 25, "v": 26, "w": 27, "x": 28, "y": 29, "z": 30, "ä": 31, "õ": 32, "ö": 33, "ü": 34, "š": 35, "ž": 36}