Tiny whisper finetuning for french speech recognition
I am attempting to fine-tune ‘whisper_tiny’ on a French dataset. Initially, the Word Error Rate (WER) metric and the evaluation loss show a decrease. However, after a certain point, both metrics start to increase. I attempted to enhance the quality of transcriptions by removing special characters. This approach was taken due to the nature of the French language, which is characterized by a wide variety of characters. Unfortunately, despite these efforts, no noticeable improvements have been observed. I am uncertain about the reasons behind this outcome.this is the link of the notebook :notebook linkthis are training and evaluation loss and wer values during training:check comments.
2023-08-15T13:49:33Z
W&amp;B Chart 8_15_2023, 2_24_03 PM1880×1328 115 KB
2023-08-15T13:50:06Z
W&amp;B Chart 8_15_2023, 2_24_51 PM1656×1328 95.6 KB
2024-09-17T14:34:48Z
I’m very late to the party, but eager to know why this may happen, or if you found a solution. I’m currently looking for ways to fine-tune whisper models for french too.