Hi guys, I started this Topic regarding fine tuning Wav2Vec2 in Portuguese language. Let’s exhange knowledge and help to implement this model at HuggingFace
2021-03-19T17:04:43Z
I’m training on Google Cloud with 2 T4 GPUs, which are quite inexpensive and usingscreento keep the code running, even if SSH disconnects for any reason. Steps to run screen properly:$ script /dev/null
$ screenENTER$ python training.pyCLOSE WINDOWTo check what part of the code is running, SSH again an run:$ screen -r3310.pts-6.training$ screen -d -r 3310To quit screen:$ screen -X -S 3310.pts-6.training quit
2021-03-19T22:23:37Z
Four hours of training on GCP, checkpoint 600:wer3895×166 11.9 KBCheckpoint 1800, batch size=8
2021-03-20T13:28:39Z
Hi@Rubens,Do you think it will be possible to do it on Colab?The dataset looks huge and becomes larger after all the caching of the processed data.
2021-03-20T20:32:27Z
Hi Gunjan, I tried to train on Colab but I had a “memory full” error. As seen in our Slack, the model can be trained on OVH Cloud:https://www.ovh.ie/fill this formOVH - Wav2Vec2 - Fine Tuning week - GPU accounts - Google Sheetsthey will send a voucher codemore in the discord channel:https://discord.gg/HaNEhBaxYou can also try with 24GB RAM and NVIDIA P100 on Google Colab, make a copy of this notebook and use it:https://colab.research.google.com/drive/1D6krVG0PPJR2Je9g5eN_2h6JP73_NUXz
2021-03-20T20:40:51Z
Current state of effort:My main modelwergot stuck at 0.3656 = 3e-4 became a high learning rate. This happened in checkpoint 2600. So, I decided to use the saved weights to run two models in parallel.First model: take original model and decrease learning rate to 0.8e-4: result so far at checkpoint 400:wer-lr-0.08913×122 21.1 KBSecond model: comment#model.freeze_feature_extractor(). Result so far at checkpoint 400 (this will take much longer to train):wer-unfreeze_start907×124 21 KBAlso, I had GPU issues so the training is not optimized.
2021-03-21T01:23:39Z
UPDATEModel 2 - comment#model.freeze_feature_extractor() aborted. GPU config issues + computing expensive.Model 1 - decreased learning rate to 0.8e-4:wer = 0.33Dataset = common voice = 1.7GB
2021-03-22T07:53:13Z
Hi RubensNice to know that you found a good set of hyperparameters.I’m not a native speaker of Portuguese, did you change the vocab dict in any manner?
2021-03-22T20:59:17Z
Git pushed the first model withwer = 20.41%for the Portuguese language.wer do 31.43907×59 7.23 KB
2021-03-23T11:33:24Z
Model still training, currently atwer = 19.30Learning rate at 0.013e-4wer 19.30904×35 2.63 KB
2021-04-16T23:20:04Z
Oi pessoal, tudo bem?Estou com dúvida de como fazer a prediçãoEstou fazendo baseado em um modelo de fine-tuning XLSR-Wav2Vec2huggingface.coFine-Tune XLSR-Wav2Vec2 for low-resource ASR with 🤗 TransformersWe’re on a journey to advance and democratize artificial intelligence through open source and open science.e não acho otimal apagar os labels em texto se depois vou usá-los para compararMas está extremamente desgastante porque eles não explicam qual o formato do input para a função compute_metrics só falam que é pred…então fica um black blox dentro de outro black box…não sei dar o forward e não tive um contato significativo com pytorch, ai quando vou tentar fazer predição por batch, pronto…só dá erro…vou acabar tendo que criar uma função que repete o que outra função faz porque não sei o que é preds…