Wav2vec2 model tuning on Mac Silicon
I am having difficulties fine tuning the wav2vec2 model on my Mac Silicon (M1) processor. When downstream training on the Commonvoice dataset, I encounter the Jupyter notebook error:NotImplementedError: The operator ‘aten::_weight_norm_interface’ is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment onGeneral MPS op coverage tracking issue · Issue #77764 · pytorch/pytorch · GitHub. As a temporary fix, you can set the environment variablePYTORCH_ENABLE_MPS_FALLBACK=1to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.The error suggests creating the `PYTORCH_ENABLE_MPS_FALLBACK=1’ environment variable which I have done in the terminal. I have restarted the kernel and confirmed the environment variable is present and set in the notebook by running %env. I have now hit a brick wall with this. Any solutions to this?
2023-11-04T14:04:37Z
With some kernel rebuilding, and setting the environment variable: :`PYTORCH_ENABLE_MPS_FALLBACK=1’ in the virtual environment terminal shell, I am now able to train the model. CPU speeds look to be a limiting factor however!
2023-12-27T14:13:33Z
How did you do the “kernel rebuilding”?