Labels in Audio Frame classification task (Wav2Vec2 For Audio Frame Classification)
I’m working on fine-tuning a speech model (either Wav2Vec or HuBERT) to classify speech at the frame level (e.g., every 20ms of audio must be classified). Specifically, I’m looking to use theWav2Vec2ForAudioFrameClassificationmethod, but I’m uncertain about the shape and format of the labels required.For this task, I’d like to input a torch tensor with a length corresponding to the audio length divided by the frame length (e.g., 20ms), containing binary values of 0s or 1s. Is this approach feasible? I’m having difficulty understanding the precise requirements for the labels.Does anyone have an example of usingWav2Vec2ForAudioFrameClassification, or can someone guide me on the correct shape and formatting for the labels? Any help would be appreciated, thanks a lot!