Question about Wav2vec2
hey@sguggerI wanted to know what’s up with Wav2vec2facebook/wav2vec2-base-960h · Hugging Faceandfacebook/wav2vec2-base · Hugging FaceI was looking into the model files and saw that a padding value had been specified but when I use the preprocessor it seems like it totally ignores the instructions in the preprocessor config and passing values to the processor call does not help either. I want to know where does the processor get the padding value if not from the processor configMy code :from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
 from datasets import load_dataset
 import soundfile as sf
 import torch
 
# load model and tokenizer
processor = Wav2Vec2Processor.from_pretrained("facebook/wav2vec2-base")

input_values3 =  processor(x, return_tensors="pt", padding="longest").input_values  # Batch size 1

input_values3Also, another question does each Wav2vec2 model have its own way of normalizing/processing audio because I passed the same audio tojonatasgrosman/wav2vec2-large-xlsr-53-chinese-zh-cn · Hugging Faceand it gave me different results,my code :from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
 from datasets import load_dataset
 import soundfile as sf
 import torch
 
# load model and tokenizer
processor = Wav2Vec2Processor.from_pretrained("jonatasgrosman/wav2vec2-large-xlsr-53-chinese-zh-cn")

input_values =  processor(x, return_tensors="pt", padding="longest").input_values  # Batch size 1

input_values
2022-05-06T21:48:19Z
I was playing around with Wav2vec2 a bit more and there were a few more things I noticed that were off, firstly in the Wav2vec2Procesor  whenever we pad, we then apply the normalization which in turn changes the padded value per sequence. Would it not make more sense to let the padded value be as is and apply the normalization before the padding? Also, Some parameters likepadded_valueanddo_normalizedo not work either. I tried sayingdo_normalize = Falsebut it still normalized. In the same case withpadded_value, I tried passing a few different values but the output from the processor did not change one bit. I think this is because we do not pass the padded_value to the pad functionimage1403×750 74.4 KBShouldn’t we pass the other parameters here as well?Let me know where I am wrong, I am not very experienced with ML/DL models yet so all of this might be just my misunderstanding