Joining SpeechEncoderDecoder embedding chunks for processing longer audio
I’m trying to find a way of processing longer audio files using SpeechEncoderDecoder. I know the CTC methods can achieve this by splitting the audio up into chunks and thenjoining the logitsbut I don’t know of any examples where a similar approach can be taken for transformer LM decoders.My initial thought is to be able to somehow join the encoder embeddings of audio chunks and then use those as theencoder_outputsvalue for the decoder generation. Initial testing hasn’t been successful, however.@anton-lyou mentioned feeding in hidden states inthat issue, could this work for theSpeechEncoderDecoderdecoder? Also, regarding the streaming inference in that issue, is this CTC only?Thanks!
2022-06-10T07:07:38Z
I’m also interested in how to do chunking for the speech encoder decoder models. Has anyone figured out how to do this?