Batching in "automatic-speech-recognition" pipelines
I am using the following code to send a batch of inputs to the automatic-speech-recognition pipeline:from transformers import pipeline
from datasets import load_dataset
import numpy as np

ds = load_dataset(
    "hf-internal-testing/librispeech_asr_demo",
    "clean",
    split="validation")

input_data = ds[0]["audio"]["array"]
batch_test = np.vstack((input_data, input_data)) 
for i in range(5):
       batch_test = np.vstack((batch_test, input_data))

task = "automatic-speech-recognition"
model_name = 'facebook/s2t-small-librispeech-asr'
batch_size = 5

model  = pipeline(
    task=task,
    model=model_name,
    batch_size=batch_size)

res = model(batch_test)
resHowever I am recieving the following error which seems that the huggingface model is not able to accepting stacked audio inputs and treats them as multi-channels outputs:ValueError: We expect a single channel audio input for AutomaticSpeechRecognitionPipelineLooking at the huggingface code it seems that following line is returning the mentioned error. I couldn’t find anything related to preprocessing batched intput in the code, how I can I enable batching for input to the huggingface models.github.comhuggingface/transformers/blob/5cd16f01db3b5499d4665e8624801ed30ba87bdd/src/transformers/pipelines/automatic_speech_recognition.py#L248if stride[0] + stride[1] > inputs.shape[0]:raise ValueError("Stride is too large for input")# Stride needs to get the chunk length here, it's going to get# swallowed by the `feature_extractor` later, and then batching# can add extra data in the inputs, so we need to keep track# of the original length in the stride so we can cut properly.stride = (inputs.shape[0], int(round(stride[0] * ratio)), int(round(stride[1] * ratio)))if not isinstance(inputs, np.ndarray):raise ValueError(f"We expect a numpy ndarray as input, got `{type(inputs)}`")if len(inputs.shape) != 1:raise ValueError("We expect a single channel audio input for AutomaticSpeechRecognitionPipeline")if chunk_length_s:if stride_length_s is None:stride_length_s = chunk_length_s / 6if isinstance(stride_length_s, (int, float)):stride_length_s = [stride_length_s, stride_length_s]# XXX: Carefuly, this variable will not exist in `seq2seq` setting.
2024-04-19T19:30:21Z
I know this is an old post, but for future readers the solution to this problem is to pass a python list of numpy arrays (instead of a single batched numpy array) to the pipelinemodel:model([waveform.cpu().numpy() for waveform in batched_waveforms])
2024-05-23T09:02:34Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.