Pipeline max_length
What does the max_length argument in the pipeline function do?pipe = pipeline('text2text-generation', model=self.model, tokenizer=self.tokenizer)
    output = []

    for ele in dataset:
        pred = pipe(ele, max_length=1024)
        output.append({'input' : ele ,'output': pred[0]['generated_text']})
2024-02-23T13:09:36Z
I think thats the max length in tokens the model can accept, could be wrong though.
2024-02-23T20:30:04Z
Hi! Themax_lengthhere controls for maximum tokens that can be generated. The generation stops when we reach the maximum. Note that the model might generate incomplete sentences, if you specifymax_lengthtoo short, by default it is 20 tokens.