Length_penalty not influencing results (Bart, Pegasus)
Hello,I am experimenting with the generative parameters of the two models Bart and Pegasus. In particular, I am having trouble with thelength_penaltyparameter, since changing it does not change the output of the model.I am summarizing two different chapters of a book (# tokens around 1k) and this is the code I am using:model.generate(
    b0ch1sec1_text_enc,
    min_length = 150,
    max_length = 350,
    num_beams = 2,
    length_penalty = lp,
    early_stopping = True)[0]Withlpvarying from 0.1 to 2 and model being eitherbart-large-cnnorpegasus-large.Do you have any idea why the output does not change at all?
2020-12-08T10:40:38Z
For those who were following this post, I tried in a more rigorous way with some (around 10) articles from the CNN/DM and thelength_penaltyparameter does change the output (actually a lot, from 200 to 500 tokens). However, it is still a mystery for me why sometimes it does not influence it at all.The code I have used:import torch
torch.manual_seed(42)

summ = {}
model.to('cuda')
for i, a in enumerate(articles):
    summ[i] = []
    for lp in [0.1, 1, 2]:
        summ[i].append(
            tokenizer.decode(
                model.generate(
                    tokenizer.encode(a, truncation=True, return_tensors='pt').to('cuda'),
                    length_penalty=lp)[0],
                skip_special_tokens=True))