How can I use diverse beam-search? (it isn't working in my code)
Hello, I want to get several promising generative candidates using thediverse beam-search decoding.But when I execute the code as follows, they all produce the same sentence.sample_outputs = self.model.generate(
      input_ids = input_ids,
      max_length = args.max_input_length,
      num_beams = 6,
      num_beam_groups = 3,
      num_return_sequences = 3,
      pad_token_id = self.tokenizer.pad_token_id,
      eos_token_id = self.tokenizer.eos_token_id,
)Ifnum_return_sequencesis set to be the same asnum_beam_groups, won’t each group’s sentence be printed one by one?However, allnum_return_sequencessentences returned are the same.I checked that deletingnum_beam_groupsreturns different sentence candidates well.I would appreciate it if you could tell me which variable I should add to use the diverse beam-search decoding.Thanks.
2023-02-26T22:43:35Z
Not sure if you’ve already figured this out, since this is somewhat of an old post, but I recently ran into this problem and fixed it by  adding thediversity_penaltyargument in.generate().This is what controls the discouragement of similar outputs in each group in the logits processor that is used within the generate code.Seethis documentation.