How to make transformer (T5) for translation return n translation inferences?
So I am trying to use this transformer from huggingfaceTranslation. The issue is that I want n translations returned and not just one. How can I  do that? I mean, I want to have ordered translations, that means the translation with index 0 would have the highest confidence, this is important for my use case, which is about translating natural language to commands language (about 40 commands without subcommands).The github repo and exact model is this onetext-to-text-transfer-transformer/t5/models/REDACTED_TOKEN.py at main · google-research/text-to-text-transfer-transformer · GitHubThis is the HuggingFace API:translator = pipeline("translation_xx_to_yy", model="my_awesome_opus_books_model")
translator(text)But I am intending to use the model directly from the google search github repo, so it seems some tweaking should be done here:predictions = []
    for batch in dataset:
      predicted_tokens = self._model.generate(
          input_ids=self.to_tensor(batch["inputs"]), **generate_kwargs
      )
      predicted_tokens = predicted_tokens.cpu().numpy().tolist()
      predictions.extend(
          [vocabs["targets"].decode(p) for p in predicted_tokens]
      )

    for inp, pred in zip(inputs, predictions):
      logging.info("%s\n  -> %s", inp, pred)

    if output_file is not None:
      utils.write_lines_to_file(predictions, output_file)Thanks in advance! Also any suggestion on some other model option to solve this natural language to cmd is welcomed!
2024-09-02T00:45:34Z
There’s thenum_beamsparam you could check out that looks for multiple possible translations. This considers multiple top hypotheses.Then thenum_return_sequencesinmodel.generatewill generate multiple possible sequences which you could then decode.
2024-09-02T07:47:27Z
raghavm1:num_return_sequencesYes, but thatnormalize_logits=Trueincompute_transition_scoresshould be the case, because I don’t want the same input length for sure.