

Here we have the scores since we passed along output_scores=True, but we don't have hidden_states and
attentions because we didn't pass output_hidden_states=True or output_attentions=True.
You can access each attribute as you would usually do, and if that attribute has not been returned by the model, you
will get None. Here for instance generation_output.scores are all the generated prediction scores of the
language modeling head, and generation_output.attentions is None.
When using our generation_output object as a tuple, it only keeps the attributes that don't have None values.
Here, for instance, it has two elements, loss then logits, so
python
generation_output[:2]
will return the tuple (generation_output.sequences, generation_output.scores) for instance.
When using our generation_output object as a dictionary, it only keeps the attributes that don't have None
values. Here, for instance, it has two keys that are sequences and scores.
We document here all output types.
PyTorch
[[autodoc]] generation.GenerateDecoderOnlyOutput
[[autodoc]] generation.GenerateEncoderDecoderOutput
[[autodoc]] generation.GenerateBeamDecoderOnlyOutput
[[autodoc]] generation.GenerateBeamEncoderDecoderOutput
TensorFlow
[[autodoc]] generation.TFGreedySearchEncoderDecoderOutput
[[autodoc]] generation.TFGreedySearchDecoderOnlyOutput
[[autodoc]] generation.TFSampleEncoderDecoderOutput
[[autodoc]] generation.TFSampleDecoderOnlyOutput
[[autodoc]] generation.TFBeamSearchEncoderDecoderOutput
[[autodoc]] generation.TFBeamSearchDecoderOnlyOutput
[[autodoc]] generation.TFBeamSampleEncoderDecoderOutput
[[autodoc]] generation.TFBeamSampleDecoderOnlyOutput
[[autodoc]] generation.TFContrastiveSearchEncoderDecoderOutput
[[autodoc]] generation.TFContrastiveSearchDecoderOnlyOutput
FLAX
[[autodoc]] generation.FlaxSampleOutput
[[autodoc]] generation.FlaxGreedySearchOutput
[[autodoc]] generation.FlaxBeamSearchOutput
LogitsProcessor
A [LogitsProcessor] can be used to modify the prediction scores of a language model head for
generation.
PyTorch
[[autodoc]] AlternatingCodebooksLogitsProcessor
    - call
[[autodoc]] ClassifierFreeGuidanceLogitsProcessor
    - call
[[autodoc]] EncoderNoRepeatNGramLogitsProcessor
    - call
[[autodoc]] EncoderRepetitionPenaltyLogitsProcessor
    - call
[[autodoc]] EpsilonLogitsWarper
    - call
[[autodoc]] EtaLogitsWarper
    - call
[[autodoc]] ExponentialDecayLengthPenalty
    - call
[[autodoc]] ForcedBOSTokenLogitsProcessor
    - call
[[autodoc]] ForcedEOSTokenLogitsProcessor
    - call
[[autodoc]] ForceTokensLogitsProcessor
    - call
[[autodoc]] HammingDiversityLogitsProcessor
    - call
[[autodoc]] InfNanRemoveLogitsProcessor
    - call
[[autodoc]] LogitNormalization
    - call
[[autodoc]] LogitsProcessor
    - call
[[autodoc]] LogitsProcessorList
    - call
[[autodoc]] LogitsWarper
    - call
[[autodoc]] MinLengthLogitsProcessor
    - call
[[autodoc]] MinNewTokensLengthLogitsProcessor
    - call
[[autodoc]] NoBadWordsLogitsProcessor
    - call
[[autodoc]] NoRepeatNGramLogitsProcessor
    - call
[[autodoc]] PrefixConstrainedLogitsProcessor
    - call
[[autodoc]] RepetitionPenaltyLogitsProcessor
    - call
[[autodoc]] SequenceBiasLogitsProcessor
    - call
[[autodoc]] SuppressTokensAtBeginLogitsProcessor
    - call
[[autodoc]] SuppressTokensLogitsProcessor
    - call
[[autodoc]] TemperatureLogitsWarper
    - call
[[autodoc]] TopKLogitsWarper
    - call
[[autodoc]] TopPLogitsWarper
    - call
[[autodoc]] TypicalLogitsWarper
    - call
[[autodoc]] UnbatchedClassifierFreeGuidanceLogitsProcessor
    - call
[[autodoc]] WhisperTimeStampLogitsProcessor
    - call
TensorFlow
[[autodoc]] TFForcedBOSTokenLogitsProcessor
    - call
[[autodoc]] TFForcedEOSTokenLogitsProcessor
    - call
[[autodoc]] TFForceTokensLogitsProcessor
    - call
[[autodoc]] TFLogitsProcessor
    - call
[[autodoc]] TFLogitsProcessorList
    - call
[[autodoc]] TFLogitsWarper
    - call
[[autodoc]] TFMinLengthLogitsProcessor
    - call
[[autodoc]] TFNoBadWordsLogitsProcessor
    - call
[[autodoc]] TFNoRepeatNGramLogitsProcessor
    - call
[[autodoc]] TFRepetitionPenaltyLogitsProcessor
    - call
[[autodoc]] TFSuppressTokensAtBeginLogitsProcessor
    - call
[[autodoc]] TFSuppressTokensLogitsProcessor
    - call
[[autodoc]] TFTemperatureLogitsWarper
    - call
[[autodoc]] TFTopKLogitsWarper
    - call
[[autodoc]] TFTopPLogitsWarper
    - call
FLAX
[[autodoc]] FlaxForcedBOSTokenLogitsProcessor
    - call
[[autodoc]] FlaxForcedEOSTokenLogitsProcessor
    - call
[[autodoc]] FlaxForceTokensLogitsProcessor
    - call
[[autodoc]] FlaxLogitsProcessor
    - call
[[autodoc]] FlaxLogitsProcessorList
    - call
[[autodoc]] FlaxLogitsWarper
    - call
[[autodoc]] FlaxMinLengthLogitsProcessor
    - call
[[autodoc]] FlaxSuppressTokensAtBeginLogitsProcessor
    - call
[[autodoc]] FlaxSuppressTokensLogitsProcessor
    - call
[[autodoc]] FlaxTemperatureLogitsWarper
    - call
[[autodoc]] FlaxTopKLogitsWarper
    - call
[[autodoc]] FlaxTopPLogitsWarper
    - call
[[autodoc]] FlaxWhisperTimeStampLogitsProcessor
    - call
StoppingCriteria
A [StoppingCriteria] can be used to change when to stop generation (other than EOS token). Please note that this is exclusively available to our PyTorch implementations.
[[autodoc]] StoppingCriteria
    - call
[[autodoc]] StoppingCriteriaList
    - call
[[autodoc]] MaxLengthCriteria
    - call
[[autodoc]] MaxTimeCriteria
    - call
Constraints
A [Constraint] can be used to force the generation to include specific tokens or sequences in the output. Please note that this is exclusively available to our PyTorch implementations.
[[autodoc]] Constraint
[[autodoc]] PhrasalConstraint
[[autodoc]] DisjunctiveConstraint
[[autodoc]] ConstraintListState
BeamSearch
[[autodoc]] BeamScorer
    - process
    - finalize
[[autodoc]] BeamSearchScorer
    - process
    - finalize
[[autodoc]] ConstrainedBeamSearchScorer
    - process
    - finalize
Utilities
[[autodoc]] top_k_top_p_filtering
[[autodoc]] tf_top_k_top_p_filtering
Streamers
[[autodoc]] TextStreamer
[[autodoc]] TextIteratorStreamer
Caches
[[autodoc]] Cache
    - update
[[autodoc]] DynamicCache
    - update
    - get_seq_length
    - reorder_cache
    - to_legacy_cache
    - from_legacy_cache
[[autodoc]] SinkCache
    - update
    - get_seq_length
    - reorder_cache
[[autodoc]] StaticCache
    - update
    - get_seq_length