

[LEDForConditionalGeneration] is an extension of
  [BartForConditionalGeneration] exchanging the traditional self-attention layer with
  Longformer's chunked self-attention layer. [LEDTokenizer] is an alias of
  [BartTokenizer].
LED works very well on long-range sequence-to-sequence tasks where the input_ids largely exceed a length of
  1024 tokens.
LED pads the input_ids to be a multiple of config.attention_window if required. Therefore a small speed-up is
  gained, when [LEDTokenizer] is used with the pad_to_multiple_of argument.
LED makes use of global attention by means of the global_attention_mask (see
  [LongformerModel]). For summarization, it is advised to put global attention only on the first
  <s> token. For question answering, it is advised to put global attention on all tokens of the question.
To fine-tune LED on all 16384, gradient checkpointing can be enabled in case training leads to out-of-memory (OOM)
  errors. This can be done by executing model.gradient_checkpointing_enable(). 
 Moreover, the use_cache=False
  flag can be used to disable the caching mechanism to save memory.
LED is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than
  the left.