Claritifcation about the `max_position_embeddings` argument
Referring to this Colab Notebook:colab.research.google.comGoogle ColaboratoryWhile initializing theRobertaConfig:config = RobertaConfig(
    vocab_size=52_000,
    max_position_embeddings=514,
    num_attention_heads=12,
    num_hidden_layers=6,
    type_vocab_size=1,
)Why do we setmax_position_embeddingsto 514 when the maximum sequence length is set to 512 in the notebook?
2023-01-27T08:45:16Z
Hey this may be late, but the other two are sep_tokens at the start and end of the tensor list.