Unusual input_id size for distilBERT tokenizer
I am tokenizing a news article using DistilBERT tokenizer whose max token length supposed to be 512, but I a getting 280,Screenshot 2024-05-14 225447945×453 19.2 KBI dont know what is the issue, I have worked with DistilBERT before and it used to give 512 as the max length…