LlamaTokenizerFast returns token_type_ids but the forward pass of the LlamaModel does not receive token_type_ids
Screenshot 2023-06-07 at 17.12.021886×204 28.5 KBgithub.comhuggingface/transformers/blob/092c14c37d9baa6a12be5481138564a720a49a8a/src/transformers/models/llama/modeling_llama.py#L484-L495def forward(self,input_ids: torch.LongTensor = None,attention_mask: Optional[torch.Tensor] = None,position_ids: Optional[torch.LongTensor] = None,past_key_values: Optional[List[torch.FloatTensor]] = None,inputs_embeds: Optional[torch.FloatTensor] = None,use_cache: Optional[bool] = None,output_attentions: Optional[bool] = None,output_hidden_states: Optional[bool] = None,return_dict: Optional[bool] = None,) -> Union[Tuple, BaseModelOutputWithPast]:Python 3.10.11transformers==4.28.1
2023-06-09T07:37:20Z
Hey thanks for reporting! This was adressed in[Lllama] Update tokenization code to ensure parsing of the special tokens [core] by ArthurZucker · Pull Request #24042 · huggingface/transformers · GitHub!