Regarding input_ids and labels while grouping texts
Hi@regisssI see input_ids and labels to be of same tensorshereIs it a mistake? Or internally in GaudiTrainer the shifting of labels happens?
2024-04-23T20:11:53Z
Hi@palashai, it happens in the forward of the model. For instance for Llama:optimum-habana/optimum/habana/transformers/models/llama/modeling_llama.py at 8c8ec55d8030f1df05288407b2e687a7664d7395 · huggingface/optimum-habana · GitHub
2024-04-24T05:38:01Z
Got it. Thanks.
2024-04-24T17:38:12Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.