When I try to use my fine-tuned Causal LM model to inference a prompt, I get nothing but the last word repeated multiple times
As the title suggest, every time I fine-tune my model, I get no actual result, I just get the last word repeated multiple times. The model I’m fine-tuning isRoFormer, and here is an example of what is happening:>> prompt = "Somatic hypermutation allows the immune system to" 
>> generator(prompt)
[{'generated_text': 'Somatic hypermutation allows the immune system to to to to'}]Here is a bit of details:-I tried multiple datasets.I tried following the official documentation for training a Casual LM.The training, and validation losses are also really small.The base language is Chinese.What am I missing?
2024-04-14T08:54:51Z
It might be the way the model is fine-tuned (how dataset is structured, how data is processed by data collator and trainer, etc.) but repetition like the example you’ve provided also seems to be a fairly common occurrence. There exists a repetition_penalty parameter that you can try toying with:Transformers - repetition_penalty parameterOut of curiosity, is there a particular reason you’re trying to generate English text with a model whose base language is Chinese?