Text generation models are producing incomplete response although max_new_tokens set to a high value
I am using langchain.model = HuggingFaceHub(repo_id=“mistralai/Mistral-7B-Instruct-v0.2”,task=“text-generation”,model_kwargs={“max_length”: 1024,“max_new_tokens”: 512,“top_k”: 30,“temperature”: 0.1,“repetition_penalty”: 1.03,},)It’s generating incomplete response or generation is stopped in the middle way.
2024-08-02T12:40:21Z
did you find solution?