Finetuning of conversational model without train data in conversation style
Hi all,I want to finetune a Language Model to obtain a chatbot that imitates a certain person in a conversation. The problem is, I only have quotes of that person as train data, no conversation format. So I guess finetuning a generative model that doesnt know how to answer to question resp. participate in a conversation is no option. What about finetuning a conversational model with train samples in a style like this: you are a chatbot that replies in the style of Donald Duck - “some quote”.I have no idea if it can work and the model will combine its conversational understanding with its donald duck style knowledge, if I enter a respective prompt.Any ideas, experiences or recommendations?Thank you!
2024-02-02T20:55:18Z
I think it depends on how much data you have.It also depends on which model you use. In simple terms, causal language modeling models try to complete documents, so with a certain prompt structure you could trick the model into a kind of conversational document, which the model then wants to complete.A model that is already fine-tuned towards chat will probably manage this task better.Maybe prompt engineering with one shot prompts or more shot prompts could help.You could create a chat template and apply it to the tokenizer.messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a donald duck",
    },
    {
		"role": "user",
		"content": "Some Question"
	},
	{
		"role": "assistant",
		"content": "Quack Quack..." # Donald Duck style answer
	},
	{
		"role": "user",
		f"content": "{question}"
	},
 ]This generates prompts like this:<|system|>
You are a friendly chatbot who always responds in the style of a donald duck</s> 
<|user|>
Some Question</s> 
<|assistant|>
Quack Quack...</s> # Donald Duck style answer
<|user|>
Your dynamically inserted question </s> 
<|assistant|>
Generated answer..prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")For further information:chat templatesI think this could work if the model is supposed to imitate a well-known person from whom there was actually data in the training set.Alternatively, like you mentioned, you could try to create a data set with this prompt template and fine tune it in that direction.Unfortunately no guarantee that it will actually work.