text
Can we align code generation models to be good at chat without compromising their base capabilities 🤔?
Can we align code generation models to be good at chat without compromising their base capabilities 🤔?
new_line
new_line
text
This was the question the H4 team asked itself when BigCode released StarCoder2 a bit over a week ago. We knew that code models like 
This was the question the H4 team asked itself when BigCode released StarCoder2 a bit over a week ago. We knew that code models like 
resource
https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct
model
deepseek-ai/deepseek-coder-6.7b-instruct
https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct
text
 and 
 and 
resource
https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-33B
model
m-a-p/OpenCodeInterpreter-DS-33B
https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-33B
text
 get impressive scores on code benchmarks like HumanEval, but they tend to score poorly on chat benchmarks like MT Bench and IFEval. We also knew that the Zephyr recipe we applied to Mistral 7B produced a strong chat model, so we wondered -- could be tweaked to produce a strong coding assistant?
 get impressive scores on code benchmarks like HumanEval, but they tend to score poorly on chat benchmarks like MT Bench and IFEval. We also knew that the Zephyr recipe we applied to Mistral 7B produced a strong chat model, so we wondered -- could be tweaked to produce a strong coding assistant?
new_line
new_line
text
It turns out the answer is yes and I'm happy to share StarChat2, a DPO fine-tune of StarCoder2 15B that scores highly on both HumanEval and MT Bench / IFEval 🌟!
It turns out the answer is yes and I'm happy to share StarChat2, a DPO fine-tune of StarCoder2 15B that scores highly on both HumanEval and MT Bench / IFEval 🌟!
new_line
new_line
text
The most interesting lesson for me was that you get better models by blending in more code/math data than chat during the SFT step - in terms of tokens, we found a ratio of 3:1 worked best.
The most interesting lesson for me was that you get better models by blending in more code/math data than chat during the SFT step - in terms of tokens, we found a ratio of 3:1 worked best.
new_line
new_line
text
Anyway, here's a demo of the model, along with all the code and datasets we used to train it:
Anyway, here's a demo of the model, along with all the code and datasets we used to train it:
new_line
new_line
text
* Demo: 
* Demo: 
resource
https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground
space
HuggingFaceH4/starchat2-playground
https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground
new_line
text
* Collection: 
* Collection: 
resource
https://huggingface.co/collections/HuggingFaceH4/starchat2-15b-65f068417b330fafad751fce
collection
HuggingFaceH4/starchat2-15b-65f068417b330fafad751fce
https://huggingface.co/collections/HuggingFaceH4/starchat2-15b-65f068417b330fafad751fce
new_line
text
* Recipe: 
* Recipe: 
link
https://github.com/huggingface/alignment-handbook
https://github.com/huggingface/alignment-handbook
new_line
new_line
text
Hope it's useful to others!
Hope it's useful to others!
new_line
Can we align code generation models to be good at chat without compromising their base capabilities 🤔?

This was the question the H4 team asked itself when BigCode released StarCoder2 a bit over a week ago. We knew that code models like https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct and https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-33B get impressive scores on code benchmarks like HumanEval, but they tend to score poorly on chat benchmarks like MT Bench and IFEval. We also knew that the Zephyr recipe we applied to Mistral 7B produced a strong chat model, so we wondered -- could be tweaked to produce a strong coding assistant?

It turns out the answer is yes and I'm happy to share StarChat2, a DPO fine-tune of StarCoder2 15B that scores highly on both HumanEval and MT Bench / IFEval 🌟!

The most interesting lesson for me was that you get better models by blending in more code/math data than chat during the SFT step - in terms of tokens, we found a ratio of 3:1 worked best.

Anyway, here's a demo of the model, along with all the code and datasets we used to train it:

* Demo: https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground
* Collection: https://huggingface.co/collections/HuggingFaceH4/starchat2-15b-65f068417b330fafad751fce
* Recipe: https://github.com/huggingface/alignment-handbook

Hope it's useful to others!

❤️
lvwerra kashif osanseviero smangrul jordangong Chunte sasikiran MexIvanov vicgalle medmac01
👀
victor
🔥
medmac01
/avatars/7769fb2bdb05116cd3ea8a112f0beeeb.svg
Sasi Kiran
sasikiran
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1594651707950-noauth.jpeg
Lewis Tunstall
lewtun
user