text
🦙🦙 LLaMA Duo project update 
🦙🦙 LLaMA Duo project update 
new_line
new_line
text
Last time, I gave a brief introduction about LLaMA Duo project with 
Last time, I gave a brief introduction about LLaMA Duo project with 
mention
@sayakpaul
sayakpaul
text
 . It is a simple toolset to aligning sLLM with service LLM with coverage dataset 👉🏻 (
 . It is a simple toolset to aligning sLLM with service LLM with coverage dataset 👉🏻 (
link
https://huggingface.co/posts/chansung/708646454991943
https://huggingface.co/posts/chansung/708646454991943
text
).
).
new_line
text
- coverage dataset is what we believe to be the most important/desired (instruction, response) pairs. In system thinking, each instruction could be an analogy of a function from traditional programming. We make unit tests and measure the coverage % for all the features/functions. Similarly, we need to ensure if our fine-tuned model could handle what % of given instructions from coverage dataset satisfactory (hence coverage dataset).
- coverage dataset is what we believe to be the most important/desired (instruction, response) pairs. In system thinking, each instruction could be an analogy of a function from traditional programming. We make unit tests and measure the coverage % for all the features/functions. Similarly, we need to ensure if our fine-tuned model could handle what % of given instructions from coverage dataset satisfactory (hence coverage dataset).
new_line
new_line
text
We have tested it with "Coding" category of data from 
We have tested it with "Coding" category of data from 
resource
https://huggingface.co/datasets/HuggingFaceH4/no_robots
dataset
HuggingFaceH4/no_robots
https://huggingface.co/datasets/HuggingFaceH4/no_robots
text
 dataset. It has about 300 SFT training data points under Coding category. After fine-tuning Gemma 7B model on that, the result was very poor. LLaMA Duo's evaluation tool gave < 20% of metrics in similarity and preciseness on the test split. 
 dataset. It has about 300 SFT training data points under Coding category. After fine-tuning Gemma 7B model on that, the result was very poor. LLaMA Duo's evaluation tool gave < 20% of metrics in similarity and preciseness on the test split. 
new_line
new_line
text
So, we used LLaMA Duo's synthetic data generation tool to generate 60k data points that looks similar to the original dataset. We first created ~10k synthetic data points, then created 50k more based on the synthetic dataset itself. 
So, we used LLaMA Duo's synthetic data generation tool to generate 60k data points that looks similar to the original dataset. We first created ~10k synthetic data points, then created 50k more based on the synthetic dataset itself. 
new_line
new_line
text
After fine-tuning Gemma 7B on the 60k synthetic dataset, the evaluation result went up to 80~90% high. Also, when testing out the model in UI, it tends to give good responses. 
After fine-tuning Gemma 7B on the 60k synthetic dataset, the evaluation result went up to 80~90% high. Also, when testing out the model in UI, it tends to give good responses. 
new_line
new_line
text
It is a good showcase to transition from service LLM to sLLM or having a backup sLLM for service LLM failure scenarios. I am going to expand this experiments on all categories of no_robots dataset. It will roughly generate > 100k data points. 
It is a good showcase to transition from service LLM to sLLM or having a backup sLLM for service LLM failure scenarios. I am going to expand this experiments on all categories of no_robots dataset. It will roughly generate > 100k data points. 
new_line
new_line
text
Here are some links:
Here are some links:
new_line
text
- LLaMA Duo project repo: 
- LLaMA Duo project repo: 
link
https://github.com/deep-diver/llamaduo
https://github.com/deep-diver/llamaduo
new_line
text
- 60k Coding synthetic dataset: 
- 60k Coding synthetic dataset: 
resource
https://huggingface.co/datasets/chansung/merged_ds_coding
dataset
chansung/merged_ds_coding
https://huggingface.co/datasets/chansung/merged_ds_coding
new_line
text
- Fine-tuned Gemma 7B model: 
- Fine-tuned Gemma 7B model: 
resource
https://huggingface.co/chansung/coding_llamaduo_60k_v0.2
model
chansung/coding_llamaduo_60k_v0.2
https://huggingface.co/chansung/coding_llamaduo_60k_v0.2
🦙🦙 LLaMA Duo project update 

Last time, I gave a brief introduction about LLaMA Duo project with @sayakpaul . It is a simple toolset to aligning sLLM with service LLM with coverage dataset 👉🏻 (https://huggingface.co/posts/chansung/708646454991943).
- coverage dataset is what we believe to be the most important/desired (instruction, response) pairs. In system thinking, each instruction could be an analogy of a function from traditional programming. We make unit tests and measure the coverage % for all the features/functions. Similarly, we need to ensure if our fine-tuned model could handle what % of given instructions from coverage dataset satisfactory (hence coverage dataset).

We have tested it with "Coding" category of data from https://huggingface.co/datasets/HuggingFaceH4/no_robots dataset. It has about 300 SFT training data points under Coding category. After fine-tuning Gemma 7B model on that, the result was very poor. LLaMA Duo's evaluation tool gave < 20% of metrics in similarity and preciseness on the test split. 

So, we used LLaMA Duo's synthetic data generation tool to generate 60k data points that looks similar to the original dataset. We first created ~10k synthetic data points, then created 50k more based on the synthetic dataset itself. 

After fine-tuning Gemma 7B on the 60k synthetic dataset, the evaluation result went up to 80~90% high. Also, when testing out the model in UI, it tends to give good responses. 

It is a good showcase to transition from service LLM to sLLM or having a backup sLLM for service LLM failure scenarios. I am going to expand this experiments on all categories of no_robots dataset. It will roughly generate > 100k data points. 

Here are some links:
- LLaMA Duo project repo: https://github.com/deep-diver/llamaduo
- 60k Coding synthetic dataset: https://huggingface.co/datasets/chansung/merged_ds_coding
- Fine-tuned Gemma 7B model: https://huggingface.co/chansung/coding_llamaduo_60k_v0.2
👍
Tonic peterschmidt85 chansung lunarflu juyongjiang
🔥
hllj chansung lunarflu Hiraishin juyongjiang
➕
cyb3rh4v0k juyongjiang
🚀
cyb3rh4v0k juyongjiang