the model(student) after finetuning outperforms [410m-deduped](https://huggingface.co/EleutherAI/pythia-410m-deduped) model on wsc accuracy.

lm-eval : aloobun/distill-p-test
![image/png](https://huggingface.co/aloobun/distill-p-test/blob/main/Screenshot%20from%202023-12-07%2019-29-32.png)

lm-eval : EleutherAI/pythia-410m-deduped
![image/png](https://huggingface.co/aloobun/distill-p-test/blob/main/Screenshot%20from%202023-12-07%2019-48-36.png)
{"tags": ["Distillation"]}
transformers safetensors gpt_neox text-generation Distillation autotrain_compatible endpoints_compatible text-generation-inference region:us