text
Full fine-tuning of Microsoft's Phi2 on a single 4090 is now supported in axolotl. Thanks to 
Full fine-tuning of Microsoft's Phi2 on a single 4090 is now supported in axolotl. Thanks to 
mention
@abacaj
abacaj
text
 and 
 and 
mention
@vikhyatk
vikhyatk
text
 for their help with gradient checkpointing and flash attention fixes. 
 for their help with gradient checkpointing and flash attention fixes. 
new_line
new_line
text
alpaca finetune: 
alpaca finetune: 
resource
https://huggingface.co/openaccess-ai-collective/phi2-alpaca
model
openaccess-ai-collective/phi2-alpaca
https://huggingface.co/openaccess-ai-collective/phi2-alpaca
new_line
text
wandb: 
wandb: 
link
https://wandb.ai/oaaic/phi2/runs/00pc4ugb?workspace=user-wing-lian
https://wandb.ai/oaaic/phi2/runs/00pc4ugb?workspace=user-wing-lian
new_line
text
merged PR: 
merged PR: 
link
https://github.com/OpenAccess-AI-Collective/axolotl/pull/1058
https://github.com/OpenAccess-AI-Collective/axolotl/pull/1058
new_line
Full fine-tuning of Microsoft's Phi2 on a single 4090 is now supported in axolotl. Thanks to @abacaj and @vikhyatk for their help with gradient checkpointing and flash attention fixes. 

alpaca finetune: https://huggingface.co/openaccess-ai-collective/phi2-alpaca
wandb: https://wandb.ai/oaaic/phi2/runs/00pc4ugb?workspace=user-wing-lian
merged PR: https://github.com/OpenAccess-AI-Collective/axolotl/pull/1058

👍
osanseviero Citaman victor kramp Tonic marcsun13 merve ybelkada Zmu samusenps mrfakename Chunte JaiSurya radames
❤️
osanseviero Tonic merve ybelkada Chunte radames SicariusSicariiStuff sbrandeis
https://cdn-avatars.huggingface.co/v1/production/uploads/1608146735109-5fcfb7c407408029ba3577e2.png
Simon Brandeis
sbrandeis
user
https://cdn-avatars.huggingface.co/v1/production/uploads/641dfddf3bae5a77636817c5/2IwNwh9kK98eCHUmOGoWD.png
wing lian
winglian
user
https://cdn-avatars.huggingface.co/v1/production/uploads/62e54f0eae9d3f10acb95cb9/VAyk05hqB3OZWXEZW-B0q.png
mrfakename
mrfakename
user
https://cdn-avatars.huggingface.co/v1/production/uploads/6538119803519fddb4a17e10/ffJMkdx-rM7VvLTCM6ri_.jpeg
samusenps
samusenps
user
/avatars/f18351bc5ce9c106ba74523d9a55567c.svg
Lone Striker
LoneStriker
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1670003187019-noauth.png
brucethemoose
brucethemoose
user
https://cdn-avatars.huggingface.co/v1/production/uploads/6569216f9c96f1a47bf45788/mCLqmAs4dOjKdxNQVAp1w.png
Sica Rius
SicariusSicariiStuff
user