Adapter_model.safetensors size is very big
I trained a model using the tutorial belowdatacamp.comFine-Tuning Llama 3 and Using It Locally: A Step-by-Step GuideIn this tutorial, we'll fine-tune Llama 3 on a dataset of patient-doctor conversations. After merging, converting, and quantizing the model, it will be ready for private local use via the Jan application.Here theKaggle Codekaggle.comFine-tune Llama 3 8B on Medical DatasetExplore and run machine learning code with Kaggle Notebooks | Using data from Llama 3they mentioned its generate aadapter_model.safetensorsat160+MBsize but when I run the same script its giving the2.27GB.image1354×563 80.7 KBBut when I use theunslothIt can able to produce the model in the160+MBsize. But I don’t want to use the Unsloth. Here the Unsloth Fine-tune code. (You can find in the reply)
2024-07-26T07:19:15Z
After changing the version oftrl&peftthe issue is resolved%pip install -U peft==0.11.1
%pip install -U "trl<0.9.0"Fore more referencegithub.com/huggingface/trlSignificant difference in size in lora artifacts between different versions of trl and peftopened01:37AM - 30 Jan 24 UTCclosed03:04PM - 10 Mar 24 UTCdebraj135For the same peft config and same based model previously my saved lora adapter f…iles seemed to have changed in size by orders of magnitude:

1. Previously: 79 MB `adapter_model.bin`
trl '0.7.3.dev0'
peft '0.5.0'

2. Currently: 1GB an adapter safetensors file
trl '0.7.10'
peft '0.7.1

Can anyone please help me understand the difference in behavior?
2024-07-26T12:54:01Z
Please check the below Kaggle discussion.kaggle.comFine-tune Llama 3 8B on Medical DatasetExplore and run machine learning code with Kaggle Notebooks | Using data from Llama 3When using themeta-llama/Meta-Llama-3-8B-Instructormeta-llama/Meta-Llama-3.1-8B-Instructboth are giving the2.27GBofadapter_model.safetensors.Here the package I used.%%capture
%pip install -U transformers 
%pip install -U datasets 
%pip install -U accelerate 
%pip install -U peft==0.11.1
%pip install -U "trl<0.9.0"
%pip install -U bitsandbytes 
%pip install -U wandb
2024-07-27T00:54:35Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.