Loading and using Autotrain model error
I trained llama-7b-chat-hf using the following command as I saw in the videoautotrain llm --train --project_name peft-tune-llama-credit-card-fraud-v2 --model meta-llama/Llama-2-7b-chat-hf --data_path . --use_peft --use_int4 --learning_rate 2e-4 --train_batch_size 6 --num_train_epochs 10 --trainer sftOn the same Ubunutu EC2 instance that I used to train the model, I am trying to load the model. The model is not pushed to the hub. I am running into issues as follows:import torchimport transformersfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer, LlamaTokenizer, StoppingCriteria, StoppingCriteriaList, TextIteratorStreamermodel=‘/home/ubuntu/LLM/meta-llama/peft-tune-llama-2-7b-chat-hf-credit-card-fraud-v2’model_name = “meta-llama/Llama-2-7b-chat-hf”adapters_name = {‘privacy/repo_name’: ‘/home/ubuntu/LLM/meta-llama/peft-tune-llama-2-7b-chat-hf-credit-card-fraud-v2’}print(f"Starting to load the model {model} into memory")m = AutoModelForCausalLM.from_pretrained(model_name,load_in_4bit=True,torch_dtype=torch.bfloat16,device_map={“”: 0})m = PeftModel.from_pretrained(m, model)m = m.merge_and_unload()tok = LlamaTokenizer.from_pretrained(model)tok.bos_token_id = 1stop_token_ids = [0]However, getting an exception as followsTraceback (most recent call last):File “/home/ubuntu/LLM/meta-llama/model_loader.py”, line 19, inm = PeftModel.from_pretrained(m, model)File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/peft/peft_model.py”, line 278, in from_pretrainedmodel.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs)File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/peft/peft_model.py”, line 554, in load_adapteradapters_weights = load_peft_weights(model_id, device=torch_device, **REDACTED_TOKEN_download_kwargs)File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/peft/utils/save_and_load.py”, line 171, in load_peft_weightshas_remote_safetensors_file = hub_file_exists(File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/peft/utils/hub_utils.py”, line 24, in hub_file_existsurl = REDACTED_TOKEN_url(repo_id=repo_id, filename=filename, repo_type=repo_type, revision=revision)File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py”, line 110, in _inner_fnvalidate_repo_id(arg_value)File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py”, line 158, in validate_repo_idraise HFValidationError(Question:How do I fix this error, load the model and use it for a query?IF this is all wrong, can you provide a sample code to load and use the local saved model, without going to hub?