Rope Factor issues with meta-llama/Meta-Llama-3.1-70B
with this code in Colabs :# Load modelmodel = AutoModelForCausalLM.from_pretrained(model_name,torch_dtype=torch.float16,device_map=“auto”,use_auth_token=True,rope_scaling={“type”: “dynamic”, “factor”: 8.0}  # Ensure correct parameter format)I am getting this error message  :ValueError:rope_scalingmust be a dictionary with two fields,typeandfactor, got {‘factor’: 8.0, ‘low_freq_factor’: 1.0, ‘high_freq_factor’: 4.0, ‘original_max_position_embeddings’: 8192, ‘rope_type’: ‘llama3’}
2024-08-30T10:50:28Z
hi@kgbearderHave you seenmeta-llama/Meta-Llama-3.1-8B-Instruct · ValueError: `rope_scaling` must be a dictionary with two fields?
2024-08-30T10:53:02Z
kgbearder:ValueError:rope_scalingmust be a dictionary with two fields,typeandfactorA simple Google search gives the answer:ValueError: `rope_scaling` must be a dictionary with two fields, `type` and `factor` · Issue #299 · meta-llama/llama3 · GitHub
2024-08-31T17:00:34Z
@nielsr@mahmutcthanks for the guidance  . was resolved  by  :pip install --upgrade transformersand then using :rope_scaling={“type”: “llama3”, “factor”: 8.0}