Inference Llama-2-13b not working
Following theblog, I was able to run inference on my gpu server formeta-llama/Llama-2-7b-chat-hfas has been done in the tutorial. (Have been granted access to Llama-2 already)Now I wish to run inference onmeta-llama/Llama-2-13b. I keep getting the error:“OSError: meta-llama/Llama-2-13b does not appear to have a file named config.json. Checkout ‘https://huggingface.co/meta-llama/Llama-2-13b/main’ for available files.”Checking the mentioned repo forconfig.json, I see there actually isn’t one formeta-llama/Llama-2-13b(and most other Llama2 models exceptmeta-llama/Llama-2-7b-chat-hf).Could the missing file(s) it be added please? Alternatively, if there’s another way to do a run the model privately, it would be great. Thanks!
2023-07-19T19:16:37Z
I was able to load llama-2-13b-chat-hf with this Google Colab notebook, but inference failed for me as well due to a RuntimeError. I’m not sure if it’s entirely related to your problem, but maybe you’d like to take a look?colab.research.google.comGoogle Colaboratory
2023-07-20T16:13:18Z
Got the model working. Just need to use the models with ‘hf’ in the name.
2023-07-21T21:36:02Z
Just to make it more clear, this runs perfectly fine:meta-llama/Llama-2-13b-hf
2023-07-30T20:47:05Z
But it says that PRO license is required to use hf ones