Inference endpoint, gated repo 401 error
I’m using the inference endpoints website to host a fine tuned model. Initialization fails and tells me that I don’t have authorization to gated repos, yet I should have access.image955×288 22.2 KBI’ve tried mistral and llama fine tuned models. I’ve tried AWS and Google server options.image1198×1025 78.8 KBI do have an “access token” associated with my hugging face account so that I can pay for services like CPUs/GPUs, inference endpoint servers, etc.
2024-07-19T18:53:22Z
Hi@RedFoxPandaIn Inference Endpoints, you now have the ability to add an env variable to your endpoint, which is needed if you’re deploying a fine-tuned gated model like Meta-Llama-3-8B-Instruct.We have some additional documentation onenvironment variablesbut the one you’d likely need isHF_TOKEN. You can add the HF_TOKEN as the key and youruser access tokenas the value. User access tokens can be generated in thesettingsof your account.Please let me know if you have additional questions!
2024-07-24T17:36:50Z
test1244×305 22.5 KBThis did work. I then had some type of error due to not having enough memory and I upgraded to a higher level gpu/cpu for the cloud computer. “Running” status is now present. Thanks.
2024-07-24T22:03:43Z
@RedFoxPandaI’m glad to hear it! Thanks for letting me know.
2024-07-25T10:04:04Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.