Cannot Setup Mixtral Models and Other Models on Inference Endpoints
Here is a link to the error logs and the discussion thread related to my issue and many others who have the same problem:huggingface.coehartford/dolphin-2.5-mixtral-8x7b · HuggingFace Inference Endpoints Issue...Hello, I saw this topic was related to one of the closed tickets, and that the user did not provide adequate information. I wanted to share my attempt to host it on the inference endpoint as I have...Even with other models, I am encountering the same issue on my account. Any ideas for possible resolutions? The Inference Endpoints as a service does not function for me right now.
2023-12-22T14:14:26Z
Having the same problem. I keep getting sharding errors like these issues:cognitivecomputations/dolphin-2.5-mixtral-8x7b · HuggingFace Inference Endpoints Issue (Detailed Information)tiiuae/falcon-7b-instruct · Trying to deploy this using Inference Endpoints