[RESOLVED] Recommended way to use guidance on an Inference Endpoint?
Greetings all,I would like to create an Inference Endpoint using the “guidance” features of thetext-generation-inferencecontainer.  According toGuidance, those features were introduced in text-generation-inference v1.4.3The default Inference Endpoint image (“container-type”) appears to be pulled from a private repo (registry.internal.huggingface.tech/api-inference/community/text-generation-inference:gemma-ie), and reports its version asv1.4.1-nativein its responses’system_fingerprintproperty.Attempting to specify a custom image pulled from the text-generation-inference github registry (Package text-generation-inference · GitHub) appears to be serving the wrong model (bigscience/bloom-560m, even though I specifiedTinyLlama/TinyLlama-1.1B-Chat-v1.0when creating the Inference Endpoint).Is there a generally accepted / recommended way I can serve a current text-generation-inference container (>= v1.4.3) using an Inference Endpoint?
2024-03-27T07:13:44Z
Update: The problem appears to have been on the Inference Endpoints side and now seems to have been resolved: Inference endpoints are now usingghcr.io/huggingface/text-generation-inference:1.4.4as the default image.Apologies for the noise.