Difference between pinned models and Inference endpoints
I am looking to deploy my model online and it seems like I have 2 options:Pinned modelInference EndpointWhat’s the difference between these two?
2022-11-14T16:05:48Z
Hi@popaqy, here is a very high-level overview:Pinned models (just a model preloaded for inference) are available through theInference API, but it is only supported and available to existing paying customers. Otherwise, Inference API is a free productInference Endpointis like the next iteration of pinned models, and it’ll build and deploy your model on its own secure Endpoint with cool autoscaling and security features. You can also choose your own CPU/GPU depending on your needs to keep costs low. Check this out if you need a production-ready environment for your model!
2022-11-17T07:35:21Z
Thank you a lot for the answer.I have a follow-up question:For Inference API it is said that the model is run on Intel Ice Lake CPU but the instance is not explicitly mentioned. Can you tell me which of the following instances does the Inference API use?Screenshot (38)1630×200 54.6 KB
2022-11-17T14:52:58Z
That part is entirely up to you!You can pick a smaller instance if you don’t anticipate needing a lot of compute or you can go for one of the larger instances if you need something more powerful.If you’re interested, check out thePricingdocs to learn how costs are calculated for these resources.