How to use it in an app?
#30
by
damc
- opened
If I would like to use Starcoder model in an application, what is the best way to run it in the production environment?
Also, is there some API that I can call or do I need to host it myself?
@damc You may be interested in this project: https://github.com/bigcode-project/starcoder.cpp, which provides a demo on IOS application of StarCoder.
You can also call the API endpoint of the model /static-proxy?url=https%3A%2F%2Fapi-inference.huggingface.co%2Fmodels%2Fbigcode%2Fstarcoder%2F%3C%2Fa%3E and use text-generation-inference library like it's done in this playground
loubnabnl
changed discussion status to
closed