Text generation. Stream output
Hi, I want to use text generation and stream the output similar to ChatGPT. How to do that?
2023-03-07T14:40:57Z
I made a streaming generation service for Hugging Facetransformersthat is fully compatible with the OpenAI API:https://github.com/hyperonym/basaranHope it meets your needs.
2023-04-04T08:05:10Z
I checked out basaran lasted week and its great! Loved the fact that it worked streaming both with t5 and gpt like models! Also the playground was really nice!