Spaces:
Running
Text/Chat generation version of this
This is pure art (pun intended), but can we have one like this but for text generation/chat models? It would be amazing since I'm trying to find LLMs that are good at ideation and creativity, and current giants suck at it.
Thanks! I'm not well versed in text generation models, for what I've seen either they required a Zero GPU space to run, or their inference API returned very poor results. This was the best one I used and I was going to recommend it: https://huggingface.co/spaces/thobuiq/GPT-4o but apparently it has suffered a 'runtime error' and is now unavailable.
I guess I'll try summoning @John6666 which has provided me with the biggest help when I needed something, he has this space running: https://huggingface.co/spaces/John6666/joy-caption-pre-alpha-mod which uses the text generation Chatbots you want to give descriptions to pictures, so I bet there would be a way to not send a picture and use them to generate text normally, though I have no idea how would such a thing be achieved.
Most of these hundreds of image models are just finetunes of Stable Diffusion 1.5, to specifically deliver better at certain pictures, but for general generation of text maybe a single model would do (best at continuing sentences or chatting?), but I can't find any space that allows you to generate text (other than the ones that describe pictures) in the way this one does, and the only working one using Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2 is the one linked above. It's odd that it's much easier to generate images than text, now that I think about it.
I've been summoned!
As a matter of fact, LLM can also be run with gr.load() if it is a single entity. However, it is limited to 10GB or less.
For larger ones, it is necessary to use other APIs, but there is already a space like the one below. Also, the new HF service that was recently released seems to be able to use a large number of models at relatively low cost or for free.
https://huggingface.co/spaces/featherless-ai/try-this-model
Also, if you want to use GGUF instead of the Inference API, you can use Zero GPU space to run models of up to around 32B.
https://huggingface.co/spaces/John6666/text2tag-llm
Thanks for the inputs! I found this a week ago - https://huggingface.co/spaces/akhaliq/anychat
It fits the criteria perfectly but unfortunately isn't good in practice. It lags a lot, and majority of models/providers throw error when you type something.
That space really has a lot of lags... probably because there are too many GUI items.π
However, I don't think it's possible to use some kind of API for free... the LLM model size is too big compared to SD1.5 and SDXL. If it were free, HF would go bankrupt.
I think the realistic thing to do is to find someone who is hosting a medium-sized LLM and use their Zero GPU space or paid GPU space. That way, the person hosting the space will pay the fee.