Using a space as a backend?
Hi Spaces!I am new to HF Spaces. Here is the project I’m imagining:I’m working on an iOS app to take a hand drawn image, pass it through ControlNet, and get back a diffusion image to then print out. The goal would be to bring this to a group of kids to let them play with diffusion and get a print out of their own work enhanced by Diffusion.I know how to run ControlNet in a Colab notebook. I am wondering what the simplest translation of this is to a HF Space. I don’t necessarily need a frontend for the experience - I think we will build that in iOS - so I don’tthinka Streamlit or Gradio frontend makes sense. Really I just want a persistent server which can serve as an endpoint to ControlNet (effectively the code from a Python notebook) for an iOS app. I see the word Docker and I flinch (“is Docker tech overkill”) but is that the right way to interact with HF spaces if what I effectively want to do is turn a Colab notebook into a persistent endpoint I can call from say an iOS app?Thoughts welcomed!!! Thank you!
2023-04-20T22:44:55Z
hello@mpnagle,Spaces are designed for demos and sharable ML apps. You might be interested in ourInference Endpoints solutionwhich provides exactly what you’re looking for, a dedicated, fully managed inference server. You might need to convert your Jupiter notebook to a custom handler python code and with a few clicks you can deploy and have an API endpointLuckily@philschmidwrote a really comprehensive tutorial about deployingControlNetas an inference endpoint, so you can just copy adapt hishandler.pyYou can also see multiple models templates on ourhub.philschmid blog – 3 Mar 23Controlled text-to-image generation with ControlNet on Inference EndpointsLearn how to deploy ControlNet Stable Diffusion Pipeline on Hugging Face Inference Endpoints to generate controlled images.huggingface.cophilschmid/ControlNet-endpoint · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2023-04-20T23:49:37Z
Hi@radames–Thank you! I think this is exactly what we’re looking for.I will look further, but two quick questions:I know HF provides grants for Spaces. Are there grants for endpoints? The end goal of this is an educational demo (exploring ControlNet with kids – draw a picture and pass it into ControlNet.) and if not –I will look into this, but I think the main thing we will need up time for is a) testing and b) actual use with kids. I assume that’s easy to manage (turning an endpoint on and off?)Thank you!
2023-04-21T00:01:15Z
Great!I’m not sure about grants for Inference endpoints, since it’s a production ready service cc@philschmidOn another note, we’re going to release theimage-to-imagetask, which includes ControlNet, in ourAPI Inferencewith offers free access with rate limit and higher rates for Pro users, however this service is not optimized for high loads and optimized inference, but it could be a good start.
2023-04-21T02:26:40Z
Thank you@radames!Do you know roughly how soon you and your team will release the image-to-image task for API Inference?
2023-04-27T19:09:35Z
Hello again@radames!I have now looked at the blog post you shared - which is exactly what we are looking for, an endpoint for ControlNet.For pricing an endpoint - is it pay by use, or is provisioned for as long as the endpoint is up? I ask because the suggested GPU option from Phil’s blogpost, “GPU medium” is estimated at $900 / month, and “GPU small” is ~$400 / month.I think if our project really got going we’d love to pay for a full time endpoint but for now we are just in the phases of building an iOS frontend to diffusion models so that we can run a few initial workshops for kids on drawing and diffusion and see if there really is a thing there. (I think we’d be using the endpoint a few hours a time at most.)Do you have any suggestions – perhaps if there is a grant we could apply for like for HF spaces – or if there is a lower cost way to provision an endpoint with a GPU?Thank you!!
2023-05-09T20:24:31Z
hi@mpnagle, sorry for the delay, the image-to-image pipeline is now supported o most models with the correct tag taskimage-to-imageHere’s a tweet with more context, not sure what’s your framework, but using ourJS libraryit’s very easyawait hf
  .imageToImage({
    inputs: imageBlob,
    model: "lllyasviel/control_v11f1p_sd15_depth",
    parameters: {
      prompt: "Elmo's Lecture in a tropical beach in the background"
    }
  })
  .then((blob) => {
    const image = new Image();
    image.src = URL.createObjectURL(blob);
    return image;
  })twitter.comRadamés Ajna@radamarOur Community Inference API now supports Image-to-Image models! Here are some examples of how to use it with our JavaScript library - huggingface.js. Any model with the image-to-image task tag can work.

Of course Controlnet models

lllyasviel/control_v11f1p_sd15_depthhttps://t.co/dqq22tcErQ8:18 PM - 9 May 202315025