Adding Preprocessing to Hosted Inference API
My question answering model requires an extra step of preprocessing the input and the output. How can I add those preprocessing scripts to work with Hosted Inference API (Widget on the website, too)?
2021-07-13T13:01:10Z
hey@yigitbekir, as far as i know the inference api does not support custom pre- / post-processing logic, but you could easily include these steps within a dedicated web / streamlit applicationif you need a custom widget, you can propose one on thehuggingface_hublibrary here:GitHub - huggingface/huggingface_hub: Client library to download and publish models and other files on the huggingface.co hub
2022-04-13T18:44:10Z
Hello! There isa sectionin the Huggingface Hub Docs that describesGeneric Inference API.In my understanding this Generic Inference API allows you to override the default pipeline used in Hosted Inference API.In order to make this work you must:put your code topipeline.pyspecify requirements inrequirements.txtspecifylibrary_name: generictag-value inREADME.mdI tried it myself but I get following errors:“Distant resource does not have an ETag, we won’t be able to reliably ensure reproducibility.”.Or something like “the model took very long to load”. Probably it’s because of long wait for Docker container to install my required dependencies and load the model.Probably there is a way to make this Generic Inference API work.Now I’m planning to switch to Huggingface Spaces - probably I can set up my own processing logic for inference there.
2022-04-13T18:45:07Z
Here are some example repositories that make use of Generic Inference API:oneandtwo.
2022-04-14T08:11:33Z
upd: I’ve deployed my demo application to Huggingface Spaces and all works fine. You can specify your custom logic of data processing there as well as list of dependencies (both python and debian system packages) that are installed while building Docker container on Space launch.@yigitbekir