504 Gateway Timeout with http request
I created space using Gradio, because I really need the API functionality. In app.py I am simply loading OpenAI’s whisper and getting predictions from audio. But always, when the request takes longer than a minute I get 504. No matter what I do. I tried setting enable_queue param (but that’s enabled by default), using the queue method, setting timeouts on the client, changing clients (from flutter http to python requests) and using fastapi in gradio space. Nothing worked so far.  The error is from nginx so I think there is some set timeout, that I cannot change. MAYBE I could use set command to change the /etc/… nginx conf from simple subprocces command, but I am not sure that that would work and I also think it should work by default or be easily changable. I would really appreciate some response since I’ve been struggling with this for a few days now.
2022-10-06T07:46:56Z
Hello@Skyrr,Could you provide a link to the failed space please ?
2022-10-06T08:43:07Z
Yes of course. Sorry I forgot that. Here:OpenAIWhisperLargePredictions - a Hugging Face Space by SkyrrRight now I am using FastAPI and the API address is:https://hf.space/embed/Skyrr/OpenAIWhisperLargePredictions/notesThank you!
2022-10-06T10:10:17Z
The API seems to reply for me.Though your endpoint seems to require a body and you defined aGETmethod for/notes
2022-10-06T10:11:00Z
Changing@app.get("/notes")to@app.post("/notes")should allow you to post to this endpoint
2022-10-06T10:19:31Z
So it replies for you even when it takes longer than a minute?
2022-10-06T10:20:17Z
I used post with gradio’s default API. The problem was still there.  But I can try post with FastAPI.
2022-10-06T10:29:21Z
Tried that, still getting the same error. Gateway timeout after a minute.
2022-10-06T10:36:28Z
Could you provide a sample body so that I can try the requests on my side.It seems your call triggers a 500 on your api, might be linked
2022-10-06T10:54:49Z
Sure thing.Flutter:final url = Uri.parse('https://hf.space/embed/Skyrr/OpenAIWhisperLargePredictions/+/api/predict');
    final response = await http.post(url,
        headers: {
          'Content-Type': 'application/json',
          'Accept': 'application/json',
        },
        body: jsonEncode(<String, List<dynamic>>{
        "data": [
        {"name": fileName, "data": readAudioFile(path)}
        ]
        }),encoding: Encoding.getByName('utf-8'))
    .timeout(const Duration(seconds: 60 * 60));

String readAudioFile(String path) {
  final file = io.File(path);
  List<int> bytes = file.readAsBytesSync();
  String base64String = base64Encode(bytes);
  final fileString = 'data:audio/wav;base64,$base64String';
  return fileString;
}Python requests:import requests
import base64

def main():
    url  = "https://hf.space/embed/Skyrr/OpenAIWhisperLargePredictions/notes"
    headers = {
        "Content-Type": "application/json",
        "Encoding": "utf-8",
        "Accept": "application/json",
        }

    
    data = {"audio": base64decode(r"C:\Users\skirl\Downloads\1664830934891.wav")}
    
    response = requests.post(url, headers=headers, json=data,timeout=3600) 
    
    print(response.text)


def base64decode(path):
    return base64.b64encode(open(path, "rb").read()).decode("utf-8")


if __name__ == "__main__":
    main()Please note that I didn’t yet update the url in flutter.
2022-10-06T11:14:07Z
So there is indeed a timeout after 1 min.Within gradio when using the UI the call is not done directly but a websocket is open and the call down internally, the result is then sent to the websocket
2022-10-06T11:33:46Z
Any ideas how to solve or bypass that?
2022-10-06T11:44:41Z
The timeout is actually expected to prevent requests from hanging forever in the event of the space is crashed and cannot restart.Using the gradio interface directly, do you also experiment the 504 or only when directly querying the API ?
2022-10-06T11:47:14Z
Only with the API. Interface works fine. Are you saying that I can open a socket connection, send request and just listen for response?
2022-10-06T12:34:51Z
Yes, it seems that’s what gradio is doing
2022-10-06T12:39:49Z
Okay. Thank you so much for helping me solve this.
2022-10-06T17:30:54Z
Hi@Skyrr! Thank you for posting and thank you@chris-rannoufor helping.If you want to use a space programmatically from another space, the best way is theloadmethod ongradio.Blocks.So if you do something likeother_app = gr.Blocks.load("spaces/<your-space>"), you can useother_applike a regular python function. For example ,output = other_app(<input>, fn_index=0). This will take care of creating the websocket for you.Here is an example using this approach to call stable diffusion from another app:Using loaded interface as a function in Blocks return error · Issue #2369 · gradio-app/gradio · GitHubHere is a guide talking about this more in depth:Using Blocks Like Functions
2022-10-07T06:43:29Z
Thank you for suggestion. The problem is that I need to connect to the app from flutter. I actually decieded that easiest option will be to just use whisper-large repo directly with inference api. Only issue there is that it’s often overloaded. I will se how to solve that. Maybe later I’ll use inference endpoints.
2022-10-23T13:26:14Z
Have you solved the problem yet? I’m having a 504 time out problem. When I request input to space, the result returned is as follows:b’\r\n504 Gateway Time-out\r\n\r\n504 Gateway Time-out\r\n\r\n\r\n’.If you have solved it please help me. Thank you
2022-10-23T13:28:32Z
image1495×30 2.19 KB
No date available
No reply text available