And there you go, now you have a good idea of how to create a webserver!
What is really important is that we load the model only once, so there are no copies
of the model on the webserver. This way, no unnecessary RAM is being used.
Then the queuing mechanism allows you to do fancy stuff like maybe accumulating a few
items before inferring to use dynamic batching: