T5 Inference using tensorflow_model_server (with grpc)
Any examples of how to grpc call a tensorflow_model_server running a T5 model?What I did so far is:Saving a model in torch format using:model.save_pretrained(“local-pt-checkpoint”)Convert the saved model to ONNX using:python -m transformers.onnx --model=local-pt-checkpoint --opset=13 onnx/Convert the ONNX to tf using:onnx-tf convert -i onnx/model.onnx -o tf_exportRun the model server using:tensorflow_model_server --port=9000 --model_name=my_model --model_base_path=./t5-base/tf_export/.Now I just need to find some example code of how to call the model server.