Model works but MultiDataModel doesn't
This is a continuation of my posthere. I’m trying to deploy BERT for text classification with tensorflow. When I use the model.deploy() method, I can successfully get inferences from BERT. Here’s my problem: I have four different models for classification and I want to run them on the same instance, not multiple instances, to save on cost. So I tried using the MultiDataModel class, but I keep getting the following error:image979×1189 72.2 KBThe CloudWatch logs don’t add any additional information, unfortunately. Here’s the structure of counterargument.tar.gz in the s3 bucket, which I cloned from myHuggingFace accountand zipped.counterargument.tar.gz
    config.json
    special_tokens_map.json
    tf_model.h5
    tokenizer.json
    tokenizer_config.json
    vocab.txtThe most puzzling thing about this error is that model.deploy() worked fine, but multi_model.deploy() doesn’t! Thanks in advance.
2021-07-21T07:11:49Z
@wsunadawongwhen using Multi-Model-Endpoints SageMaker stores the models differently. That’s whymodel.deploy()seems to work and MME does not. We (Amazon & HF) are looking into it. Hopefully, we can come back with a fix as soon as possible!
2021-07-21T16:19:13Z
Could you share logs from the endpoint’s cloudwatch logs? Also, the counterargument.tar.gz is a flat directory or does that tar have a directory inside which these files exist?
2021-07-21T16:20:36Z
The counterargument.tar.gz has a flat directory structure. I git cloned my huggingface repo and just zipped it. Here’s a picture of the contents:image706×577 21.9 KB
2021-07-21T16:24:46Z
And here’s a picture of the logsimage1082×1046 68.4 KB
2021-07-21T20:07:44Z
Could you share more of the log. I see that “Modelmodelloaded” message. I want to understand where invocation failed.
2021-07-21T20:13:02Z
Here’s the logs which were above what I sent. There’s nothing useful below–just ping messages.image1426×959 42.1 KB
2021-07-21T20:25:19Z
Oh my mistake, there were more non-ping logs below. Is this what you were looking for?image1413×1175 246 KB
2021-07-22T00:10:22Z
The container seems to be using default model directory to lookup for model instead of MME platforms model directory. We will try to reproduce this and get back.
2021-08-18T14:10:38Z
Any news on this by any chance?
2021-08-18T14:37:51Z
Hey@Ilias,The MultiDataModel was solved. Themodel.tar.gzused was wrong. You can take a look here.Error deploying BERT on SageMakerAmazon SageMakerThat worked! The MultiDataModel works now! When I zipped the first time, I included the folder with tar zcvf model.tar.gz counterargument_hugging when I should’ve just included the folder contents.Thank you so much for your help.