Availability of models pushed to Hub
I pushed my first model to the hub usingpush_to_hubfunction around an hour ago. I can see it up on the Hub website athttps://huggingface.co/zyl1024/bert-base-cased-finetuned-qqp. However, when I try to use it in transformer, by following the instruction on that page,from transformers import AutoTokenizer, AutoModelForSequenceClassification  
tokenizer = AutoTokenizer.from_pretrained("zyl1024/bert-base-cased-finetuned-qqp")
model = AutoModelForSequenceClassification.from_pretrained("zyl1024/bert-base-cased-finetuned-qqp")I get an error sayingOSError: Can't load tokenizer for 'zyl1024/bert-base-cased-finetuned-qqp'. Make sure that:
- 'zyl1024/bert-base-cased-finetuned-qqp' is a correct model identifier listed on 'https://huggingface.co/models'
- or 'zyl1024/bert-base-cased-finetuned-qqp' is the correct path to a directory containing relevant tokenizer filesDo I need do something else or just wait for a while (if so, how long?) for it to become available for download?
2021-09-22T09:47:03Z
Do I need do something else or just wait for a while (if so, how long?) for it to become available for download?No, normally it should be directly accessible.However, I see why it doesn’t work: your model repository only contains modeling files (config.jsonandpytorch_model.bin), but no tokenizer files (such asvocab.txt). Hence, only loading the model will work. You can easily save the files of a tokenizer as follows:tokenizer.save_pretrained("path_to_directory")You can then upload those files to your repo on the hub.
2021-09-22T14:13:13Z
Ah yes. I thought the model pushing takes care of the tokenizer automatically. After I rantokenizer.push_to_hub(), it works perfectly. Thanks very much!