Load a cached custom model in offline mode
I am having trouble loading a custom model from the HuggingFace hub in offline mode. My steps are as follows:With an internet connection, download and cache the modelfrom transformers import AutoModelForSeq2SeqLM

_ = AutoModelForSeq2SeqLM.from_pretrained("ccdv/lsg-bart-base-4096-wcep", force_download=True, trust_remote_code=True)Without an internet connection, try to use said modelimport os
from transformers import AutoModelForSeq2SeqLM

os.environ["TRANSFORMERS_OFFLINE"] = "1"

# This hangs, because it tries to make a request in an offline environment
_ = AutoModelForSeq2SeqLM.from_pretrained("ccdv/lsg-bart-base-4096-wcep", force_download=True, trust_remote_code=True)Unfortunately, the code just hangs. If I command/ctrl-c it, I can see it was trying to make a requestline 85, in create_connection
    sock.connect(sa)
KeyboardInterruptWhich of course hangs because there is no access to the internet. A couple notes:This works fine for models without custom codeI need this to work in an offline environment because the compute nodes of the cluster I am working on do not have access to the internetIs this a known/unknown bug? Or am I making a mistake in how I download and cache the custom model?
2022-09-19T18:26:53Z
One solution is to load the modelwith internet access, save it to your local disk (withsave_pretrained()) and then load it withAutoModel.from_pretrainedfrom that path. Ideally, you would be able to load it right from the model’s name and avoid explicitly saving it to disk, but this works.