Load a cached custom model in offline mode

🤗Transformers
1

I am having trouble loading a custom model from the HuggingFace hub in offline mode. My steps are as follows:

  1. With an internet connection, download and cache the model
from transformers import AutoModelForSeq2SeqLM

_ = AutoModelForSeq2SeqLM.from_pretrained("ccdv/lsg-bart-base-4096-wcep", force_download=True, trust_remote_code=True)
  1. Without an internet connection, try to use said model
import os
from transformers import AutoModelForSeq2SeqLM

os.environ["TRANSFORMERS_OFFLINE"] = "1"

# This hangs, because it tries to make a request in an offline environment
_ = AutoModelForSeq2SeqLM.from_pretrained("ccdv/lsg-bart-base-4096-wcep", force_download=True, trust_remote_code=True)

Unfortunately, the code just hangs. If I command/ctrl-c it, I can see it was trying to make a request

line 85, in create_connection
    sock.connect(sa)
KeyboardInterrupt

Which of course hangs because there is no access to the internet. A couple notes:

  • This works fine for models without custom code
  • I need this to work in an offline environment because the compute nodes of the cluster I am working on do not have access to the internet

Is this a known/unknown bug? Or am I making a mistake in how I download and cache the custom model?

2

One solution is to load the model with internet access, save it to your local disk (with save_pretrained()) and then load it with AutoModel.from_pretrained from that path. Ideally, you would be able to load it right from the model’s name and avoid explicitly saving it to disk, but this works.