`flan-t5-xl` model does not appear to have a file named `pytorch_model.bin`
I am trying to fine-tune aflan-t5-xlmodel usingrun_summarization.pyas the training script on Amazon SageMaker.This is my main script:from sagemaker.huggingface import HuggingFace

git_config = {'repo': 'https://github.com/huggingface/transformers.git','branch': 'v4.17.0'}

# hyperparameters, which are passed into the training job
hyperparameters={'per_device_train_batch_size': 8,
                 'per_device_eval_batch_size': 8,
                 'model_name_or_path': 'google/flan-t5-xl',
                 'dataset_name': 'samsum',
                 'do_train': True,
                 'do_eval': True,
                 'do_predict': True,
                 'predict_with_generate': True,
                 'output_dir': f'{output_location}/model',
                 'num_train_epochs': 1,
                 'learning_rate': 5e-5,
                 'seed': 7,
                 'fp16': True,
                 'max_source_length': 1153,
                 'max_target_length': 95,
                 'source_prefix': 'summarize: '
                 }

# create the Estimator
huggingface_estimator = HuggingFace(
      entry_point='run_summarization.py',
      source_dir='./examples/pytorch/summarization',
      git_config=git_config,
      code_location=code_location,
      instance_type='ml.g4dn.xlarge',
      instance_count=1,
      transformers_version='4.17',
      pytorch_version='1.10',
      py_version='py38',
      role=role,
      hyperparameters=hyperparameters,
      output_path=output_location
)

# starting the train job
huggingface_estimator.fit()However, I get this error:OSError: google/flan-t5-xl does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack."It seems that the model is split into 2 chunks due to 10gb file size limit (pytorch_model-00001-of-00002.binandpytorch_model-00002-of-00002.bin)How could I approach this problem?I have thought about downloading and merging the model files into a singlepytorch_model.binfile and then specify the appropriate model path in  ‘model_name_or_path’.Would something like this work?:cat pytorch_model-00001-of-00002.bin pytorch_model-00002-of-00002.bin > pytorch_model.binOr perhaps I can download thepytorch_model.binfile directly from somewhere?
2023-01-25T21:06:07Z
The issue is here that you are usingtransformers==4.17.0, which is not having support for sharded models. To fix this you just need to create arequirements.txtin the./examples/pytorch/summarizationdirectory if it doesn’t exist yet and then addtransformers==4.25.1there.
2023-01-26T08:53:52Z
Thanks a lot for the answer@philschmidThe directory is located at thetransformersrepository fromhugginfaceMay I make a pull request in thev4.25-releasebranch and addtransformers==4.25.1?
2023-01-26T09:35:10Z
No this is not necessary. Since the scripts assumes the correct version. We are working on updating the container version that way no change is needed.
2023-01-26T09:53:01Z
Cool!Is there an approximate date for the update to be completed?
2023-01-26T10:03:51Z
sometime in February. i ll let you know.
2023-03-27T19:06:12Z
I’m unable to usetransformers==4.25.1withhfp = HuggingFaceProcessor(
    role=get_execution_role(), 
    instance_count=1,
    instance_type='ml.g4dn.2xlarge',
    transformers_version='4.25.1',
    pytorch_version='1.10.2', 
    base_job_name='frameworkprocessor-hf',
    py_version = 'py38'
)Here is the error`ValueError: Unsupported huggingface version: 4.25.1. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.4.2, 4.5.0, 4.6.1, 4.10.2, 4.11.0, 4.12.3, 4.17.0, 4.4, 4.5, 4.6, 4.10, 4.11, 4.12, 4.17.`Any idea when we’ll be able to use transformers==4.25.1 in this way?
2023-03-28T06:49:18Z
Can you upgrade your sagemaker-sdk version? Transformers 4.26.0 is now available.
2023-08-22T03:40:43Z
I’m having a similar problem withdeclare-lab/flan-alpaca-xl.There is a sharded model:pytorch_model-00001-of-00002.binandpytorch_model-00002-of-00002.binand I am usingtransformers==4.31.0yet I am also getting this error:OSError: Can't load weights for 'declare-lab/flan-alpaca-xl'. Make sure that:

- 'declare-lab/flan-alpaca-xl' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'declare-lab/flan-alpaca-xl' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt.I can create a new post if its not advisable to restart this old thread.
2023-08-22T06:36:59Z
Hey@wilkinsjle,have you tried using the LLM Container yet?Introducing the Hugging Face LLM Inference Container for Amazon SageMaker
2023-08-22T07:03:18Z
Nice, thanks for the suggestion. Sorry, I forgot to add that I was actually deploying locally without AWS Sagemaker, so my case is slightly different from the OP, but I’ll give this a look.