How to save model in S3 with Trainer?
When I doTrainer.save_model(‘s3://path_to_folder’)I got no error with messageSaving model checkpoint to s3://path_to_folderConfiguration saved in s3://path_to_folder/config.jsonModel weights saved in s3://path_to_folder/pytorch_model.bintokenizer config file saved in s3://path_to_folder/tokenizer_config.jsonSpecial tokens file saved in s3://path_to_folder/special_tokens_map.jsonBut corresponding folder in S3 is empty.How correctly save model in S3 with Trainer.save_model?
2022-08-17T13:41:20Z
Hey@AndriiPdid you figure this out? I have a similar issue. I’m running from a Sagemaker notebook instance and it seems to be saving the files as per the log being printed, but the s3 bucket is emptyThank you
2022-08-17T14:02:29Z
Hityatabe,I did not find the right solution.I just zip checkpoint folder, save on S3, when needed load, unzip and use it.
2022-08-18T13:53:59Z
Thank you@AndriiPI ended up doing something along the same lines. I found comfort in knowing I wasn’t the only one with this issue
2023-05-25T16:28:28Z
I believe you could use TrainerCallback to send it to S3 on_savehuggingface.coCallbacksWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2023-05-26T11:42:54Z
To use trained models in Sagemaker you can use Sagemaker Training Job. It will train and upload.tar.gzmodel to S3 for you to use. See here for more:huggingface.coRun training on Amazon SageMakerWe’re on a journey to advance and democratize artificial intelligence through open source and open science.