Transformers 4.6.0 on SageMaker?
Hi all,Is there a timeline for whenTransformers 4.6.0will be supported in the HuggingFace SDK on SageMaker?I’ve recently been having issues with CUDA running out of memory while training a distilBert model:RuntimeError: CUDA out of memory. Tried to allocate 6.87 GiB (GPU 0; 15.78 GiB total capacity; 7.35 GiB already allocated; 2.79 GiB free; 11.78 GiB reserved in total by PyTorch)It seems like this has been acknowledged and fixed in arecent commit- also similarly describedhere. Looks like this has also been added toTransformers 4.6.0and I can confirm that using this latest version (without the HuggingFace SDK) fixes the OOM issues for me.When configuring the HuggingFace estimator, it seems like the latest supported version ofTransformersis version4.5.0ValueError: Unsupported huggingface version: 4.6.0. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.4.2, 4.5.0, 4.4, 4.5.Does anyone have an idea of when we can expect version4.6.0to be supported?Thanks!
2021-05-18T06:21:48Z
hey@nreamaroon,We already opened a PR for a DLC withtransformers 4.6.0. I hope we can get as merged as soon as possible.github.com/aws/deep-learning-containers[huggingface_tensorflow, huggingface_pytorch] update for Transformers to 4.6.0aws:master←philschmid:patch-1opened05:50PM - 12 May 21 UTCphilschmid+30-14… 4.6.0

*Issue #, if available:*

## PR Checklist
- [x] I've prepended PR…tag with frameworks/job this applies to : [mxnet, tensorflow, pytorch] | [ei/neuron] | [build] | [test] | [benchmark] | [ec2, ecs, eks, sagemaker]
- [x] (If applicable) I've documented below the DLC image/dockerfile this relates to
- [x] (If applicable) I've documented below the tests I've run on the DLC image
- [x] (If applicable) I've reviewed the licenses of updated and new binaries and their dependencies to make sure all licenses are on the Apache Software Foundation Third Party License Policy Category A or Category B license list.  See [https://www.apache.org/legal/resolved.html](https://www.apache.org/legal/resolved.html).
- [x] (If applicable) I've scanned the updated and new binaries to make sure they do not have vulnerabilities associated with them.

## Pytest Marker Checklist
- [ ] (If applicable) I have added the marker `@pytest.mark.model("<model-type>")` to the new tests which I have added, to specify the Deep Learning model that is used in the test (use `"N/A"` if the test doesn't use a model)
- [ ] (If applicable) I have added the marker `@pytest.mark.integration("<feature-being-tested>")` to the new tests which I have added, to specify the feature that will be tested
- [ ] (If applicable) I have added the marker `@pytest.mark.multinode(<integer-num-nodes>)` to the new tests which I have added, to specify the number of nodes used on a multi-node test
- [ ] (If applicable) I have added the marker `@pytest.mark.processor(<"cpu"/"gpu"/"eia"/"neuron">)` to the new tests which I have added, if a test is specifically applicable to only one processor type

#### EIA/NEURON Checklist
* When creating a PR:
- [ ] I've modified `src/config/build_config.py` in my PR branch by setting `ENABLE_EI_MODE = True` or `ENABLE_NEURON_MODE = True`
* When PR is reviewed and ready to be merged:
- [ ] I've reverted the code change on the config file mentioned above
#### Benchmark Checklist
* When creating a PR:
- [ ] I've modified `src/config/test_config.py` in my PR branch by setting `ENABLE_BENCHMARK_DEV_MODE = True`
* When PR is reviewed and ready to be merged:
- [ ] I've reverted the code change on the config file mentioned above

## Reviewer Checklist
* For reviewer, before merging, please cross-check:
- [ ] I've verified the code change on the config file mentioned above has already been reverted

*Description:*

This PRs updates the `transformers` version inside the `huggingface` PyTorch and TensorFlow DLC.

*Tests run:*

https://github.com/huggingface/transformers/tree/master/tests/sagemaker


| ID                                  | Description                                                       | Platform                   | #GPUS | Collected & evaluated metrics            |
|-------------------------------------|-------------------------------------------------------------------|-----------------------------|-------|------------------------------------------|
| pytorch-transfromers-test-single    | test bert finetuning using BERT fromtransformerlib+PT             | SageMaker createTrainingJob | 1     | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-2-ddp     | test bert finetuning using BERT from transformer lib+ PT DPP      | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-2-smd     | test bert finetuning using BERT from transformer lib+ PT SM DDP   | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |
| pytorch-transfromers-test-1-smp     | test roberta finetuning using BERT from transformer lib+ PT SM MP | SageMaker createTrainingJob | 8     | train_runtime, eval_accuracy & eval_loss |
| tensorflow-transfromers-test-single | Test bert finetuning using BERT from transformer lib+TF           | SageMaker createTrainingJob | 1     | train_runtime, eval_accuracy & eval_loss |
| tensorflow-transfromers-test-2-smd  | test bert finetuning using BERT from transformer lib+ TF SM DDP   | SageMaker createTrainingJob | 16    | train_runtime, eval_accuracy & eval_loss |

*DLC image/dockerfile:*

The Hugging Face DLCs for PyTorch and TensorFlow


*Additional context:*
![Bildschirmfoto 2021-04-06 um 14 20 34](https://user-images.githubusercontent.com/32632186/114026575-afec6480-9876-11eb-892e-55144707367d.png)



By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
2021-06-11T00:28:33Z
Hi@philschmid-Thank you for all your work on this. I have just tried to useTransformersversion4.6.0in a Huggingface estimator in Sagemaker, but get the same error as reported by@nreamaroonValueError: Unsupported huggingface version: 4.6. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.4.2, 4.5.0, 4.4, 4.5.I saw that the PR that youreferencedhas now been merged so am wondering if there’s anything else I need to do in order to be able to useTransformersversion4.6.0?Thanks again!
2021-06-11T07:07:01Z
Hi@benG,There were some issues with the python sdk. But it is now merged and should be released tomorrow. You can use the new sagemaker-sdk withpip install git+https://github.com/aws/sagemaker-python-sdk.gitor you can wait until tomorrow and installpip install sagemakerand then definetransformers_version='4.6'(4.6.1).huggingface_estimator = HuggingFace(entry_point='train.py',
                            source_dir='./scripts',
                            instance_type='ml.p3.2xlarge',
                            instance_count=1,
                            role=role,
                            transformers_version='4.6',
                            pytorch_version='1.7',
                            py_version='py36',
                            hyperparameters = hyperparameters)
2021-12-01T19:51:23Z
Hi@philschmid.In theAWS SageMaker HF workshops notebooks, we need to install the following versions!pip install "sagemaker>=2.48.0" "transformers==4.6.1" "datasets[s3]==1.6.2"Are they still the right versions?How could we know in the future which compatible versions to install? Thanks.
2021-12-02T07:10:58Z
Hey@pierreguillou,Yes, versions still are compatible, but you can upgrade them as you want. You can find an overview of all DLCs + versions atReference, which should be updated soon. I created a PR to update themadd new container version by philschmid · Pull Request #514 · huggingface/huggingface_hub · GitHub.The latestTransformersversion is currently4.12.3. You can also check here for notebook example with newer versionsnotebooks/sagemaker-notebook.ipynb at master · huggingface/notebooks · GitHub
2021-12-02T13:00:14Z
Thanks@philschmidJust to be sure, here is my understanding.NotebookIn the notebooknotebooks/sagemaker-notebook.ipynb at master · huggingface/notebooks · GitHubthat you mentioned, the libraries versions are the following ones:sagemaker>=2.69.0transformers==4.12.3datasets==1.13The installation is done by the following code:!pip install "sagemaker>=2.69.0" "transformers==4.12.3" --upgrade
# using older dataset due to incompatibility of sagemaker notebook & aws-cli with > s3fs and fsspec to >= 2021.10
!pip install  "datasets==1.13" --upgradeI noticed that unlike the first notebook (used in your workshop), you no longer installdatasets[s3]but directlydatasets.I’m just curious as to why and what are the differences between these 2 libraries?Thanks.Reference & PRIn theReferencepage, the transformers version goes up to but4.6.1(and compatible datasets1.6.2) but your recentPRshows that the current situation of Training DLC istransformers 4.12.3, datasets 1.15.1, PyTorch 1.9.1, python 3.8Does it mean I could run the following code in a notebook instance of AWS SageMaker?!pip install "sagemaker>=2.69.0" "transformers==4.12.3" --upgrade
!pip install  "datasets==1.15.1" --upgradeIf yes, I would need to update as well Pytorch and python as I do not see in the available kernel, this configuration.How can I do this?(or do you recommend to create a new conda kernel? through the jupyter terminal? could you give use the commands?)
2021-12-03T07:54:55Z
ThePRis the overview of the version inside the DLCs. Meaning the container you use when runninghuggingface_estimator.fit(). In the notebook, you can install every version you want, but it makes most of the time sense to use the same version as you will in theHuggingFaceestimator.Thanks for finding the pip installation it should bedatasets[s3]to make sure we install the S3 utils to push our datasets to s3.
2022-09-09T14:34:57Z
I’m currently getting this error with the Sagemaker SDK trying to useTransformers==4.19.4. Hoping to use >=4.19.0 to take advantage of some deBERTa models.
2022-09-09T15:03:46Z
What error are you getting?
2022-09-09T15:06:41Z
This -ValueError: Unsupported huggingface version: 4.19.4. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.6.1, 4.10.2, 4.11.0, 4.12.3, 4.17.0, 4.6, 4.10, 4.11, 4.12, 4.17.In my pipeline’srequirements.txtI have it usingsagemaker==2.108.0.
2022-09-09T15:18:10Z
There is currently no available DLC with thistransformersversion, but you can add arequirements.txtwith the version you want to install.
2022-09-09T15:22:06Z
Ah, ok. So even if the DLC/Sagemaker SDK doesn’t have the version I need, I can just specify it myself as a requirement – thanks.
2022-09-09T15:30:21Z
Yes, thats correct but we are working on updating the version.
2022-09-09T16:28:40Z
Good to hear, thank you!We’re getting errors in our terragrunt apply when we add any transformers version to our requirements.txt for some reason. Strangely the build works fine specifying any other packages besidestransformers. We’ll keep digging on that unless you have any recommendations.