“No matching distribution found for wordninja==2.0.0” when using HuggingFace + SageMaker
Hi,Greetings!!I’m trying to deploy trained NER model on sagemaker with huggingface DLC then I’m getting this error.Could you please help on this issue?DLC: image_uri = “763104351884.dkr.ecr.us-east-2.amazonaws.com/huggingface-pytorch-inference:1.9.1-transformers4.12.3-gpu-py38-cu111-ubuntu20.04”wordninja package is part of requirements.txt file:wordninja==2.0.0Cloud watch log details:Collecting wordninja==2.0.0Downloading wordninja-2.0.0.tar.gz (541 kB)Preparing metadata (setup.py): startedPreparing metadata (setup.py): finished with status ‘error’ERROR: Command errored out with exit status 1:command: /opt/conda/bin/python -c ‘import io, os, sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/home/model-server/tmp/pip-install-hn_fwkxb/wordninja_8ffb1cc5dfed4f5d81ff142e00e3b06c/setup.py’"’"’;file=’"’"’/home/model-server/tmp/pip-install-hn_fwkxb/wordninja_8ffb1cc5dfed4f5d81ff142e00e3b06c/setup.py’"’"’;f = getattr(tokenize, ‘"’"‘open’"’"’, open)(file) if os.path.exists(file) else io.StringIO(’"’"‘from setuptools import setup; setup()’"’"’);code = f.read().replace(’"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code,file, ‘"’"‘exec’"’"’))’ egg_info --egg-base /home/model-server/tmp/pip-pip-egg-info-4xgnsg33cwd: /home/model-server/tmp/pip-install-hn_fwkxb/wordninja_8ffb1cc5dfed4f5d81ff142e00e3b06c/Complete output (5 lines):Traceback (most recent call last):File “”, line 1, inFile “/home/model-server/tmp/pip-install-hn_fwkxb/wordninja_8ffb1cc5dfed4f5d81ff142e00e3b06c/setup.py”, line 6, inscheme[‘data’] = scheme[‘purelib’]KeyError: ‘purelib’WARNING: Discardinghttps://files.pythonhosted.org/packages/30/15/abe4af50f4be92b60c25e43c1c64d08453b51e46c32981d80b3aebec0260/wordninja-2.0.0.tar.gz#sha256=1a1cc7ec146ad19d6f71941ee82aef3d31221700f0d8bf844136cf8df79d281a(fromLinks for wordninja). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.ERROR: Could not find a version that satisfies the requirement wordninja==2.0.0 (from versions: 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 2.0.0)ERROR: No matching distribution found for wordninja==2.0.0failed to install required packages, exitingNote: If I use Pytorch DLC then able to install wordninja==2.0.0 package but not with Huggingface DLC.Even I tried with wordninja==0.1.5 but no luck.Thanks in advance.
2022-04-06T12:09:48Z
could you try the latest DLC with PyTorch 1.10.2 and Transformers 4.17.0?You said with PyTorch it is working, which version?
2022-04-06T13:44:20Z
Thanks for your prompt reply.It’s working with Pytroch DLC, Pytorch version = 1.9 and Python version = py38As suggested I tried the huggingface DLC with PyTorch 1.10.2 and Transformers 4.17.0 but I’m getting this error.Failure reasonThe image ‘763104351884.dkr.ecr.us-east-2.amazonaws.com/huggingface-pytorch-inference:1.10.2-transformers4.17.0-gpu-py38-cu111-ubuntu20.04’ does not exist.I noticed that the latest huggingface DLC details are not available in this repo.github.comaws/sagemaker-python-sdk/blob/e0b9d38e1e3b48647a02af23c4be54980e53dc61/src/sagemaker/image_uri_config/huggingface.json{
    "training": {
        "processors": ["gpu"],
        "version_aliases": {
            "4.4": "4.4.2",
            "4.5": "4.5.0",
            "4.6": "4.6.1",
            "4.10": "4.10.2",
            "4.11": "4.11.0",
            "4.12": "4.12.3"
        },
        "versions": {
            "4.4.2": {
                "version_aliases": {
                    "pytorch1.6": "pytorch1.6.0",
                    "tensorflow2.4": "tensorflow2.4.1"
                },
                "pytorch1.6.0": {
                    "py_versions": ["py36"],
                    "registries": {This file has been truncated.show original
2022-04-06T15:17:33Z
They are available check:https://github.com/aws/sagemaker-python-sdk/blob/6ce11cb541ec46692ba61aaaf5976ca5e6e0ed88/src/sagemaker/image_uri_config/huggingface.json#L1023763104351884.dkr.ecr.us-east-2.amazonaws.com/huggingface-pytorch-inference:1.10.2-transformers4.17.0-gpu-py38-cu111-ubuntu20.04’the cuda version changed tocu113
2022-04-06T16:06:42Z
Thanks a lot@philschmidIt’s working.