Runtime Error Due to Mac M1 Architecture?
I am trying to use AutoModelForTokenClassification from transformers. It is importing correctly but I’m getting the following runtime error when executingmodel = AutoModelForTokenClassification.from_pretrained("distilbert-base-uncased", num_labels=len(label_list))Error:"name": "RuntimeError", "message": "Failed to import transformers.models.camembert.configuration_camembert because of the following error (look up to see its traceback):\ndlopen(/Users/etan/Projects/Proj1/venv/lib/python3.9/site-packages/tokenizers/tokenizers.cpython-39-darwin.so, 0x0002): tried: '/Users/etan/Projects/Proj1/venv/lib/python3.9/site-packages/tokenizers/tokenizers.cpython-39-darwin.so' (**mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')**), '/usr/local/lib/tokenizers.cpython-39-darwin.so' (no such file), '/usr/lib/tokenizers.cpython-39-darwin.so' (no such file)"Looks like it has something to do with the fact that I have an x86_64 architecture, but when I execute the commandexec arch -arm64 zshit exits my virtual environment and applies the arch to my global env. What should I do? For reference, I have a 2021 MacBook Pro with M1 chip running MacOS Monterey (12.2.1).
2022-06-30T12:29:36Z
I think this is a relevant issue for you:wrong architecture `tokenizers.cpython-39-darwin.so` (x86_64) when installing on apple silicon (arm64) · Issue #712 · huggingface/tokenizers · GitHub
2022-07-04T01:11:37Z
I tried building manually by following the instructions in the post, but it’s still building for the arm64 architecture instead of x86_64 (the .whl file it gives me istokenizers-0.12.1-cp39-cp39-macosx_12_0_arm64.whl. Is there any way that I can just download the correct .whl file and build it?