Running Llama model in Google colab
Hi,I’m trying to see a Llama 8B model demo in Google colab.I just tried to run the following code in the colab prompt.huggingface.cometa-llama/Meta-Llama-3.1-8B-Instruct · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.It ran into several errors.I tried to fix using  !pip install transformers[sentencepiece]or  !pip install --upgrade transformersbut to no availAny  help will be much appreciated.NameError                                 Traceback (most recent call last)in <cell line: 5>()3 get_ipython().system(‘pip install transformers[sentencepiece]’)4----> 5 from transformers import pipeline67 model_id = “meta-llama/Meta-Llama-3.1-8B-Instruct”4 frames/usr/local/lib/python3.10/dist-packages/torch/init.py in762763 __name, __obj = ‘’, None→ 764 for __name in dir(C):765     if __name[0] != '’ and not __name.endswith(‘Base’):766all.append(__name)NameError: name ‘_C’ is not defined
2024-09-05T06:03:57Z
Restarting the kernel will solve the problem, maybe
2024-09-05T07:16:53Z
Hi,Thanks for the suggestion. I tried restarting the kernel but it still doesn’t seem to work.There’s another error in the following code snippet for text generationUse a pipeline as a high-level helperfrom transformers import pipelinemessages = [{“role”: “user”, “content”: “Who are you?”},]pipe = pipeline(“text-generation”, model=“meta-llama/Meta-Llama-3.1-8B-Instruct”)pipe(messages)I still get errors likeHTTPError                                 Traceback (most recent call last)/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py in REDACTED_TOKEN_for_status(response, endpoint_name)303     try:→ 304         response.raise_for_status()305     except HTTPError as e:18 framesHTTPError: 401 Client Error: Unauthorized for url:https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/resolve/main/config.jsonThe above exception was the direct cause of the following exception:GatedRepoError                            Traceback (most recent call last)GatedRepoError: 401 Client Error. (Request ID: Root=1-66d95297-65f74eaa1c17c1b55ae0ff7c;c1376837-e11c-4833-909c-9437d3f2b778)Cannot access gated repo for urlAccess to model meta-llama/Meta-Llama-3.1-8B-Instruct is restricted. You must be authenticated to access it.The above exception was the direct cause of the following exception:OSError                                   Traceback (most recent call last)/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs)418         if resolved_file is not None or not _raise_exceptions_for_gated_repo:419             return resolved_file→ 420         raise EnvironmentError(421             “You are trying to access a gated repo.\nMake sure to have access to it at "422             f"https://huggingface.co/{path_or_repo_id}.\n{str(e)}”OSError: You are trying to access a gated repo.Make sure to have access to it at401 Client Error. (Request ID: Root=1-66d95297-65f74eaa1c17c1b55ae0ff7c;c1376837-e11c-4833-909c-9437d3f2b778)
2024-09-05T08:05:04Z
I also meet the problem for the begin.You miss the token and you can go to the huggingface website → setting → Access Tokensimport os
os.environ['HF_TOKEN'] = 'my_value'

value = os.getenv('HF_TOKEN')
print(value)  # my_value
2024-09-05T08:33:08Z
Yeah…figured it out myself. Thanks. Works with token but additionally you need to request access to models and must be approved by the lllama forum .
2024-09-05T20:33:10Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.