Removal of assert from phi-3-small init
When enabling phi-3-small on non-cuda devices, the assert of flash_attn package’s availability ininitmakes it difficult to run w/o modeling script changes. Can we replace the assert with a warning message? In this way it offers a chance for modeling to be replaced afterwards with alternative implementations of flash_attn.huggingface.comodeling_phi3_small.py · microsoft/Phi-3-small-8k-instruct at mainWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-08-19T12:00:21Z
Hi,Phi-3 is available natively in the Transformers library, hence please usefrom_pretrainedwithout specifyingtrust_remote_code=True.Docs:Phi-3
2024-08-20T07:40:53Z
Thanks Niels. With transformers.from_pretrained(trust_remote_code=False), phi-3-small seems insistently asking to set it as True. Do you know any plan for transformers to support phi3-small natively?  Seeking for a way to mitigate the assertion in theinitfunction of the original modeling script.Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linuxType “help”, “copyright”, “credits” or “license” for more information.import torchfrom transformers import AutoModelForCausalLMmodel=“microsoft/Phi-3-small-8k-instruct”model = AutoModelForCausalLM.from_pretrained(model, torch_dtype=“auto”, trust_remote_code=False)config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.57k/1.57k [00:00<00:00, 15.7MB/s]Traceback (most recent call last):File “”, line 1, inFile “/home/xfeng8/miniforge3/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py”, line 524, in from_pretrainedconfig, kwargs = AutoConfig.from_pretrained(File “/home/xfeng8/miniforge3/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py”, line 979, in from_pretrainedtrust_remote_code = resolve_trust_remote_code(File “/home/xfeng8/miniforge3/lib/python3.10/site-packages/transformers/dynamic_module_utils.py”, line 640, in resolve_trust_remote_coderaise ValueError(ValueError: Loading microsoft/Phi-3-small-8k-instruct requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the optiontrust_remote_code=Trueto remove this error.$ pip list|grep transformerstransformers                4.44.0