Can not resume my endpoints, always receiving download Error
I have deployed an endpoit with mistralai/Mistral-7B-v0.1, and was able to send requst to three days ago. But today when I tryied to resume the endpoint it kept failing, returning download error.What can be the problem? Thank you!the following is what I get:2023/12/14 13:51:12 ~ {“timestamp”:“2023-12-14T05:51:12.753891Z”,“level”:“ERROR”,“fields”:{“message”:“Download encountered an error: Traceback (most recent call last):\n\n File "/opt/conda/bin/text-generation-server", line 8, in \n sys.exit(app())\n\n File "/opt/conda/lib/python3.9/site-packages/text_generation_server/cli.py", line 156, in download_weights\n local_pt_files = utils.weight_files(model_id, revision, ".bin")\n\n File "/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/hub.py", line 90, in weight_files\n raise FileNotFoundError(\n\nFileNotFoundError: No local weights found in /repository with extension .bin\n\n”},“target”:“text_generation_launcher”,“span”:{“name”:“download”},“spans”:[{“name”:“download”}]}2023/12/14 13:51:41 ~ {“timestamp”:“2023-12-14T05:51:41.931879Z”,“level”:“INFO”,“fields”:{“message”:“Starting download process.”},“target”:“text_generation_launcher”,“span”:{“name”:“download”},“spans”:[{“name”:“download”}]}2023/12/14 13:51:41 ~ {“timestamp”:“2023-12-14T05:51:41.931740Z”,“level”:“INFO”,“fields”:{“message”:“Args { model_id: "/repository", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 4096, max_total_tokens: 4097, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, hostname: "sxresearch16b4-mistral7b-768978877c-xv4vv", port: 80, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/data"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: true, otlp_endpoint: None, cors_allow_origin:, watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, env: false }”},“target”:“text_generation_launcher”}2023/12/14 13:51:44 ~ Error: DownloadError
2024-04-10T14:14:34Z
Same issue here!Server message:Endpoint failed to start. 319Z",“level”:“ERROR”,“fields”:{“message”:“Download encountered an error: \nTraceback (most recent call last):\n\n File "/opt/conda/lib/python3.10/site-packages/text_generation_server/utils/peft.py", line 15, in download_and_unload_peft\n model = AutoPeftModelForCausalLM.from_pretrained(\n\n File "/opt/conda/lib/python3.10/site-packages/peft/auto.py", line 72, in from_pretrained\n peft_config = PeftConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)\n\n File "/opt/conda/lib/python3.10/site-packages/peft/config.py", line 151, in from_pretrained\n return cls.from_peft_type(**kwargs)\n\n File "/opt/conda/lib/python3.10/site-packages/peft/config.py", line 118, in from_peft_type\n return config_cls(**kwargs)\n\nTypeError: LoraConfig.init() got an unexpected keyword argument ‘layer_replication’\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File "/opt/conda/bin/text-generation-server", line 8, in \n sys.exit(app())\n\n File "/opt/conda/lib/python3.10/site-packages/text_generation_server/cli.py", line 225, in download_weights\n utils.download_and_unload_peft(\n\n File "/opt/conda/lib/python3.10/site-packages/text_generation_server/utils/peft.py", line 23, in download_and_unload_peft\n model = AutoPeftModelForSeq2SeqLM.from_pretrained(\n\n File "/opt/conda/lib/python3.10/site-packages/peft/auto.py", line 72, in from_pretrained\n peft_config = PeftConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)\n\n File "/opt/conda/lib/python3.10/site-packages/peft/config.py", line 151, in from_pretrained\n return cls.from_peft_type(**kwargs)\n\n File "/opt/conda/lib/python3.10/site-packages/peft/config.py", line 118, in from_peft_type\n return config_cls(**kwargs)\n\nTypeError: LoraConfig.init() got an unexpected keyword argument ‘layer_replication’\n”},“target”:“text_generation_launcher”,“span”:{“name”:“download”},“spans”:[{“name”:“download”}]} Error: DownloadError