Issue: ERROR Failed to load the model. Traceback (most recent call last): File "W:\AI\modules\ui_model_menu.py",
I'm new to the site so Idk if this is the place to posts issues when trying to deploy this specific model.
But I seem to have an issue with loading this model. the error I get is:
22:53:37-726197 ERROR Failed to load the model.
Traceback (most recent call last):
File "W:\AI\modules\ui_model_menu.py", line 220, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\AI\modules\models.py", line 87, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\AI\modules\models.py", line 311, in AutoAWQ_loader
model = AutoAWQForCausalLM.from_quantized(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\AI\installer_files\env\Lib\site-packages\awq\models\auto.py", line 56, in from_quantized
return AWQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\AI\installer_files\env\Lib\site-packages\awq\models\base.py", line 208, in from_quantized
load_checkpoint_and_dispatch(
File "W:\AI\installer_files\env\Lib\site-packages\accelerate\big_modeling.py", line 545, in load_checkpoint_and_dispatch
load_checkpoint_in_model(
File "W:\AI\installer_files\env\Lib\site-packages\accelerate\utils\modeling.py", line 1494, in load_checkpoint_in_model
set_module_tensor_to_device(
File "W:\AI\installer_files\env\Lib\site-packages\accelerate\utils\modeling.py", line 276, in set_module_tensor_to_device
raise ValueError(f"{module} does not have a parameter or a buffer named {tensor_name}.")
ValueError: WQLinear_GEMM(in_features=14336, out_features=4096, bias=False, w_bit=4, group_size=128) does not have a parameter or a buffer named weight.
23:00:37-364382 INFO Received Ctrl+C. Shutting down Text generation web UI gracefully.
23:00:37-540648 INFO Received Ctrl+C. Shutting down Text generation web UI gracefully.
Exception ignored in atexit callback: <function dump_compile_times at 0x0000022AA0CF8360>
Traceback (most recent call last):
File "W:\AI\installer_files\env\Lib\site-packages\torch_dynamo\utils.py", line 247, in dump_compile_times
log.info(compile_times(repr="str", aggregate=True))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\AI\installer_files\env\Lib\site-packages\torch_dynamo\utils.py", line 234, in compile_times
out += tabulate(rows, headers=("Function", "Runtimes (s)"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\AI\installer_files\env\Lib\site-packages\torch_dynamo\utils.py", line 88, in tabulate
return tabulate.tabulate(rows, headers=headers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\AI\installer_files\env\Lib\site-packages\tabulate_init_.py", line 2129, in tabulate
numparses = expand_numparse(disable_numparse, len(cols))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "W:\AI\installer_files\env\Lib\site-packages\tabulate_init.py", line 2211, in _expand_numparse
if isinstance(disable_numparse, Iterable):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 119, in instancecheck
File "", line 123, in subclasscheck
File "", line 123, in subclasscheck
File "", line 123, in subclasscheck
[Previous line repeated 1 more time]
File "", line 409, in subclasshook
File "W:\AI\server.py", line 66, in signal_handler
sys.exit(0)
SystemExit: 0
Is there a solution for this?