"normal_kernel_cpu" not implemented for 'Char' when trying to import 8-bit model
Didn’t see this issue anywhere else while googling, so posting here; tried loading an 8-bit SFTT model as part of a TRL-based reinforcement learning pipeline. Returned the error as stated in the title:RuntimeError: "normal_kernel_cpu" not implemented for 'Char'Model config is as follows, with the loading line throwing the error:config = PPOConfig(
    model_name="drive/MyDrive/fg_proj/lr_pipeline/sftt_models/feb7_2024_8bit",
    learning_rate=1.41e-5,
    log_with="tensorboard",#"wandb",
    project_kwargs={"logging_dir": "run_logs_107_3"}
)

model = AutoModelForCausalLMWithValueHead.from_pretrained(config.model_name)My hunch is this related to loading in 8-bit, since the full-bit and 4-bit models I tried did not throw this error. Any ideas as to what I’d be missing for this to occur would be helpful.
2024-02-29T09:14:49Z
Received similar error. when i try to load the 8 bit quantized model from local system.I tried it for owl-vit model.
2024-04-30T13:42:17Z
I receive the same error, also when loading in with 8-bit but not when loading in with 4-bit…
2024-05-11T11:30:06Z
I have this problem too
2024-07-02T00:45:20Z
Same problem. Someone please help
2024-08-20T11:35:54Z
I am also having this problem trying to load longformer with a bitsandbytes config.