ValueError: The model is quantized with QuantizationMethod.QUANTO and is not serializable
Hi, I have quantized the model flan-t5-base to bit8, and I am trying to upload it to huggingface but I keep getting this error:ValueError: The model is quantized with QuantizationMethod.QUANTO and is not serializable - check out the warnings from the logger on the traceback to understand the reason why the quantized model is not serializable.Here’s the code:from transformers import T5Tokenizer, T5ForConditionalGeneration, QuantoConfig

quantization_config = QuantoConfig(weights="int8")
quantized_model = T5ForConditionalGeneration.from_pretrained(model_id, low_cpu_mem_usage=True, quantization_config=quantization_config)

quantized_model.push_to_hub("flan-t5-base-8bit")
2024-05-20T20:23:49Z
Same problem when trying to save the quantized model usingsave_pretrained