Why is use_cache incompatible with gradient checkpointing?
Hi,I see the below snippet in modeling_t5.py. I wanted to understand why use_cache is incompatible with gradient checkpointing.github.comhuggingface/transformers/blob/9aa230aa2fb17e4a7da4c61fccaddafe410f7ed1/src/transformers/models/t5/modeling_t5.py#L1008-L1011if use_cache:logger.warning("`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...")Thanks.
2022-06-08T05:46:28Z
Hi, there. I face the same problem inrun_clm.pywhen I set--gradient_checkpointing true. However I do not find any config that I can set onrun_clm.py, does anyone know?
2022-10-27T21:11:09Z
lianghsun:–gradient_checkpointing trueAnybody knows how to fix this?
2023-03-24T16:33:03Z
Honestly, I’ve just ignored it. It automatically disables the use_cache function. I’m about to remove the warning altogether.use_cache=Trueis incompatible with gradient checkpointing.Settinguse_cache=False
2023-07-07T12:44:16Z
its related topast_key_values, you can disable this warning by settingmodel.config.use_cache = False, when using gradient checkpointing during training.but during inference make sure to set it back toTrue.
2023-08-25T21:32:46Z
Could you please share what are the purposes ofuse_cache? Thanks.
2023-09-15T01:39:01Z
huggingface.coGenerationWe’re on a journey to advance and democratize artificial intelligence through open source and open science.