RuntimeError: a leaf Variable that requires grad is being used in an in-place operation
Hi there, i’am trying to finetune JAIS-13B and LLM for arabic on a custom summarization dataset and when i runtrainer.train()i got the ERROR below ! any help plzRuntimeError                              Traceback (most recent call last)in <cell line: 2>()1 trainer.deprecated=True----> 2 trainer.train()21 frames~/.cache/huggingface/modules/transformers_modules/inception-mbzuai/jais-13b-chat/96080d1c163804428c4792b8618c2d39661e9d7f/modeling_jais.py in forward(self, input_ids, past_key_values, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, use_cache, output_attentions, output_hidden_states, return_dict)867         else:868             hidden_states = inputs_embeds→ 869         hidden_states *= torch.tensor(870             float(self.embeddings_scale), dtype=hidden_states.dtype, device=hidden_states.device871         )RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.
2023-09-18T16:00:01Z
@sguggerany help from your side plz !?
2023-09-25T14:01:15Z
You can change the code of modeling_jais.py from:hidden_states *= torch.tensor(float(self.embeddings_scale), dtype=hidden_states.dtype, device=hidden_states.device)to:scale_factor_hidden = torch.tensor(float(self.embeddings_scale), dtype=hidden_states.dtype, device=hidden_states.device)hidden_states = hidden_states * scale_factor_hidden
2023-09-25T17:59:28Z
i will make a pull request and hopefully the authors would accept it, Thank you for your help
2023-11-14T15:22:54Z
i remember it worked ! otherwise you can check the Asas AI version :https://huggingface.co/asas-ai/jais-13b-chat-8bit