How can i skip GPT2LMHeadModel embedding layers?
I am using GPT2LMHeadModel model but want to skip embedding layers of this model, and i will also be using the model.generate function for text generation task. is there any way to do this?My purpose to skip embedding is to use embedding vectors from some other model.
2023-02-15T10:37:40Z
Hey@dharmendraSoft prompting was added very recently! You can read it in more detail onthis GitHub comment, but TL;DRInstall transformers from main, as this feature will only be in transformers v4.27 (pip install --upgrade git+https://github.com/huggingface/transformers.git)You can now passinputs_embedsto.generate(), which will be used as the prompt.from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")

text = "Hello world"
input_ids = tokenizer.encode(text, return_tensors="pt")

# Traditional way of generating text
outputs = model.generate(input_ids)
print("\ngenerate + input_ids:", tokenizer.decode(outputs[0], skip_special_tokens=True))

# From inputs_embeds -- exact same output if you also pass `input_ids`. If you don't
# pass `input_ids`, you will get the same generated content but without the prompt
inputs_embeds = model.transformer.wte(input_ids)
outputs = model.generate(input_ids, inputs_embeds=inputs_embeds)
print("\ngenerate + inputs_embeds:", tokenizer.decode(outputs[0], skip_special_tokens=True))
2023-02-16T07:06:13Z
I have transformers of version 2.11.0 and i gave “inputs_embed” as a parameter and it generate some sequence without any error, so i think it works on transformers below the version you mentioned.thank you very much for the response, it saved lot of my crucial time , thanks again
2023-02-17T06:37:25Z
one more question, I don’t want to use ‘generate’ function with pre-trained model but with my custom model.I tried to import GenerationMixIn class and use the generation function but it says import error.can you plz suggest some way to deal with this issue.
2023-02-17T12:42:08Z
@dharmendraIf your model inherits fromPreTrainedModel, it should work out of the box. If it doesn’t work, please open an issue intransformers