Generate constraint words within the output sentence and not at its start/end
OverviewI am new to the transformers-library and I want to generate text using GPT2 while influencing its output. This I do via thePhrasalConstraintthat lets me add certain tokens that are to apear in the sentence generated by GPT2.Mainly I oriented myself onthis blogpost.ProblemNow the problem is, that the constraint words will almost certainly appear after the input or at the end of the generated sentence. Never in the middle of the sentence.→ I suspect that the constraints do not fit the generated sentence, and are therfore “discarded” to the end/ beginning, to keep up the overall probability of the generation.Exampleinput: I like toconstraint: Franceoutput_1 → I like to France is a beautiful countyoutput_2 → I like to become a carpenter when I grow up FranceDoes anybody know how I can improve this? Is there a way to make the constrains fit more natural into the generated sentence? Or is there another way to infuse constraints into the generation, that you are aware of?Thanks for any help!Codefrom transformers import AutoModelForCausalLM, AutoTokenizer, PhrasalConstraint
tokenizer = AutoTokenizer.from_pretrained("dbmdz/german-gpt2",add_prefix_pace=True)
# the model is a german version of gpt2
model = AutoModelForCausalLM.from_pretrained("dbmdz/german-gpt2")
tokenizer.add_special_tokens({"pad_token": "[PAD]"})

encoder_input_str = " Im Krankenhaus musste ihr ein"
force_words = [" Morphium"]

constraints = []
for constraint_entitiy in force_words:
    constraints.append(PhrasalConstraint(tokenizer(constraint_entitiy).input_ids))

input_ids = tokenizer(encoder_input_str, return_tensors="pt").input_ids
force_words_ids = tokenizer(force_words, add_special_tokens=False).input_ids

outputs = model.generate(
    input_ids,
    constraints = constraints,
    num_beams=10,
    num_return_sequences=1,
    no_repeat_ngram_size=1,
    min_length=20,
)
print(tokenizer.decode(outputs[0]))
2022-09-16T13:13:51Z
There seems to be a bug that causes this above transformers==4.20.1If you downgrade it should work as described in the tutorial linked above.I will write an Issue on this…→ Issue:PhraseConstraints apearing only directly after input or at the end of the generated sentence · Issue #19070 · huggingface/transformers · GitHub