Sequence to Sequence Modelling Doubt
Sequence to Sequence Modelling - Repitition and uncontrolled generationHi guys,I am interested in building a description/caption generator using a template based approach.I have a rule based method which converts an input sentence into a template as shown below.InputTemplateShaped to a shirt silhouette, it’s easy to wearShaped to a SILHOUETTE, it’s easy to wearThe paisley print will look and make you feel amazingThe PRINT will look and make you feel amazingNow, I can fill in these templates to obtain sentences by filling in the placeholders like SILHOUETTE with tags pertaining to SILHOUETTE likeshift,aline,bodyconetc.There’s 215 templates and after filling in the placeholders, I have 215 * 3 = 645 total sentences. The goal is to now formulate this as a sequence to sequence problem with the following i/p and o/pInput Sequence: <start> <tag> <tag> <tag> <end>Output Sequence: <start> sentence with tags <end>A few examples of the same are as follows:InputFilled Template<start> shirt <end><start> Shaped to a shirt silhouette, it’s easy to wear <end><start> paisley <end><start> The paisley print will look and make you feel amazing<end><start> thin puff <end><start> This dress features thin straps and puff sleeves <end>I trained aT5-basemodel withsimple_transformerswith the following model hyper-parameters{
    "reprocess_input_data": True,
    "overwrite_output_dir": True,
    "max_seq_length": 128,
    "train_batch_size": 8,
    "num_train_epochs": 5,
    "save_eval_checkpoints": True,
    "save_steps": -1,
    "use_multiprocessing": False,
    "evaluate_during_training": True,
    "evaluate_during_training_steps": 15000,
    "evaluate_during_training_verbose": True,
    "fp16": True,
    "num_beams": None,
    "do_sample": True,
    "max_length": 50,
    "top_k": 50,
    "top_p": 0.95,
    "num_return_sequences": 3,
}However in the results, I observed that some results are alright as followshttps://imgur.com/nWlyieObut in many results, there’s a lot of repetitionhttps://imgur.com/cZaSVB0and in some other results, there’s inclusion of tags which are not intentional i.e. they were not present in the input but still somehow appear in the output.[['Covered in a blue color, this mini length dress will brighten up your summer season wardrobe, styled in a holiday silhouette and SHAPE with a holiday sleeve of a minimum of 10 inch',
  'Printed all over with blue color mini length dress, this holiday dress is designed in a holiday silhouette that falls to a HEM',
  'Covered in a blue color, this mini length dress will brighten up your summer season wardrobe, styled in a holiday silhouette and SHAPE with a holiday sleeve and sleeve holiday']]If we see in the above prompt, we provided the color of dress, it’s length and occasion; however, season was added in along with other tags just out of the blue, why would this generation be uncontrolled when we’re providing the model with properly curated data without extraneous tags in the output?Also in some results, one attribute will be rampantly repeated and the description would then make no sense for eg. in the following sentence, length is repeatedly predicted as a word so many times in the description…[['Put some swing in this mini length length length length length length length length dress',
'Put some swing in this mini length length length length length length length length length length formal dress']]I am just beginning my journey with transformer based NLP and would appreciate if someone could clarify why this might be happening and how could it be circumvented.Thanks & Regards,Vinayak.