Create custom data_collator for Huggingface Trainer
I need to create a customdata_collatorfor finetuning with HuggingfaceTrainer API.HuggingFace offersDataCollatorForWholeWordMaskfor masking whole words within the sentences with a given probability.model_ckpt    = "vinai/bertweet-base"
tokenizer     = AutoTokenizer.from_pretrained(model_ckpt, normalization=True)
data_collator = DataCollatorForWholeWordMask(tokenizer=tokenizer, mlm_probability=args.mlm_prob)I am using the collator asTrainerargument:trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=tokenized_dataset['train'],
        eval_dataset=None,
        data_collator=data_collator,
    )
    trainer.train()But in my usecase, a sample input looks like:<sent1>.<sent2>. I want to mask the tokens only in<sent2>and not in<sent1>. How can I go about it? Any pointers on getting started with it are also welcome.
2022-07-22T15:00:31Z
Subclass DataCollatorForWholeWordMask and override thetorch_mask_tokensfunction.