Get warning "Could not estimate the number of tokens of the input, floating-point operations will not be computed" when use a customize Trainer and customize data collator
I want to use triplet loss to fine tune my Bert model, so I define a customize Trainer  like that:class TripletLossTrainer(Trainer):
        def compute_loss(self, model, inputs, return_outputs=False):
            anchor_data, positive_data, negative_data = inputs.values()
            anchor_output = self.use_avg_2(anchor_data, model)
            positive_output = self.use_avg_2(positive_data, model)
            negative_output = self.use_avg_2(negative_data, model)
            triplet_loss = nn.TripletMarginWithDistanceLoss(distance_function=lambda x, y: 1.0 - 
            F.cosine_similarity(x, y),margin=0.5)
            # compute custom loss
            loss = triplet_loss(anchor_output, positive_output, negative_output)
            return losswhere  anchor_data, positive_data, negative_data  are tokenized result with type <class ‘transformers.tokenization_utils_base.BatchEncoding’>, and the use _avg_2 function is for computing the average value for the last hidden state for each data with a Bert model.When I do the training process, it will keep reporting warning likeCould not estimate the number of tokens of the input, floating-point operations will not be computedCould not estimate the number of tokens of the input, floating-point operations will not be computedCould not estimate the number of tokens of the input, floating-point operations will not be computedI wonder what does the floating-point operations mean? will it influence the performance of my training? Thanks!
2022-05-31T19:04:45Z
You can completely ignore that warning, it’s just to compute the total FLOPS done during your training.
2022-06-01T01:17:08Z
Thanks a lot! Is there any method to stop outputting these warn information during the training?
2022-06-01T13:54:58Z
No, but in the latest version you should see it only once.
2022-09-30T19:24:17Z
Hi, I have the same problem when using the framework of huggingface to compute a contrastive loss. In my case, this warning information is repeatedin the console which really makes me a big headache.The version of my huggingface is4.17.0. So how can I stop this information?
2022-11-30T21:50:11Z
Me too, on 4.23.1:Screen Shot 2022-11-30 at 4.49.02 PM1920×1079 431 KB