Target size (torch.Size([8])) must be the same as input size (torch.Size([8, 2]))
Hello all,I am aware of all the solutions which are discussed here previously regarding the same problem but still I had no luck with those solutions.I’m trying to implement abinary classifier. I’m using is acustomized datasetand having one text column with german text data and the label column has two classes either 0 or 1.I’m using here thedeepset/gbert-basemodel and number of labels as2.I have followed the official tutorial of hugging faceA full training - Hugging Face NLP CourseI’m getting everything similar till the step:outputs = model(**batch)I have tried the following work arounds suggested in this forum and other coding forums. Which are mentioned below:I checked thepytorch version(Suggested by online forums : to update the pytorch version which are below verison 2) and I’m using the following:2.0.0+cu118The labels are of thefloat typeand does not contain anynull value(Suggested by online forums : to check if the data type of labels is float as the model expect it in that format)Also tried to change thelabel shapefrom [0] and [1] to [1,0] for class zero and [0,1] for class 1 because the error says the input from the model to the loss function is of size [16,2] and the target size which are labels here are of size [16] . But changing the shape from [0] and [1] to [1,0] for class zero and [0,1] for class 1 also did not solve the problem.I tried to implement throughTrainer APIfollowing the official tutorial of hugging faceFine-tuning a model with the Trainer API - Hugging Face NLP Courseand tried tocustomize the loss functionfrombinary_cross_entropy_with_logits to nn.CrossEntropyLoss(). Just tried to change the loss function to see if the code runs but ended up with the same error.Code:> from transformers import AutoTokenizer, DataCollatorWithPadding
> tokenizer = AutoTokenizer.from_pretrained("deepset/gbert-base")
> 
> def tokenize_function(examples):
>     return tokenizer(examples["text1"], truncation=True)
> 
> tokenized_datasets = final_dataset_dict.map(tokenize_function, batched=True)
> data_collator= DataCollatorWithPadding(tokenizer)
> tokenized_datasets = tokenized_datasets.remove_columns(["text1"])
> tokenized_datasets["train"].column_names
> tokenized_datasets.set_format("torch")
> 
> from torch.utils.data import DataLoader
> 
> train_dataloader = DataLoader(tokenized_datasets["train"], shuffle = True, batch_size = 8, collate_fn = data_collator)
> eval_dataloader = DataLoader(tokenized_datasets["unsupervised"], batch_size = 8, collate_fn = data_collator)
> 
> for batch in train_dataloader:
>   break
> print({k: v.shape for k, v in batch.items()})
> #print(batch)
> 
> from transformers import AutoModelForSequenceClassification
> checkpoint = "deepset/gbert-base"
> model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels =2)
> 
> outputs = model(**batch)
> print(outputs.loss, outputs.logits.shape)After tokenization my data looks like this :> DatasetDict({
>     train: Dataset({
>         features: ['text1', 'labels', 'input_ids', 'token_type_ids', 'attention_mask'],
>         num_rows: 2512
>     })
>     test: Dataset({
>         features: ['text1', 'labels', 'input_ids', 'token_type_ids', 'attention_mask'],
>         num_rows: 1255
>     })
>     validation: Dataset({
>         features: ['text1', 'labels', 'input_ids', 'token_type_ids', 'attention_mask'],
>         num_rows: 1255
>     })
> })The batch items in thetrain_dataloaderlooks like this.{'labels': torch.Size([8]), 'input_ids': torch.Size([8, 69]), 'token_type_ids': torch.Size([8, 69]), 'attention_mask': torch.Size([8, 69])}The detailed error is as follows:> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> <ipython-input-36-b84c8f6552ab> in <cell line: 1>()
> ----> 1 outputs = model(**batch)
>       2 #print(outputs.shape)
>       3 print(outputs.loss, outputs.logits.shape)
> 
> 4 frames
> /usr/local/lib/python3.9/dist-packages/torch/nn/functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
>    3161 
>    3162     if not (target.size() == input.size()):
> -> 3163         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
>    3164 
>    3165     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)
> 
> ValueError: Target size (torch.Size([8])) must be the same as input size (torch.Size([8, 2]))Any lead from this problem will be very much appreciated.
2023-05-10T08:50:39Z
Can anybody please give any idea why such error is happening ?I also tried other German BERT model But getting the same error.nlptown/bert-base-multilingual-uncased-sentimentpapluca/xlm-roberta-base-language-detectionoliverguhr/german-sentiment-bertAny lead will be a big help.
2023-05-10T10:29:09Z
Changing the label datatype to integer solved the problem.df[‘labels’] = df[‘labels’].astype(int)
2023-09-05T01:27:10Z
Thank you very much for the solution! I was getting the same error and spent 2 hours debugging, and finally found a fix, thank u! Changing the label datatype did work for me
2023-10-09T14:28:38Z
Thanks for sharing this ! It also solved my issue ! I just had to convert the labels from float to int !
2023-10-13T16:12:33Z
What if the labels are already typed as int64I have the same error for sequence classification with BERT model, even my labels start from 0, here’s what my training dataset looks likeDataset({
    features: ['label', 'text', 'input_ids', 'token_type_ids', 'attention_mask'],
    num_rows: 393
})