Cardinality issue when training bert from scratch (tensorflow)
Hello there!I am trying to adapt the officialGoogle Colabfor language generation totensorflowand everything seems to work wonderfully by simply appendingTFto most of thehuggingfacefunction calls (TFAutoModel, etc)Unfortunately, this strategy fails at thetrainingstep:from transformers import TFTrainer, TFTrainingArguments
import tensorflow as tf

training_args = TFTrainingArguments(
    "test-clm",
    evaluation_strategy = "epoch",
    learning_rate=2e-5)

trainer = TFTrainer(
    model=model,
    args = training_args,
    train_dataset=lm_datasets[0:1000],
    eval_dataset=lm_datasets[1000:])

trainer.train()

 self.num_train_examples = self.train_dataset.cardinality().numpy()
AttributeError: 'dict' object has no attribute 'cardinality'I have absolutely no idea what thiscardinalityis. Do you know what the issue can be?Thanks!
2021-08-29T00:20:40Z
I saw a related issue hereTensorFlow Question-Answering example fails to run (cardinality error) · Issue #10246 · huggingface/transformers · GitHub… huggingface masters, do you have an idea?Thanks!
2021-08-29T20:56:28Z
@BramVanroysorry to pull you in, but I wonder if you have any clues about what is happening here? is this a tensorflow specific issue?
2021-08-30T06:50:23Z
Seems like a potential bug. I’d recommend you to create a bug report on Github. But please provide the whole error trace and not just the last line, thanks.