Convert tensorflow tokenclassifier checkpoint to pytorch
Hi,Is there a way to convert the checkpoint of a Bert model that was fine-tuned for token classification task (NER) using tensorflow Bert original script?I am trying to use “transformer-cli convert” but it doesn’t work except on pretrained models.I want to share my fine-tuned models on models hub but it can’t be converted.Thanks,Maaly
2022-01-01T19:02:33Z
Hi,In HuggingFace, any model that has both an implementation in PyTorch and Tensorflow can be easily converted as follows:from transformers import BertForTokenClassification

model = BertForTokenClassification.from_pretrained("name_of_your_directory", from_tf=True)
2022-01-02T17:51:06Z
Hi,Thank you for your reply. I tried this approach and got this error:OSError: Error no file named [‘pytorch_model.bin’, ‘tf_model.h5’, ‘model.ckpt.index’, ‘flax_model.msgpack’] found in directory hub_models/ena_all_9010/anatomy/4e-05-90 orfrom_tfandfrom_flaxset to False.I was thinking of a way to convert the checkpoints of tensorflow fine-tuned model to pytorch as I did with the checkpoints for pretrained Bert models. Here are the files of one Bert fine-tuned model:checkpointconfig.jsonevaleval_results.txteval.tf_recordgraph.pbtxtlabel2id.pkllabel_test.txtmodel.ckpt-3000.data-00000-of-00001model.ckpt-3000.indexmodel.ckpt-3000.metamodel.ckpt-4000.data-00000-of-00001model.ckpt-4000.indexmodel.ckpt-4000.metamodel.ckpt-5000.data-00000-of-00001model.ckpt-5000.indexmodel.ckpt-5000.metamodel.ckpt-6000.data-00000-of-00001model.ckpt-6000.indexmodel.ckpt-6000.metamodel.ckpt-6316.data-00000-of-00001model.ckpt-6316.indexmodel.ckpt-6316.metaNER_result_conll.txtpredict.tf_recordtoken_test.txttrain.tf_recordI tried to convert the checkpoints using convert_bert_original_tf_checkpoint_to_pytorch.py but I got the following error:raise ModuleAttributeError("’{}’ object has no attribute ‘{}’".format(torch.nn.modules.module.ModuleAttributeError: ‘BertForPreTraining’ object has no attribute ‘bias’I tried to fix it by replacing line 33 with  model = BertForTokenClassification(config), then I got the following error:transformers/src/transformers/models/bert/modeling_bert.py", line 156, in load_tf_weights_in_bertif pointer.shape != array.shape:AttributeError: ‘NoneType’ object has no attribute ‘shape’investigating this issue more showed that pointer.shape != array.shape. Infact pointer = None for the following layers:[‘bert’, ‘pooler’, ‘dense’, ‘bias’] , where array.shape = (768,)[‘bert’, ‘pooler’, ‘dense’, ‘kernel’], where array.shape = (768, 768)[‘output_bias’], where array.shape = (7,)[‘output_weights’], where array.shape = (7, 768)I’d appreciate if someone can help me in figuring out how to fix the pointer for those layers.Thanks