What is default value for number of labels in BertForSequenceClassification?
Hi,I try to rebuild the TFBertForSequenceClassification model from the plain TFBertModel.TFBertForSequenceClassification uses a dropout and a dense layer on top of the main BERT model. The number of neurons is config.num_labels.I wonder, where is this values because it seems it has no default value.And finetuning TFBertForSequenceClassification like here:huggingface.coTraining and fine-tuning — transformers 3.3.0 documentationdoes not handle a value for num_labels.So where does the 2 neurons come from in the last layer (by default)?Look here:huggingface.cotransformers.modeling_tf_bert — transformers 3.3.0 documentationthere isself.num_labels = config.num_labelsbut config.num_labels has no default value?See
2020-10-05T13:37:47Z
No it has no default value because it is supposed to be passed at init either because:the user is loading a pretrained model and its config has somenum_labelsthe user is passesnum_labels=nwhen using a generic pretrained model and it udpates the config.
2020-10-05T16:11:55Z
sgugger:the user is loading a pretrained model and its config has somenum_labelsBut as I posted I did not see where the value in the config is set… Where can I look what value is set?
2020-10-05T17:00:59Z
If you’re loading a pretrained model that has been finetuned for a classification task, you can look at the value in its config file with:from transformers import AutoConfig
config = AutoConfig.from_pretrained('checkpoint_name')
config.num_labels