What FineTuning can be done with a available models
Hi, I am new to transformers and have a general question although I will ask it to my task at hand. I need to classify amino acid sequences and wanted to use the ProtBERT model to do so. This means I want to add a classification head at the end of the model.Having seen the tutorial , I have seen how to do this using DistilBertForSequenceClassification. I was wondering if such a module is available for all language models i.e is there ProtBertForSequenceClassification or is this available for only specific language models
2021-02-21T15:05:35Z
It seems theProtBERTcheckpoints use the bert architecture so you can use them withBertForSequenceClassification.In general you can just tryAutoModelForSequenceClassification.from_pretrained(checkpoint_name)and it will return an error if there is no sequence classification model in the library for the architecture you are using withcheckpoint_name.
2021-02-21T15:46:31Z
Thank you!
2021-02-23T12:06:12Z
Can this be used for MultiClass classification or only for 2?I am running into a  this error:[CUDA error: CUBLAS_STATUS_ALLOC_FAILED when callingcublasCreate(handle)]and have found that this may be an issue due to an inconsistency between number of labels and number of output units.
2021-02-23T13:41:38Z
The number of labels can be specified withnum_labels=xxxin your call tofrom_pretrained.