Can I use "AutoModel For Sequence Classification" class for generative models?
I want to use AutoModelForSequenceClassification with a llama 7b modelHow will the input flow in the model if I load the model with this class ?
2023-09-20T15:06:00Z
Yes it works, I used a GPT2-XL to do NLU tasks.But it also depends on the weights you use whether it has a class calledtransformers.XXXForSequenceClassification, where XXX is the model class/ name.Such as here:https://github.com/huggingface/transformers/blob/v4.33.2/src/transformers/models/gpt2/modeling_gpt2.py#L1376
2024-04-15T09:24:35Z
Hi, did you try this? Does it give better results as compared to the BERT models? I am trying the same, but the model is overfitting on the training dataset.