Fine tune Transformers for text generation
HelloIs there an example like this one (Fine-tune a pretrained model) for fine tuning HF transformers for text generation?
2022-04-04T10:10:26Z
@mwitiderrickHelloYou can check outthis linkfor all example notebooks.
2022-04-04T15:01:51Z
Hello@merve, thanks for the response, indeed I found the notebooks very useful. One follow-up question.When I run predictions like this for a binary problemimport tensorflow as tf
predicted_class_id = int(tf.math.argmax(logits, axis=-1)[0])
bert.config.id2label[predicted_class_id]I get the result asLABEL_1, How do I know if this is the prediction for class 0 or 1.Thanks.
2022-04-04T15:46:45Z
Hello Derrick,Can you send me the model repo so that I can see the config file?
2022-04-04T17:01:15Z
Hello@merveNot sure about the repo but the model isfrom transformers import TFAutoModelForSequenceClassification
model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)
2022-04-05T08:13:14Z
Hello Derrick,Sorry it’s my fault, label is already a labelI meant, which dataset is the model fine-tuned on?
2022-04-05T08:46:22Z
Hello@merveIt’simdbdataset = load_dataset("imdb")
2022-04-06T05:10:23Z
Any update on this@merve?
2022-04-06T10:16:15Z
@mwitiderrickIn thepage of the datasetyou can see the label for 1 is positive.
2022-04-06T12:01:40Z
merve:page of the datasetWanted to clarify thatLABEL_1means label 1 and not 0
2023-05-22T17:11:10Z
hi@mwitiderrick, which HF notebook did you use for fine-tuning a model for text generation? thank you, Tom
2023-07-27T06:08:09Z
Hi@mwitiderrickI am a beginner I was trying to fine-tune the model to generate text I will be a great help if you will be able to provide your notebook to me.I am ready to use any model of LlamaForCasualLM architecture any dataset will be fine for me I just want to understand the Concept and practical Implementation Behind that.