Tutorial notebooks
Notebooks are now automatically created from the tutorials in the documentation of transformers. you can find them allhereor click on the brand new “Open in colab” button on the doc pages that have one.Plan is to add more tutorials and the corresponding notebooks very soon. I’ll focus on integration with nlp datasets and the Trainer in the next few weeks, but if you have ideas of tutorials you’d like to see, please let me know here.
2020-08-10T15:46:54Z
Awesome!  Thanks@sgugger
2020-08-10T16:02:05Z
I would like to see a tutorial about integrating nlp with train a LM from scratch
2020-08-11T17:53:31Z
Very excited for nlp datasets and the Trainer as I’ve had difficulty implementing them after the recent changes to load_dataset()
2020-08-11T22:20:18Z
If I’m correct, the examples are planned to be converted so that they will use the nlp datasets package.@mrm8488@rbin
2020-08-12T01:27:40Z
Yes, I am waiting for it!
2020-08-12T20:16:16Z
@mrm8488, this may not be exactly what you’re looking for but I have been finetuning GPT2 with a language model head using my own loss function. I tried to detail inthis postas much of the implementation as possible. Maybe parts of it will be useful for you as well since I am also usingTrainer.
2020-08-13T07:46:24Z
Thank you so much! I will give a try!
2020-08-24T15:08:01Z
Hi, I think an example of multi-task learning would be great. Especially one with different loss functions for each task.
2022-02-14T19:27:45Z
do you have any multi-task learning code for reference?