Online Machine Learning for transformers
I was wondering if there is any library that is compatible to train transformers in a incremental manner. Just like River and Creme are there to support many Supervised and Unsupervised learning task in a online manner; is there any library for transformers too. Any advice will be highly appreciated.
2023-02-07T10:27:21Z
Hi,That’s exactly the purpose of theonline language modeling (OLM)projectThe repo can be found here:GitHub - huggingface/olm-training: Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.