How much data is required to pretrain a model
Hello, I am looking to find an estimate of the amount of data generally required to pre-train a transformers model? Is there models which are less data intensive?