
A small version of `DeBERTa` trained on the clean version of google C4 dataset. For more info about the size of the model, see `config.json`.

The model has been trained for **100K** steps with a batch size of **2048** and a sequence length of **512**, for a total of **104B** tokens.

The vocabulary and the tokenizer are the same as `microsoft/deberta-base`.
{"language": ["en"], "datasets": ["c4"], "metrics": ["accuracy"], "pipeline_tag": "fill-mask"}
transformers pytorch deberta fill-mask en dataset:c4 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #deberta #fill-mask #en #dataset-c4 #autotrain_compatible #endpoints_compatible #region-us 


A small version of 'DeBERTa' trained on the clean version of google C4 dataset. For more info about the size of the model, see 'URL'.

The model has been trained for 100K steps with a batch size of 2048 and a sequence length of 512, for a total of 104B tokens.

The vocabulary and the tokenizer are the same as 'microsoft/deberta-base'.
TAGS
#transformers #pytorch #deberta #fill-mask #en #dataset-c4 #autotrain_compatible #endpoints_compatible #region-us 

TAGS
#transformers #pytorch #deberta #fill-mask #en #dataset-c4 #autotrain_compatible #endpoints_compatible #region-us 
