dtamayo commited on
Commit
8977e3b
·
verified ·
1 Parent(s): aef8e21

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -225,8 +225,8 @@ This adjustment resulted in a total of 2.68 trillion tokens, distributed as outl
225
 
226
  ![lang distrib](./images/corpus_languages_1.1.png)
227
 
228
- The pretraining corpus is predominantly composed of data from Colossal OSCAR, which contributes a significant 53,05% of the total tokens.
229
- Following this, Starcoder provides 13,67%, and FineWeb-Edu (350BT subset) adds 10,24%. The next largest sources are HPLT at 4,21% and French-PD at 3,59%.
230
  Other notable contributions include MaCoCu, Legal-ES, and EurLex, each contributing around 1.72% to 1.41%.
231
  These major sources collectively form the bulk of the corpus, ensuring a rich and diverse dataset for training the language model.
232
  The remaining 10% comes from smaller sources in various languages.
 
225
 
226
  ![lang distrib](./images/corpus_languages_1.1.png)
227
 
228
+ The pretraining corpus is predominantly composed of data from Colossal OSCAR, which contributes a significant 53.05% of the total tokens.
229
+ Following this, Starcoder provides 13.67%, and FineWeb-Edu (350BT subset) adds 10.24%. The next largest sources are HPLT at 4.21% and French-PD at 3.59%.
230
  Other notable contributions include MaCoCu, Legal-ES, and EurLex, each contributing around 1.72% to 1.41%.
231
  These major sources collectively form the bulk of the corpus, ensuring a rich and diverse dataset for training the language model.
232
  The remaining 10% comes from smaller sources in various languages.