Ebtihal commited on
Commit
048413f
·
1 Parent(s): 7cebf12

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -7
README.md CHANGED
@@ -11,17 +11,11 @@ widget:
11
 
12
  ---
13
  # Arabic BERT Model
14
- **AraBERTMo** is an Arabic pre-trained language model based on [Google's BERT architechture](https://github.com/google-research/bert).
15
- AraBERTMo_base uses the same BERT-Base config. AraBERTMo_base now comes in 10 new variants
16
- All models are available on the `HuggingFace` model page under the [Ebtihal](https://huggingface.co/Ebtihal/) name.
17
- Checkpoints are available in PyTorch formats.
18
-
19
  ## Pretraining Corpus
20
  `AraBertMo_base_V8' model was pre-trained on ~3 million words: [OSCAR](https://traces1.inria.fr/oscar/) - Arabic version "unshuffled_deduplicated_ar".
21
-
22
  ## Training results
23
  this model achieves the following results:
24
-
25
  | Task | Num examples | Num Epochs | Batch Size | steps | Wall time | training loss|
26
  |:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|
27
  | Fill-Mask| 40032| 8 | 64 | 5008 | 10h 5m 57s | 7.2367 |
 
11
 
12
  ---
13
  # Arabic BERT Model
14
+ **AraBERTMo** is an Arabic pre-trained language model based on [Google's BERT architechture](https://github.com/google-research/bert). AraBERTMo_base uses the same BERT-Base config. AraBERTMo_base now comes in 10 new variants All models are available on the `HuggingFace` model page under the [Ebtihal](https://huggingface.co/Ebtihal/) name. Checkpoints are available in PyTorch formats.
 
 
 
 
15
  ## Pretraining Corpus
16
  `AraBertMo_base_V8' model was pre-trained on ~3 million words: [OSCAR](https://traces1.inria.fr/oscar/) - Arabic version "unshuffled_deduplicated_ar".
 
17
  ## Training results
18
  this model achieves the following results:
 
19
  | Task | Num examples | Num Epochs | Batch Size | steps | Wall time | training loss|
20
  |:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|
21
  | Fill-Mask| 40032| 8 | 64 | 5008 | 10h 5m 57s | 7.2367 |