Update README.md
Browse files
README.md
CHANGED
@@ -11,17 +11,11 @@ widget:
|
|
11 |
|
12 |
---
|
13 |
# Arabic BERT Model
|
14 |
-
**AraBERTMo** is an Arabic pre-trained language model based on [Google's BERT architechture](https://github.com/google-research/bert).
|
15 |
-
AraBERTMo_base uses the same BERT-Base config. AraBERTMo_base now comes in 10 new variants
|
16 |
-
All models are available on the `HuggingFace` model page under the [Ebtihal](https://huggingface.co/Ebtihal/) name.
|
17 |
-
Checkpoints are available in PyTorch formats.
|
18 |
-
|
19 |
## Pretraining Corpus
|
20 |
`AraBertMo_base_V8' model was pre-trained on ~3 million words: [OSCAR](https://traces1.inria.fr/oscar/) - Arabic version "unshuffled_deduplicated_ar".
|
21 |
-
|
22 |
## Training results
|
23 |
this model achieves the following results:
|
24 |
-
|
25 |
| Task | Num examples | Num Epochs | Batch Size | steps | Wall time | training loss|
|
26 |
|:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|
|
27 |
| Fill-Mask| 40032| 8 | 64 | 5008 | 10h 5m 57s | 7.2367 |
|
|
|
11 |
|
12 |
---
|
13 |
# Arabic BERT Model
|
14 |
+
**AraBERTMo** is an Arabic pre-trained language model based on [Google's BERT architechture](https://github.com/google-research/bert). AraBERTMo_base uses the same BERT-Base config. AraBERTMo_base now comes in 10 new variants All models are available on the `HuggingFace` model page under the [Ebtihal](https://huggingface.co/Ebtihal/) name. Checkpoints are available in PyTorch formats.
|
|
|
|
|
|
|
|
|
15 |
## Pretraining Corpus
|
16 |
`AraBertMo_base_V8' model was pre-trained on ~3 million words: [OSCAR](https://traces1.inria.fr/oscar/) - Arabic version "unshuffled_deduplicated_ar".
|
|
|
17 |
## Training results
|
18 |
this model achieves the following results:
|
|
|
19 |
| Task | Num examples | Num Epochs | Batch Size | steps | Wall time | training loss|
|
20 |
|:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|
|
21 |
| Fill-Mask| 40032| 8 | 64 | 5008 | 10h 5m 57s | 7.2367 |
|