Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ metrics:
|
|
11 |
|
12 |
<!-- Provide a quick summary of what the model is/does. -->
|
13 |
|
14 |
-
This medium-sized BERT model was created using the [Masked Latent Semantic Modeling] (MLSM) pre-training objective, which is a sample efficient alternative for classic Masked Language Modeling (MLM).
|
15 |
During MLSM, the objective is to recover the latent semantic profile of the masked tokens, as opposed to recovering their exact identity.
|
16 |
The contextualized latent semantic profile during pre-training is determined by performing sparse coding of the hidden representation of an already pre-trained model (a base-sized BERT model in this particular case).
|
17 |
|
|
|
11 |
|
12 |
<!-- Provide a quick summary of what the model is/does. -->
|
13 |
|
14 |
+
This medium-sized BERT model was created using the [Masked Latent Semantic Modeling](https://aclanthology.org/2023.findings-acl.876/) (MLSM) pre-training objective, which is a sample efficient alternative for classic Masked Language Modeling (MLM).
|
15 |
During MLSM, the objective is to recover the latent semantic profile of the masked tokens, as opposed to recovering their exact identity.
|
16 |
The contextualized latent semantic profile during pre-training is determined by performing sparse coding of the hidden representation of an already pre-trained model (a base-sized BERT model in this particular case).
|
17 |
|