berendg commited on
Commit
f919e34
·
1 Parent(s): 3bc69b5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -11,7 +11,7 @@ metrics:
11
 
12
  <!-- Provide a quick summary of what the model is/does. -->
13
 
14
- This medium-sized BERT model was created using the [Masked Latent Semantic Modeling] (MLSM) pre-training objective, which is a sample efficient alternative for classic Masked Language Modeling (MLM).
15
  During MLSM, the objective is to recover the latent semantic profile of the masked tokens, as opposed to recovering their exact identity.
16
  The contextualized latent semantic profile during pre-training is determined by performing sparse coding of the hidden representation of an already pre-trained model (a base-sized BERT model in this particular case).
17
 
 
11
 
12
  <!-- Provide a quick summary of what the model is/does. -->
13
 
14
+ This medium-sized BERT model was created using the [Masked Latent Semantic Modeling](https://aclanthology.org/2023.findings-acl.876/) (MLSM) pre-training objective, which is a sample efficient alternative for classic Masked Language Modeling (MLM).
15
  During MLSM, the objective is to recover the latent semantic profile of the masked tokens, as opposed to recovering their exact identity.
16
  The contextualized latent semantic profile during pre-training is determined by performing sparse coding of the hidden representation of an already pre-trained model (a base-sized BERT model in this particular case).
17