bclavie commited on
Commit
e5629bf
1 Parent(s): 11aa1a9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md CHANGED
@@ -41,6 +41,12 @@ It is available in the following sizes:
41
 
42
  You can use these models directly with the `transformers` library. Since ModernBERT is a Masked Language Model (MLM), you can use the `fill-mask` pipeline or load it via `AutoModelForMaskedLM`.
43
 
 
 
 
 
 
 
44
  Using `AutoModelForMaskedLM`:
45
 
46
  ```python
 
41
 
42
  You can use these models directly with the `transformers` library. Since ModernBERT is a Masked Language Model (MLM), you can use the `fill-mask` pipeline or load it via `AutoModelForMaskedLM`.
43
 
44
+ **⚠️ We strongly suggest using ModernBERT with Flash Attention 2, as it is by far the best performing variant of the model, and is a 1:1 match of our research implementation. To do so, install Flash Attention as follows, then use the model as normal:**
45
+
46
+ ```bash
47
+ pip install flash-attn
48
+ ```
49
+
50
  Using `AutoModelForMaskedLM`:
51
 
52
  ```python