Update README.md
Browse files
README.md
CHANGED
@@ -41,6 +41,12 @@ It is available in the following sizes:
|
|
41 |
|
42 |
You can use these models directly with the `transformers` library. Since ModernBERT is a Masked Language Model (MLM), you can use the `fill-mask` pipeline or load it via `AutoModelForMaskedLM`.
|
43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
44 |
Using `AutoModelForMaskedLM`:
|
45 |
|
46 |
```python
|
|
|
41 |
|
42 |
You can use these models directly with the `transformers` library. Since ModernBERT is a Masked Language Model (MLM), you can use the `fill-mask` pipeline or load it via `AutoModelForMaskedLM`.
|
43 |
|
44 |
+
**⚠️ We strongly suggest using ModernBERT with Flash Attention 2, as it is by far the best performing variant of the model, and is a 1:1 match of our research implementation. To do so, install Flash Attention as follows, then use the model as normal:**
|
45 |
+
|
46 |
+
```bash
|
47 |
+
pip install flash-attn
|
48 |
+
```
|
49 |
+
|
50 |
Using `AutoModelForMaskedLM`:
|
51 |
|
52 |
```python
|