Update README.md

README.md

@@ -7,7 +7,7 @@ datasets:
 - uonlp/CulturaX
 ---
 
-## ruRoPEBert Classic Model for Russian language
+# ruRoPEBert Classic Model for Russian language
 
 This is an encoder model from **Tochka AI** based on the **RoPEBert** architecture, using the cloning method described in [our article on Habr](https://habr.com/ru/companies/tochka/articles/797561/).
 
@@ -17,12 +17,12 @@ The model source code is available in the file [modeling_rope_bert.py](https://h
 
 The model is trained on contexts **up to 512 tokens** in length, but can be used on larger contexts. For better quality, use the version of this model with extended context - [Tochka-AI/ruRoPEBert-classic-base-2k](https://huggingface.co/Tochka-AI/ruRoPEBert-classic-base-2k)
 
-### Usage
+## Usage
 
 **Important**: To load the model correctly, you must enable dowloading code from the model's repository: `trust_remote_code=True`, this will download the **modeling_rope_bert.py** script and load the weights into the correct architecture.
 Otherwise, you can download this script manually and use classes from it directly to load the model.
 
-#### Basic usage (no efficient attention)
+### Basic usage (no efficient attention)
 
 ```python
 model_name = 'Tochka-AI/ruRoPEBert-classic-base-512'
@@ -30,13 +30,13 @@ tokenizer = AutoTokenizer.from_pretrained(model_name)
 model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
 ```
 
-#### With SDPA (efficient attention)
+### With SDPA (efficient attention)
 
 ```python
 model = AutoModel.from_pretrained(model_name, trust_remote_code=True, attn_implementation='sdpa')
 ```
 
-#### Getting embeddings
+### Getting embeddings
 
 The correct pooler (`mean`) is already **built into the model architecture**, which averages embeddings based on the attention mask. You can also select the pooler type (`first_token_transform`), which performs a learnable linear transformation on the first token.
 
@@ -55,7 +55,7 @@ import torch.nn.functional as F
 F.normalize(pooled_output, dim=1) @ F.normalize(pooled_output, dim=1).T
 ```
 
-#### Using as classifier
+### Using as classifier
 
 To load the model with trainable classification head on top (change `num_labels` parameter):
 
@@ -63,7 +63,7 @@ To load the model with trainable classification head on top (change `num_labels`
 model = AutoModelForSequenceClassification.from_pretrained(model_name, trust_remote_code=True, attn_implementation='sdpa', num_labels=4)
 ```
 
-#### With RoPE scaling
+### With RoPE scaling
 
 Allowed types for RoPE scaling are: `linear` and `dynamic`. To extend the model's context window you need to change tokenizer max length and add `rope_scaling` parameter.
 
@@ -80,7 +80,7 @@ model = AutoModel.from_pretrained(model_name,
 
 P.S. Don't forget to specify the dtype and device you need to use resources efficiently.
 
-### Metrics
+## Metrics
 
 Evaluation of this model on encodechka benchmark:
 
@@ -89,6 +89,6 @@ Evaluation of this model on encodechka benchmark:
 | **ruRoPEBert-classic-base-512** | 0.695 | 0.605 | 0.396 | 0.794 | 0.975 | 0.797 | 0.769 | 0.386 | 0.410 | 0.609 | 0.677 | 0.630 |
 | ai-forever/ruBert-base  | 0.670 | 0.533 | 0.391 | 0.773 | 0.975 | 0.783 | 0.765 | 0.384 | - | - | 0.659       | -                |
 
-### Authors
+## Authors
 - Sergei Bratchikov (Tochka AI Team, [HF](https://huggingface.co/hivaze), [GitHub](https://huggingface.co/hivaze))
 - Maxim Afanasiev (Tochka AI Team, [HF](https://huggingface.co/mrapplexz), [GitHub](https://github.com/mrapplexz))