Hugging Face's logo

ai-forever
/
FRIDA

Feature Extraction
sentence-transformers
Safetensors
Transformers
Russian
English
t5
mteb
Eval Results
Inference Endpoints
Model card Files Community
ai-forever commited on
Commit
720a8a7
·
verified ·
1 Parent(s): 5cc05c6

add ruMTEB metrics

Browse files
Files changed (1) hide show
  1. README.md +1553 -4
README.md CHANGED
@@ -1,10 +1,1559 @@
1
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: mit
3
  language:
4
- - ru
5
- - en
6
- base_model:
7
- - ai-forever/FRED-T5-1.7B
 
 
 
8
  ---
9
 
10
  ---
 
1
  ---
2
+ model-index:
3
+ - name: FRIDA
4
+ results:
5
+ - dataset:
6
+ config: default
7
+ name: MTEB CEDRClassification (default)
8
+ revision: c0ba03d058e3e1b2f3fd20518875a4563dd12db4
9
+ split: test
10
+ type: ai-forever/cedr-classification
11
+ metrics:
12
+ - type: accuracy
13
+ value: 64.60148777895856
14
+ - type: f1
15
+ value: 70.36630348039266
16
+ - type: lrap
17
+ value: 92.47290116896953
18
+ - type: main_score
19
+ value: 64.60148777895856
20
+ task:
21
+ type: MultilabelClassification
22
+ - dataset:
23
+ config: default
24
+ name: MTEB GeoreviewClassification (default)
25
+ revision: 3765c0d1de6b7d264bc459433c45e5a75513839c
26
+ split: test
27
+ type: ai-forever/georeview-classification
28
+ metrics:
29
+ - type: accuracy
30
+ value: 57.70996093750001
31
+ - type: f1
32
+ value: 53.18542982057098
33
+ - type: f1_weighted
34
+ value: 53.17663229582108
35
+ - type: main_score
36
+ value: 57.70996093750001
37
+ task:
38
+ type: Classification
39
+ - dataset:
40
+ config: default
41
+ name: MTEB GeoreviewClusteringP2P (default)
42
+ revision: 97a313c8fc85b47f13f33e7e9a95c1ad888c7fec
43
+ split: test
44
+ type: ai-forever/georeview-clustering-p2p
45
+ metrics:
46
+ - type: main_score
47
+ value: 78.25468393043356
48
+ - type: v_measure
49
+ value: 78.25468393043356
50
+ - type: v_measure_std
51
+ value: 0.5094366871364238
52
+ task:
53
+ type: Clustering
54
+ - dataset:
55
+ config: default
56
+ name: MTEB HeadlineClassification (default)
57
+ revision: 2fe05ee6b5832cda29f2ef7aaad7b7fe6a3609eb
58
+ split: test
59
+ type: ai-forever/headline-classification
60
+ metrics:
61
+ - type: accuracy
62
+ value: 89.0185546875
63
+ - type: f1
64
+ value: 88.993933120612
65
+ - type: f1_weighted
66
+ value: 88.99276764225768
67
+ - type: main_score
68
+ value: 89.0185546875
69
+ task:
70
+ type: Classification
71
+ - dataset:
72
+ config: default
73
+ name: MTEB InappropriatenessClassification (default)
74
+ revision: 601651fdc45ef243751676e62dd7a19f491c0285
75
+ split: test
76
+ type: ai-forever/inappropriateness-classification
77
+ metrics:
78
+ - type: accuracy
79
+ value: 78.330078125
80
+ - type: ap
81
+ value: 73.17856750532495
82
+ - type: ap_weighted
83
+ value: 73.17856750532495
84
+ - type: f1
85
+ value: 78.20169867599041
86
+ - type: f1_weighted
87
+ value: 78.20169867599041
88
+ - type: main_score
89
+ value: 78.330078125
90
+ task:
91
+ type: Classification
92
+ - dataset:
93
+ config: default
94
+ name: MTEB KinopoiskClassification (default)
95
+ revision: 5911f26666ac11af46cb9c6849d0dc80a378af24
96
+ split: test
97
+ type: ai-forever/kinopoisk-sentiment-classification
98
+ metrics:
99
+ - type: accuracy
100
+ value: 70.46666666666665
101
+ - type: f1
102
+ value: 65.83951766538878
103
+ - type: f1_weighted
104
+ value: 65.83951766538878
105
+ - type: main_score
106
+ value: 70.46666666666665
107
+ task:
108
+ type: Classification
109
+ - dataset:
110
+ config: ru
111
+ name: MTEB MIRACLReranking (ru)
112
+ revision: 6d1962c527217f8927fca80f890f14f36b2802af
113
+ split: dev
114
+ type: miracl/mmteb-miracl-reranking
115
+ metrics:
116
+ - type: MAP@1(MIRACL)
117
+ value: 39.023
118
+ - type: MAP@10(MIRACL)
119
+ value: 60.208
120
+ - type: MAP@100(MIRACL)
121
+ value: 61.672000000000004
122
+ - type: MAP@1000(MIRACL)
123
+ value: 61.672000000000004
124
+ - type: MAP@20(MIRACL)
125
+ value: 61.30799999999999
126
+ - type: MAP@3(MIRACL)
127
+ value: 53.33
128
+ - type: MAP@5(MIRACL)
129
+ value: 57.289
130
+ - type: NDCG@1(MIRACL)
131
+ value: 63.352
132
+ - type: NDCG@10(MIRACL)
133
+ value: 66.042
134
+ - type: NDCG@100(MIRACL)
135
+ value: 68.702
136
+ - type: NDCG@1000(MIRACL)
137
+ value: 68.702
138
+ - type: NDCG@20(MIRACL)
139
+ value: 67.768
140
+ - type: NDCG@3(MIRACL)
141
+ value: 61.925
142
+ - type: NDCG@5(MIRACL)
143
+ value: 63.327
144
+ - type: P@1(MIRACL)
145
+ value: 63.352
146
+ - type: P@10(MIRACL)
147
+ value: 16.512
148
+ - type: P@100(MIRACL)
149
+ value: 1.9529999999999998
150
+ - type: P@1000(MIRACL)
151
+ value: 0.19499999999999998
152
+ - type: P@20(MIRACL)
153
+ value: 9.13
154
+ - type: P@3(MIRACL)
155
+ value: 37.878
156
+ - type: P@5(MIRACL)
157
+ value: 27.586
158
+ - type: Recall@1(MIRACL)
159
+ value: 39.023
160
+ - type: Recall@10(MIRACL)
161
+ value: 72.35000000000001
162
+ - type: Recall@100(MIRACL)
163
+ value: 79.952
164
+ - type: Recall@1000(MIRACL)
165
+ value: 79.952
166
+ - type: Recall@20(MIRACL)
167
+ value: 76.828
168
+ - type: Recall@3(MIRACL)
169
+ value: 57.769999999999996
170
+ - type: Recall@5(MIRACL)
171
+ value: 64.91900000000001
172
+ - type: main_score
173
+ value: 66.042
174
+ - type: nAUC_MAP@1000_diff1(MIRACL)
175
+ value: 27.150388833033052
176
+ - type: nAUC_MAP@1000_max(MIRACL)
177
+ value: 55.15672274267081
178
+ - type: nAUC_MAP@1000_std(MIRACL)
179
+ value: 30.088939934575553
180
+ - type: nAUC_MAP@100_diff1(MIRACL)
181
+ value: 27.150388833033052
182
+ - type: nAUC_MAP@100_max(MIRACL)
183
+ value: 55.15672274267081
184
+ - type: nAUC_MAP@100_std(MIRACL)
185
+ value: 30.088939934575553
186
+ - type: nAUC_MAP@10_diff1(MIRACL)
187
+ value: 27.853691773641742
188
+ - type: nAUC_MAP@10_max(MIRACL)
189
+ value: 52.89390350055654
190
+ - type: nAUC_MAP@10_std(MIRACL)
191
+ value: 28.08732516551691
192
+ - type: nAUC_MAP@1_diff1(MIRACL)
193
+ value: 43.23179150244192
194
+ - type: nAUC_MAP@1_max(MIRACL)
195
+ value: 29.923943954188864
196
+ - type: nAUC_MAP@1_std(MIRACL)
197
+ value: 7.447084370195121
198
+ - type: nAUC_MAP@20_diff1(MIRACL)
199
+ value: 27.328384072311675
200
+ - type: nAUC_MAP@20_max(MIRACL)
201
+ value: 54.60286379835721
202
+ - type: nAUC_MAP@20_std(MIRACL)
203
+ value: 29.8084128980043
204
+ - type: nAUC_MAP@3_diff1(MIRACL)
205
+ value: 31.244971536944554
206
+ - type: nAUC_MAP@3_max(MIRACL)
207
+ value: 43.63984692803854
208
+ - type: nAUC_MAP@3_std(MIRACL)
209
+ value: 18.609234683765887
210
+ - type: nAUC_MAP@5_diff1(MIRACL)
211
+ value: 29.088760492638286
212
+ - type: nAUC_MAP@5_max(MIRACL)
213
+ value: 48.30474364461509
214
+ - type: nAUC_MAP@5_std(MIRACL)
215
+ value: 23.817514353844224
216
+ - type: nAUC_NDCG@1000_diff1(MIRACL)
217
+ value: 23.12754356408408
218
+ - type: nAUC_NDCG@1000_max(MIRACL)
219
+ value: 64.24894553363303
220
+ - type: nAUC_NDCG@1000_std(MIRACL)
221
+ value: 38.19318050598967
222
+ - type: nAUC_NDCG@100_diff1(MIRACL)
223
+ value: 23.12754356408408
224
+ - type: nAUC_NDCG@100_max(MIRACL)
225
+ value: 64.24894553363303
226
+ - type: nAUC_NDCG@100_std(MIRACL)
227
+ value: 38.19318050598967
228
+ - type: nAUC_NDCG@10_diff1(MIRACL)
229
+ value: 24.779856373697275
230
+ - type: nAUC_NDCG@10_max(MIRACL)
231
+ value: 60.4054459738118
232
+ - type: nAUC_NDCG@10_std(MIRACL)
233
+ value: 35.148950441182784
234
+ - type: nAUC_NDCG@1_diff1(MIRACL)
235
+ value: 35.605865569438556
236
+ - type: nAUC_NDCG@1_max(MIRACL)
237
+ value: 65.77787399715454
238
+ - type: nAUC_NDCG@1_std(MIRACL)
239
+ value: 34.34726892885082
240
+ - type: nAUC_NDCG@20_diff1(MIRACL)
241
+ value: 23.71231783125691
242
+ - type: nAUC_NDCG@20_max(MIRACL)
243
+ value: 62.89676599488004
244
+ - type: nAUC_NDCG@20_std(MIRACL)
245
+ value: 37.697052941884316
246
+ - type: nAUC_NDCG@3_diff1(MIRACL)
247
+ value: 26.109027741640865
248
+ - type: nAUC_NDCG@3_max(MIRACL)
249
+ value: 56.22356793638693
250
+ - type: nAUC_NDCG@3_std(MIRACL)
251
+ value: 29.9437568508688
252
+ - type: nAUC_NDCG@5_diff1(MIRACL)
253
+ value: 25.98644715327336
254
+ - type: nAUC_NDCG@5_max(MIRACL)
255
+ value: 56.25032008404774
256
+ - type: nAUC_NDCG@5_std(MIRACL)
257
+ value: 31.581899860862578
258
+ - type: nAUC_P@1000_diff1(MIRACL)
259
+ value: -18.29912787064644
260
+ - type: nAUC_P@1000_max(MIRACL)
261
+ value: 31.811344878776087
262
+ - type: nAUC_P@1000_std(MIRACL)
263
+ value: 30.163820183304914
264
+ - type: nAUC_P@100_diff1(MIRACL)
265
+ value: -18.299127870646405
266
+ - type: nAUC_P@100_max(MIRACL)
267
+ value: 31.811344878776133
268
+ - type: nAUC_P@100_std(MIRACL)
269
+ value: 30.163820183304956
270
+ - type: nAUC_P@10_diff1(MIRACL)
271
+ value: -15.96416268531149
272
+ - type: nAUC_P@10_max(MIRACL)
273
+ value: 36.989578896466526
274
+ - type: nAUC_P@10_std(MIRACL)
275
+ value: 34.54507111688143
276
+ - type: nAUC_P@1_diff1(MIRACL)
277
+ value: 35.605865569438556
278
+ - type: nAUC_P@1_max(MIRACL)
279
+ value: 65.77787399715454
280
+ - type: nAUC_P@1_std(MIRACL)
281
+ value: 34.34726892885082
282
+ - type: nAUC_P@20_diff1(MIRACL)
283
+ value: -17.443963421383287
284
+ - type: nAUC_P@20_max(MIRACL)
285
+ value: 34.309618168778385
286
+ - type: nAUC_P@20_std(MIRACL)
287
+ value: 33.38820956485373
288
+ - type: nAUC_P@3_diff1(MIRACL)
289
+ value: -8.533621861815652
290
+ - type: nAUC_P@3_max(MIRACL)
291
+ value: 45.90408386776497
292
+ - type: nAUC_P@3_std(MIRACL)
293
+ value: 34.50459351305535
294
+ - type: nAUC_P@5_diff1(MIRACL)
295
+ value: -13.207968899314865
296
+ - type: nAUC_P@5_max(MIRACL)
297
+ value: 40.37718282248973
298
+ - type: nAUC_P@5_std(MIRACL)
299
+ value: 35.601417332196206
300
+ - type: nAUC_Recall@1000_diff1(MIRACL)
301
+ value: 7.907304198177226
302
+ - type: nAUC_Recall@1000_max(MIRACL)
303
+ value: 77.82197832361145
304
+ - type: nAUC_Recall@1000_std(MIRACL)
305
+ value: 52.66957487246724
306
+ - type: nAUC_Recall@100_diff1(MIRACL)
307
+ value: 7.907304198177226
308
+ - type: nAUC_Recall@100_max(MIRACL)
309
+ value: 77.82197832361145
310
+ - type: nAUC_Recall@100_std(MIRACL)
311
+ value: 52.66957487246724
312
+ - type: nAUC_Recall@10_diff1(MIRACL)
313
+ value: 15.498121023488693
314
+ - type: nAUC_Recall@10_max(MIRACL)
315
+ value: 62.24320529338724
316
+ - type: nAUC_Recall@10_std(MIRACL)
317
+ value: 40.60221460946224
318
+ - type: nAUC_Recall@1_diff1(MIRACL)
319
+ value: 43.23179150244192
320
+ - type: nAUC_Recall@1_max(MIRACL)
321
+ value: 29.923943954188864
322
+ - type: nAUC_Recall@1_std(MIRACL)
323
+ value: 7.447084370195121
324
+ - type: nAUC_Recall@20_diff1(MIRACL)
325
+ value: 11.457044176116248
326
+ - type: nAUC_Recall@20_max(MIRACL)
327
+ value: 70.3493054342368
328
+ - type: nAUC_Recall@20_std(MIRACL)
329
+ value: 49.27124296325928
330
+ - type: nAUC_Recall@3_diff1(MIRACL)
331
+ value: 25.12077828977941
332
+ - type: nAUC_Recall@3_max(MIRACL)
333
+ value: 42.903379317937166
334
+ - type: nAUC_Recall@3_std(MIRACL)
335
+ value: 20.324501722161497
336
+ - type: nAUC_Recall@5_diff1(MIRACL)
337
+ value: 20.925701235197977
338
+ - type: nAUC_Recall@5_max(MIRACL)
339
+ value: 49.85323960390812
340
+ - type: nAUC_Recall@5_std(MIRACL)
341
+ value: 29.04484539530469
342
+ task:
343
+ type: Reranking
344
+ - dataset:
345
+ config: ru
346
+ name: MTEB MIRACLRetrieval (ru)
347
+ revision: main
348
+ split: dev
349
+ type: miracl/mmteb-miracl
350
+ metrics:
351
+ - type: main_score
352
+ value: 71.882
353
+ - type: map_at_1
354
+ value: 37.913000000000004
355
+ - type: map_at_10
356
+ value: 62.604000000000006
357
+ - type: map_at_100
358
+ value: 64.925
359
+ - type: map_at_1000
360
+ value: 64.992
361
+ - type: map_at_20
362
+ value: 64.081
363
+ - type: map_at_3
364
+ value: 55.212
365
+ - type: map_at_5
366
+ value: 59.445
367
+ - type: mrr_at_1
368
+ value: 73.24281150159744
369
+ - type: mrr_at_10
370
+ value: 81.65043866321825
371
+ - type: mrr_at_100
372
+ value: 81.85391378818977
373
+ - type: mrr_at_1000
374
+ value: 81.85753390802569
375
+ - type: mrr_at_20
376
+ value: 81.81045606130179
377
+ - type: mrr_at_3
378
+ value: 80.56443024494146
379
+ - type: mrr_at_5
380
+ value: 81.30724174653893
381
+ - type: nauc_map_at_1000_diff1
382
+ value: 26.962150235593356
383
+ - type: nauc_map_at_1000_max
384
+ value: 29.234958037854568
385
+ - type: nauc_map_at_1000_std
386
+ value: -2.4294465103633884
387
+ - type: nauc_map_at_100_diff1
388
+ value: 26.92990252114163
389
+ - type: nauc_map_at_100_max
390
+ value: 29.206328533120118
391
+ - type: nauc_map_at_100_std
392
+ value: -2.437371090941197
393
+ - type: nauc_map_at_10_diff1
394
+ value: 25.758265691179226
395
+ - type: nauc_map_at_10_max
396
+ value: 26.949978490795317
397
+ - type: nauc_map_at_10_std
398
+ value: -5.484961002106038
399
+ - type: nauc_map_at_1_diff1
400
+ value: 34.70849461278043
401
+ - type: nauc_map_at_1_max
402
+ value: 12.778570893623042
403
+ - type: nauc_map_at_1_std
404
+ value: -13.018292652743938
405
+ - type: nauc_map_at_20_diff1
406
+ value: 26.659923008218268
407
+ - type: nauc_map_at_20_max
408
+ value: 28.341440871568185
409
+ - type: nauc_map_at_20_std
410
+ value: -3.614549844913084
411
+ - type: nauc_map_at_3_diff1
412
+ value: 27.197629021438203
413
+ - type: nauc_map_at_3_max
414
+ value: 20.701094874050856
415
+ - type: nauc_map_at_3_std
416
+ value: -12.062992301112041
417
+ - type: nauc_map_at_5_diff1
418
+ value: 25.51793537203295
419
+ - type: nauc_map_at_5_max
420
+ value: 23.80396771243794
421
+ - type: nauc_map_at_5_std
422
+ value: -8.920465695323575
423
+ - type: nauc_mrr_at_1000_diff1
424
+ value: 45.14819989592967
425
+ - type: nauc_mrr_at_1000_max
426
+ value: 53.29202156141053
427
+ - type: nauc_mrr_at_1000_std
428
+ value: 18.037336462510524
429
+ - type: nauc_mrr_at_100_diff1
430
+ value: 45.15287600228451
431
+ - type: nauc_mrr_at_100_max
432
+ value: 53.29979751928615
433
+ - type: nauc_mrr_at_100_std
434
+ value: 18.04996604778386
435
+ - type: nauc_mrr_at_10_diff1
436
+ value: 44.96865105944474
437
+ - type: nauc_mrr_at_10_max
438
+ value: 53.53323465323092
439
+ - type: nauc_mrr_at_10_std
440
+ value: 18.25001344917689
441
+ - type: nauc_mrr_at_1_diff1
442
+ value: 46.16604946873163
443
+ - type: nauc_mrr_at_1_max
444
+ value: 48.573651103547874
445
+ - type: nauc_mrr_at_1_std
446
+ value: 13.764871626330915
447
+ - type: nauc_mrr_at_20_diff1
448
+ value: 45.11925458479102
449
+ - type: nauc_mrr_at_20_max
450
+ value: 53.35685123898342
451
+ - type: nauc_mrr_at_20_std
452
+ value: 18.127344968819905
453
+ - type: nauc_mrr_at_3_diff1
454
+ value: 45.377195452730234
455
+ - type: nauc_mrr_at_3_max
456
+ value: 53.35146309217089
457
+ - type: nauc_mrr_at_3_std
458
+ value: 17.47105877186237
459
+ - type: nauc_mrr_at_5_diff1
460
+ value: 45.00525578771549
461
+ - type: nauc_mrr_at_5_max
462
+ value: 53.76227254707128
463
+ - type: nauc_mrr_at_5_std
464
+ value: 18.437290060746957
465
+ - type: nauc_ndcg_at_1000_diff1
466
+ value: 31.19215594457491
467
+ - type: nauc_ndcg_at_1000_max
468
+ value: 38.09555406458668
469
+ - type: nauc_ndcg_at_1000_std
470
+ value: 7.225628621238009
471
+ - type: nauc_ndcg_at_100_diff1
472
+ value: 30.726331247999934
473
+ - type: nauc_ndcg_at_100_max
474
+ value: 37.81369589418277
475
+ - type: nauc_ndcg_at_100_std
476
+ value: 7.242855238555071
477
+ - type: nauc_ndcg_at_10_diff1
478
+ value: 27.514048333744835
479
+ - type: nauc_ndcg_at_10_max
480
+ value: 33.10990399385253
481
+ - type: nauc_ndcg_at_10_std
482
+ value: 0.3051899572112002
483
+ - type: nauc_ndcg_at_1_diff1
484
+ value: 47.06089085235751
485
+ - type: nauc_ndcg_at_1_max
486
+ value: 47.7300872370495
487
+ - type: nauc_ndcg_at_1_std
488
+ value: 12.468605493613916
489
+ - type: nauc_ndcg_at_20_diff1
490
+ value: 29.404215438764496
491
+ - type: nauc_ndcg_at_20_max
492
+ value: 35.26967886796471
493
+ - type: nauc_ndcg_at_20_std
494
+ value: 3.7214697890813353
495
+ - type: nauc_ndcg_at_3_diff1
496
+ value: 29.448848639643067
497
+ - type: nauc_ndcg_at_3_max
498
+ value: 33.85912412370657
499
+ - type: nauc_ndcg_at_3_std
500
+ value: 0.895453646819452
501
+ - type: nauc_ndcg_at_5_diff1
502
+ value: 26.916649012613526
503
+ - type: nauc_ndcg_at_5_max
504
+ value: 30.899005979291644
505
+ - type: nauc_ndcg_at_5_std
506
+ value: -1.0001575639156615
507
+ - type: nauc_precision_at_1000_diff1
508
+ value: -8.492004667432635
509
+ - type: nauc_precision_at_1000_max
510
+ value: 14.970190384017679
511
+ - type: nauc_precision_at_1000_std
512
+ value: 32.871386621137816
513
+ - type: nauc_precision_at_100_diff1
514
+ value: -8.287314133999967
515
+ - type: nauc_precision_at_100_max
516
+ value: 17.794821961284736
517
+ - type: nauc_precision_at_100_std
518
+ value: 35.092483550562
519
+ - type: nauc_precision_at_10_diff1
520
+ value: -7.594128993028063
521
+ - type: nauc_precision_at_10_max
522
+ value: 24.691446370325732
523
+ - type: nauc_precision_at_10_std
524
+ value: 30.126552282608493
525
+ - type: nauc_precision_at_1_diff1
526
+ value: 47.06089085235751
527
+ - type: nauc_precision_at_1_max
528
+ value: 47.7300872370495
529
+ - type: nauc_precision_at_1_std
530
+ value: 12.468605493613916
531
+ - type: nauc_precision_at_20_diff1
532
+ value: -6.503872195775146
533
+ - type: nauc_precision_at_20_max
534
+ value: 21.789730053141312
535
+ - type: nauc_precision_at_20_std
536
+ value: 32.61349377558794
537
+ - type: nauc_precision_at_3_diff1
538
+ value: 0.67417079971061
539
+ - type: nauc_precision_at_3_max
540
+ value: 30.793871354370662
541
+ - type: nauc_precision_at_3_std
542
+ value: 18.35266479252011
543
+ - type: nauc_precision_at_5_diff1
544
+ value: -7.088881730215777
545
+ - type: nauc_precision_at_5_max
546
+ value: 26.539771712769006
547
+ - type: nauc_precision_at_5_std
548
+ value: 24.116262291865834
549
+ - type: nauc_recall_at_1000_diff1
550
+ value: 34.53263588412461
551
+ - type: nauc_recall_at_1000_max
552
+ value: 63.54157869100173
553
+ - type: nauc_recall_at_1000_std
554
+ value: 64.19854844792808
555
+ - type: nauc_recall_at_100_diff1
556
+ value: 22.86564728642275
557
+ - type: nauc_recall_at_100_max
558
+ value: 40.350507162549825
559
+ - type: nauc_recall_at_100_std
560
+ value: 29.24492545863015
561
+ - type: nauc_recall_at_10_diff1
562
+ value: 15.384818367225009
563
+ - type: nauc_recall_at_10_max
564
+ value: 24.41108571453699
565
+ - type: nauc_recall_at_10_std
566
+ value: -3.9216160585776323
567
+ - type: nauc_recall_at_1_diff1
568
+ value: 34.70849461278043
569
+ - type: nauc_recall_at_1_max
570
+ value: 12.778570893623042
571
+ - type: nauc_recall_at_1_std
572
+ value: -13.018292652743938
573
+ - type: nauc_recall_at_20_diff1
574
+ value: 18.122499000084208
575
+ - type: nauc_recall_at_20_max
576
+ value: 26.63104220179424
577
+ - type: nauc_recall_at_20_std
578
+ value: 3.969217732521512
579
+ - type: nauc_recall_at_3_diff1
580
+ value: 21.413050725250116
581
+ - type: nauc_recall_at_3_max
582
+ value: 16.18894988386887
583
+ - type: nauc_recall_at_3_std
584
+ value: -15.24884339282375
585
+ - type: nauc_recall_at_5_diff1
586
+ value: 16.35673072212927
587
+ - type: nauc_recall_at_5_max
588
+ value: 18.607003829267846
589
+ - type: nauc_recall_at_5_std
590
+ value: -10.463525876945454
591
+ - type: ndcg_at_1
592
+ value: 72.923
593
+ - type: ndcg_at_10
594
+ value: 71.882
595
+ - type: ndcg_at_100
596
+ value: 77.09899999999999
597
+ - type: ndcg_at_1000
598
+ value: 77.835
599
+ - type: ndcg_at_20
600
+ value: 74.497
601
+ - type: ndcg_at_3
602
+ value: 68.504
603
+ - type: ndcg_at_5
604
+ value: 69.068
605
+ - type: precision_at_1
606
+ value: 72.923
607
+ - type: precision_at_10
608
+ value: 19.936
609
+ - type: precision_at_100
610
+ value: 2.6310000000000002
611
+ - type: precision_at_1000
612
+ value: 0.27799999999999997
613
+ - type: precision_at_20
614
+ value: 11.33
615
+ - type: precision_at_3
616
+ value: 45.927
617
+ - type: precision_at_5
618
+ value: 33.131
619
+ - type: recall_at_1
620
+ value: 37.913000000000004
621
+ - type: recall_at_10
622
+ value: 78.365
623
+ - type: recall_at_100
624
+ value: 94.348
625
+ - type: recall_at_1000
626
+ value: 98.187
627
+ - type: recall_at_20
628
+ value: 85.229
629
+ - type: recall_at_3
630
+ value: 61.42999999999999
631
+ - type: recall_at_5
632
+ value: 69.56700000000001
633
+ task:
634
+ type: Retrieval
635
+ - dataset:
636
+ config: ru
637
+ name: MTEB MassiveIntentClassification (ru)
638
+ revision: 4672e20407010da34463acc759c162ca9734bca6
639
+ split: test
640
+ type: mteb/amazon_massive_intent
641
+ metrics:
642
+ - type: accuracy
643
+ value: 79.11903160726294
644
+ - type: f1
645
+ value: 76.22609082694545
646
+ - type: f1_weighted
647
+ value: 77.81461248063566
648
+ - type: main_score
649
+ value: 79.11903160726294
650
+ task:
651
+ type: Classification
652
+ - dataset:
653
+ config: ru
654
+ name: MTEB MassiveScenarioClassification (ru)
655
+ revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
656
+ split: test
657
+ type: mteb/amazon_massive_scenario
658
+ metrics:
659
+ - type: accuracy
660
+ value: 88.80632145258912
661
+ - type: f1
662
+ value: 87.53157475314829
663
+ - type: f1_weighted
664
+ value: 88.22733432521495
665
+ - type: main_score
666
+ value: 88.80632145258912
667
+ task:
668
+ type: Classification
669
+ - dataset:
670
+ config: default
671
+ name: MTEB RUParaPhraserSTS (default)
672
+ revision: 43265056790b8f7c59e0139acb4be0a8dad2c8f4
673
+ split: test
674
+ type: merionum/ru_paraphraser
675
+ metrics:
676
+ - type: cosine_pearson
677
+ value: 72.70307124858925
678
+ - type: cosine_spearman
679
+ value: 78.09439086920204
680
+ - type: euclidean_pearson
681
+ value: 76.2033672014715
682
+ - type: euclidean_spearman
683
+ value: 78.09439086920204
684
+ - type: main_score
685
+ value: 78.09439086920204
686
+ - type: manhattan_pearson
687
+ value: 76.11750470223116
688
+ - type: manhattan_spearman
689
+ value: 78.01081063503413
690
+ - type: pearson
691
+ value: 72.70307124858925
692
+ - type: spearman
693
+ value: 78.09439086920204
694
+ task:
695
+ type: STS
696
+ - dataset:
697
+ config: default
698
+ name: MTEB RiaNewsRetrieval (default)
699
+ revision: 82374b0bbacda6114f39ff9c5b925fa1512ca5d7
700
+ split: test
701
+ type: ai-forever/ria-news-retrieval
702
+ metrics:
703
+ - type: main_score
704
+ value: 86.819
705
+ - type: map_at_1
706
+ value: 78.79
707
+ - type: map_at_10
708
+ value: 84.516
709
+ - type: map_at_100
710
+ value: 84.68
711
+ - type: map_at_1000
712
+ value: 84.685
713
+ - type: map_at_20
714
+ value: 84.624
715
+ - type: map_at_3
716
+ value: 83.722
717
+ - type: map_at_5
718
+ value: 84.246
719
+ - type: mrr_at_1
720
+ value: 78.78
721
+ - type: mrr_at_10
722
+ value: 84.51815476190441
723
+ - type: mrr_at_100
724
+ value: 84.68390840473289
725
+ - type: mrr_at_1000
726
+ value: 84.68947095200002
727
+ - type: mrr_at_20
728
+ value: 84.62958130822527
729
+ - type: mrr_at_3
730
+ value: 83.74499999999964
731
+ - type: mrr_at_5
732
+ value: 84.23849999999955
733
+ - type: nauc_map_at_1000_diff1
734
+ value: 82.09914867708899
735
+ - type: nauc_map_at_1000_max
736
+ value: 43.02024854784386
737
+ - type: nauc_map_at_1000_std
738
+ value: -22.919695880762777
739
+ - type: nauc_map_at_100_diff1
740
+ value: 82.09705922783733
741
+ - type: nauc_map_at_100_max
742
+ value: 43.02697379581718
743
+ - type: nauc_map_at_100_std
744
+ value: -22.90719212899522
745
+ - type: nauc_map_at_10_diff1
746
+ value: 82.04404594672894
747
+ - type: nauc_map_at_10_max
748
+ value: 43.06752103182731
749
+ - type: nauc_map_at_10_std
750
+ value: -23.007870153273576
751
+ - type: nauc_map_at_1_diff1
752
+ value: 83.89134152210333
753
+ - type: nauc_map_at_1_max
754
+ value: 38.083626428503415
755
+ - type: nauc_map_at_1_std
756
+ value: -25.817960401194252
757
+ - type: nauc_map_at_20_diff1
758
+ value: 82.08534662247806
759
+ - type: nauc_map_at_20_max
760
+ value: 43.074305042312346
761
+ - type: nauc_map_at_20_std
762
+ value: -22.91785703613217
763
+ - type: nauc_map_at_3_diff1
764
+ value: 81.7967508697558
765
+ - type: nauc_map_at_3_max
766
+ value: 42.90927479098251
767
+ - type: nauc_map_at_3_std
768
+ value: -24.01312203859392
769
+ - type: nauc_map_at_5_diff1
770
+ value: 81.90704517505098
771
+ - type: nauc_map_at_5_max
772
+ value: 43.05204677044616
773
+ - type: nauc_map_at_5_std
774
+ value: -23.267331507554896
775
+ - type: nauc_mrr_at_1000_diff1
776
+ value: 82.11902348082472
777
+ - type: nauc_mrr_at_1000_max
778
+ value: 43.04118936353063
779
+ - type: nauc_mrr_at_1000_std
780
+ value: -22.858804296830773
781
+ - type: nauc_mrr_at_100_diff1
782
+ value: 82.11685562002263
783
+ - type: nauc_mrr_at_100_max
784
+ value: 43.0482537895494
785
+ - type: nauc_mrr_at_100_std
786
+ value: -22.84431127787993
787
+ - type: nauc_mrr_at_10_diff1
788
+ value: 82.06909958688058
789
+ - type: nauc_mrr_at_10_max
790
+ value: 43.07921689466605
791
+ - type: nauc_mrr_at_10_std
792
+ value: -22.957623576663234
793
+ - type: nauc_mrr_at_1_diff1
794
+ value: 83.91147637794326
795
+ - type: nauc_mrr_at_1_max
796
+ value: 37.91917159543152
797
+ - type: nauc_mrr_at_1_std
798
+ value: -26.141868289283266
799
+ - type: nauc_mrr_at_20_diff1
800
+ value: 82.10314004731809
801
+ - type: nauc_mrr_at_20_max
802
+ value: 43.09295406509764
803
+ - type: nauc_mrr_at_20_std
804
+ value: -22.862091782178787
805
+ - type: nauc_mrr_at_3_diff1
806
+ value: 81.82117067269036
807
+ - type: nauc_mrr_at_3_max
808
+ value: 42.94628953323521
809
+ - type: nauc_mrr_at_3_std
810
+ value: -23.852510312400714
811
+ - type: nauc_mrr_at_5_diff1
812
+ value: 81.92857441701598
813
+ - type: nauc_mrr_at_5_max
814
+ value: 43.129719354492934
815
+ - type: nauc_mrr_at_5_std
816
+ value: -23.145342272624085
817
+ - type: nauc_ndcg_at_1000_diff1
818
+ value: 81.75015729717991
819
+ - type: nauc_ndcg_at_1000_max
820
+ value: 44.7266586308995
821
+ - type: nauc_ndcg_at_1000_std
822
+ value: -20.60663899715267
823
+ - type: nauc_ndcg_at_100_diff1
824
+ value: 81.6897808298767
825
+ - type: nauc_ndcg_at_100_max
826
+ value: 44.99492791287099
827
+ - type: nauc_ndcg_at_100_std
828
+ value: -20.09637266506936
829
+ - type: nauc_ndcg_at_10_diff1
830
+ value: 81.46290312197337
831
+ - type: nauc_ndcg_at_10_max
832
+ value: 45.30218378452244
833
+ - type: nauc_ndcg_at_10_std
834
+ value: -20.70393523891777
835
+ - type: nauc_ndcg_at_1_diff1
836
+ value: 83.89134152210333
837
+ - type: nauc_ndcg_at_1_max
838
+ value: 38.083626428503415
839
+ - type: nauc_ndcg_at_1_std
840
+ value: -25.817960401194252
841
+ - type: nauc_ndcg_at_20_diff1
842
+ value: 81.61080772657213
843
+ - type: nauc_ndcg_at_20_max
844
+ value: 45.36571800492172
845
+ - type: nauc_ndcg_at_20_std
846
+ value: -20.278763852504042
847
+ - type: nauc_ndcg_at_3_diff1
848
+ value: 80.95965359410461
849
+ - type: nauc_ndcg_at_3_max
850
+ value: 44.756971949205834
851
+ - type: nauc_ndcg_at_3_std
852
+ value: -23.07797617717319
853
+ - type: nauc_ndcg_at_5_diff1
854
+ value: 81.12417712163976
855
+ - type: nauc_ndcg_at_5_max
856
+ value: 45.15727381406512
857
+ - type: nauc_ndcg_at_5_std
858
+ value: -21.52861766165519
859
+ - type: nauc_precision_at_1000_diff1
860
+ value: 76.80566850396093
861
+ - type: nauc_precision_at_1000_max
862
+ value: 82.45685370922442
863
+ - type: nauc_precision_at_1000_std
864
+ value: 46.93570976777808
865
+ - type: nauc_precision_at_100_diff1
866
+ value: 77.21645520953484
867
+ - type: nauc_precision_at_100_max
868
+ value: 73.43604108309935
869
+ - type: nauc_precision_at_100_std
870
+ value: 31.978176891671367
871
+ - type: nauc_precision_at_10_diff1
872
+ value: 77.88251664302092
873
+ - type: nauc_precision_at_10_max
874
+ value: 60.58112638995018
875
+ - type: nauc_precision_at_10_std
876
+ value: -3.674424315180332
877
+ - type: nauc_precision_at_1_diff1
878
+ value: 83.89134152210333
879
+ - type: nauc_precision_at_1_max
880
+ value: 38.083626428503415
881
+ - type: nauc_precision_at_1_std
882
+ value: -25.817960401194252
883
+ - type: nauc_precision_at_20_diff1
884
+ value: 78.16426786697438
885
+ - type: nauc_precision_at_20_max
886
+ value: 66.0723612699222
887
+ - type: nauc_precision_at_20_std
888
+ value: 6.121527084555938
889
+ - type: nauc_precision_at_3_diff1
890
+ value: 77.43122492166451
891
+ - type: nauc_precision_at_3_max
892
+ value: 52.50727288548085
893
+ - type: nauc_precision_at_3_std
894
+ value: -19.036076920799427
895
+ - type: nauc_precision_at_5_diff1
896
+ value: 77.1127254320532
897
+ - type: nauc_precision_at_5_max
898
+ value: 56.100901899221135
899
+ - type: nauc_precision_at_5_std
900
+ value: -12.009191140844198
901
+ - type: nauc_recall_at_1000_diff1
902
+ value: 76.80566850396035
903
+ - type: nauc_recall_at_1000_max
904
+ value: 82.45685370922577
905
+ - type: nauc_recall_at_1000_std
906
+ value: 46.93570976777776
907
+ - type: nauc_recall_at_100_diff1
908
+ value: 77.21645520953459
909
+ - type: nauc_recall_at_100_max
910
+ value: 73.43604108310011
911
+ - type: nauc_recall_at_100_std
912
+ value: 31.978176891671993
913
+ - type: nauc_recall_at_10_diff1
914
+ value: 77.88251664302089
915
+ - type: nauc_recall_at_10_max
916
+ value: 60.58112638994999
917
+ - type: nauc_recall_at_10_std
918
+ value: -3.6744243151805427
919
+ - type: nauc_recall_at_1_diff1
920
+ value: 83.89134152210333
921
+ - type: nauc_recall_at_1_max
922
+ value: 38.083626428503415
923
+ - type: nauc_recall_at_1_std
924
+ value: -25.817960401194252
925
+ - type: nauc_recall_at_20_diff1
926
+ value: 78.16426786697409
927
+ - type: nauc_recall_at_20_max
928
+ value: 66.07236126992217
929
+ - type: nauc_recall_at_20_std
930
+ value: 6.121527084555941
931
+ - type: nauc_recall_at_3_diff1
932
+ value: 77.43122492166454
933
+ - type: nauc_recall_at_3_max
934
+ value: 52.507272885480816
935
+ - type: nauc_recall_at_3_std
936
+ value: -19.036076920799776
937
+ - type: nauc_recall_at_5_diff1
938
+ value: 77.11272543205318
939
+ - type: nauc_recall_at_5_max
940
+ value: 56.10090189922128
941
+ - type: nauc_recall_at_5_std
942
+ value: -12.009191140843809
943
+ - type: ndcg_at_1
944
+ value: 78.79
945
+ - type: ndcg_at_10
946
+ value: 86.819
947
+ - type: ndcg_at_100
948
+ value: 87.599
949
+ - type: ndcg_at_1000
950
+ value: 87.761
951
+ - type: ndcg_at_20
952
+ value: 87.208
953
+ - type: ndcg_at_3
954
+ value: 85.222
955
+ - type: ndcg_at_5
956
+ value: 86.164
957
+ - type: precision_at_1
958
+ value: 78.79
959
+ - type: precision_at_10
960
+ value: 9.384
961
+ - type: precision_at_100
962
+ value: 0.975
963
+ - type: precision_at_1000
964
+ value: 0.099
965
+ - type: precision_at_20
966
+ value: 4.769
967
+ - type: precision_at_3
968
+ value: 29.842999999999996
969
+ - type: precision_at_5
970
+ value: 18.362000000000002
971
+ - type: recall_at_1
972
+ value: 78.79
973
+ - type: recall_at_10
974
+ value: 93.84
975
+ - type: recall_at_100
976
+ value: 97.45
977
+ - type: recall_at_1000
978
+ value: 98.76
979
+ - type: recall_at_20
980
+ value: 95.37
981
+ - type: recall_at_3
982
+ value: 89.53
983
+ - type: recall_at_5
984
+ value: 91.81
985
+ task:
986
+ type: Retrieval
987
+ - dataset:
988
+ config: default
989
+ name: MTEB RuBQReranking (default)
990
+ revision: 2e96b8f098fa4b0950fc58eacadeb31c0d0c7fa2
991
+ split: test
992
+ type: ai-forever/rubq-reranking
993
+ metrics:
994
+ - type: main_score
995
+ value: 77.07394404835635
996
+ - type: map
997
+ value: 77.07394404835635
998
+ - type: mrr
999
+ value: 82.53144412718882
1000
+ - type: nAUC_map_diff1
1001
+ value: 45.29805217456628
1002
+ - type: nAUC_map_max
1003
+ value: 34.39894042439188
1004
+ - type: nAUC_map_std
1005
+ value: 21.11309674418275
1006
+ - type: nAUC_mrr_diff1
1007
+ value: 54.783994737367046
1008
+ - type: nAUC_mrr_max
1009
+ value: 45.68526733900048
1010
+ - type: nAUC_mrr_std
1011
+ value: 28.22466385500339
1012
+ task:
1013
+ type: Reranking
1014
+ - dataset:
1015
+ config: default
1016
+ name: MTEB RuBQRetrieval (default)
1017
+ revision: e19b6ffa60b3bc248e0b41f4cc37c26a55c2a67b
1018
+ split: test
1019
+ type: ai-forever/rubq-retrieval
1020
+ metrics:
1021
+ - type: main_score
1022
+ value: 72.392
1023
+ - type: map_at_1
1024
+ value: 47.370000000000005
1025
+ - type: map_at_10
1026
+ value: 65.503
1027
+ - type: map_at_100
1028
+ value: 66.38
1029
+ - type: map_at_1000
1030
+ value: 66.42099999999999
1031
+ - type: map_at_20
1032
+ value: 66.071
1033
+ - type: map_at_3
1034
+ value: 61.439
1035
+ - type: map_at_5
1036
+ value: 63.922999999999995
1037
+ - type: mrr_at_1
1038
+ value: 67.37588652482269
1039
+ - type: mrr_at_10
1040
+ value: 76.0066747345116
1041
+ - type: mrr_at_100
1042
+ value: 76.25754138969413
1043
+ - type: mrr_at_1000
1044
+ value: 76.26968825657428
1045
+ - type: mrr_at_20
1046
+ value: 76.17548265904622
1047
+ - type: mrr_at_3
1048
+ value: 74.61583924349881
1049
+ - type: mrr_at_5
1050
+ value: 75.46690307328608
1051
+ - type: nauc_map_at_1000_diff1
1052
+ value: 42.52570720187294
1053
+ - type: nauc_map_at_1000_max
1054
+ value: 37.40318318724238
1055
+ - type: nauc_map_at_1000_std
1056
+ value: 0.6037788201535506
1057
+ - type: nauc_map_at_100_diff1
1058
+ value: 42.493410029691226
1059
+ - type: nauc_map_at_100_max
1060
+ value: 37.39802489244377
1061
+ - type: nauc_map_at_100_std
1062
+ value: 0.6071359951887154
1063
+ - type: nauc_map_at_10_diff1
1064
+ value: 42.09833519659916
1065
+ - type: nauc_map_at_10_max
1066
+ value: 37.1184138958874
1067
+ - type: nauc_map_at_10_std
1068
+ value: 0.4063543094010351
1069
+ - type: nauc_map_at_1_diff1
1070
+ value: 49.56605205141156
1071
+ - type: nauc_map_at_1_max
1072
+ value: 26.251096698710384
1073
+ - type: nauc_map_at_1_std
1074
+ value: -4.580748485387834
1075
+ - type: nauc_map_at_20_diff1
1076
+ value: 42.33372393482018
1077
+ - type: nauc_map_at_20_max
1078
+ value: 37.416955604649985
1079
+ - type: nauc_map_at_20_std
1080
+ value: 0.6050577802787294
1081
+ - type: nauc_map_at_3_diff1
1082
+ value: 42.362234475441845
1083
+ - type: nauc_map_at_3_max
1084
+ value: 34.56001379838821
1085
+ - type: nauc_map_at_3_std
1086
+ value: -1.507636598929042
1087
+ - type: nauc_map_at_5_diff1
1088
+ value: 42.0202264882535
1089
+ - type: nauc_map_at_5_max
1090
+ value: 36.64306050200848
1091
+ - type: nauc_map_at_5_std
1092
+ value: -0.09509025708798424
1093
+ - type: nauc_mrr_at_1000_diff1
1094
+ value: 58.99601742026931
1095
+ - type: nauc_mrr_at_1000_max
1096
+ value: 49.61561872452777
1097
+ - type: nauc_mrr_at_1000_std
1098
+ value: 2.3956102974352356
1099
+ - type: nauc_mrr_at_100_diff1
1100
+ value: 58.9865943101085
1101
+ - type: nauc_mrr_at_100_max
1102
+ value: 49.6248111507265
1103
+ - type: nauc_mrr_at_100_std
1104
+ value: 2.411155095066369
1105
+ - type: nauc_mrr_at_10_diff1
1106
+ value: 58.81758131092919
1107
+ - type: nauc_mrr_at_10_max
1108
+ value: 49.780365572616695
1109
+ - type: nauc_mrr_at_10_std
1110
+ value: 2.7068696565195944
1111
+ - type: nauc_mrr_at_1_diff1
1112
+ value: 61.67036882487055
1113
+ - type: nauc_mrr_at_1_max
1114
+ value: 45.455271042821714
1115
+ - type: nauc_mrr_at_1_std
1116
+ value: -0.9370526815458349
1117
+ - type: nauc_mrr_at_20_diff1
1118
+ value: 58.93674818203478
1119
+ - type: nauc_mrr_at_20_max
1120
+ value: 49.703218108625215
1121
+ - type: nauc_mrr_at_20_std
1122
+ value: 2.4473106598190415
1123
+ - type: nauc_mrr_at_3_diff1
1124
+ value: 59.046856598788445
1125
+ - type: nauc_mrr_at_3_max
1126
+ value: 49.37161726123392
1127
+ - type: nauc_mrr_at_3_std
1128
+ value: 1.5110936686701506
1129
+ - type: nauc_mrr_at_5_diff1
1130
+ value: 58.92289378915668
1131
+ - type: nauc_mrr_at_5_max
1132
+ value: 49.847638994134144
1133
+ - type: nauc_mrr_at_5_std
1134
+ value: 2.420421880131702
1135
+ - type: nauc_ndcg_at_1000_diff1
1136
+ value: 45.56062215161734
1137
+ - type: nauc_ndcg_at_1000_max
1138
+ value: 41.507152286702
1139
+ - type: nauc_ndcg_at_1000_std
1140
+ value: 2.79388283208751
1141
+ - type: nauc_ndcg_at_100_diff1
1142
+ value: 44.84064192570408
1143
+ - type: nauc_ndcg_at_100_max
1144
+ value: 41.50353573562353
1145
+ - type: nauc_ndcg_at_100_std
1146
+ value: 3.1804999773629357
1147
+ - type: nauc_ndcg_at_10_diff1
1148
+ value: 43.341482144213614
1149
+ - type: nauc_ndcg_at_10_max
1150
+ value: 41.159590898395074
1151
+ - type: nauc_ndcg_at_10_std
1152
+ value: 2.945242338240843
1153
+ - type: nauc_ndcg_at_1_diff1
1154
+ value: 62.23623985611396
1155
+ - type: nauc_ndcg_at_1_max
1156
+ value: 45.04945770947091
1157
+ - type: nauc_ndcg_at_1_std
1158
+ value: -0.8804967656575725
1159
+ - type: nauc_ndcg_at_20_diff1
1160
+ value: 43.905372612093664
1161
+ - type: nauc_ndcg_at_20_max
1162
+ value: 41.797709837872446
1163
+ - type: nauc_ndcg_at_20_std
1164
+ value: 3.1853356915569653
1165
+ - type: nauc_ndcg_at_3_diff1
1166
+ value: 44.18163998834299
1167
+ - type: nauc_ndcg_at_3_max
1168
+ value: 38.352891017864636
1169
+ - type: nauc_ndcg_at_3_std
1170
+ value: -0.8235767021150929
1171
+ - type: nauc_ndcg_at_5_diff1
1172
+ value: 43.41374688421302
1173
+ - type: nauc_ndcg_at_5_max
1174
+ value: 40.390365601593956
1175
+ - type: nauc_ndcg_at_5_std
1176
+ value: 1.6743650108127537
1177
+ - type: nauc_precision_at_1000_diff1
1178
+ value: -9.711058370691381
1179
+ - type: nauc_precision_at_1000_max
1180
+ value: 6.97321343449286
1181
+ - type: nauc_precision_at_1000_std
1182
+ value: 7.933531916622121
1183
+ - type: nauc_precision_at_100_diff1
1184
+ value: -8.247029644152319
1185
+ - type: nauc_precision_at_100_max
1186
+ value: 10.86740140944616
1187
+ - type: nauc_precision_at_100_std
1188
+ value: 9.581885544675918
1189
+ - type: nauc_precision_at_10_diff1
1190
+ value: -2.409043695429943
1191
+ - type: nauc_precision_at_10_max
1192
+ value: 21.04733206074314
1193
+ - type: nauc_precision_at_10_std
1194
+ value: 10.03334651647101
1195
+ - type: nauc_precision_at_1_diff1
1196
+ value: 62.23623985611396
1197
+ - type: nauc_precision_at_1_max
1198
+ value: 45.04945770947091
1199
+ - type: nauc_precision_at_1_std
1200
+ value: -0.8804967656575725
1201
+ - type: nauc_precision_at_20_diff1
1202
+ value: -5.230303656931621
1203
+ - type: nauc_precision_at_20_max
1204
+ value: 17.77799716919181
1205
+ - type: nauc_precision_at_20_std
1206
+ value: 10.739127998618654
1207
+ - type: nauc_precision_at_3_diff1
1208
+ value: 10.40376424999862
1209
+ - type: nauc_precision_at_3_max
1210
+ value: 30.933333400254035
1211
+ - type: nauc_precision_at_3_std
1212
+ value: 6.126209127968004
1213
+ - type: nauc_precision_at_5_diff1
1214
+ value: 3.147398101830739
1215
+ - type: nauc_precision_at_5_max
1216
+ value: 27.1746309955971
1217
+ - type: nauc_precision_at_5_std
1218
+ value: 8.874723615388788
1219
+ - type: nauc_recall_at_1000_diff1
1220
+ value: 5.055940692380908
1221
+ - type: nauc_recall_at_1000_max
1222
+ value: 22.42031123370267
1223
+ - type: nauc_recall_at_1000_std
1224
+ value: 27.75476692527869
1225
+ - type: nauc_recall_at_100_diff1
1226
+ value: 17.86391178198642
1227
+ - type: nauc_recall_at_100_max
1228
+ value: 34.776134863678955
1229
+ - type: nauc_recall_at_100_std
1230
+ value: 18.96377158778504
1231
+ - type: nauc_recall_at_10_diff1
1232
+ value: 24.863097695413597
1233
+ - type: nauc_recall_at_10_max
1234
+ value: 37.697411651507444
1235
+ - type: nauc_recall_at_10_std
1236
+ value: 9.519849994253967
1237
+ - type: nauc_recall_at_1_diff1
1238
+ value: 49.56605205141156
1239
+ - type: nauc_recall_at_1_max
1240
+ value: 26.251096698710384
1241
+ - type: nauc_recall_at_1_std
1242
+ value: -4.580748485387834
1243
+ - type: nauc_recall_at_20_diff1
1244
+ value: 22.440602811005636
1245
+ - type: nauc_recall_at_20_max
1246
+ value: 39.538861316515
1247
+ - type: nauc_recall_at_20_std
1248
+ value: 11.363269553121468
1249
+ - type: nauc_recall_at_3_diff1
1250
+ value: 32.80302839873736
1251
+ - type: nauc_recall_at_3_max
1252
+ value: 32.53105685012729
1253
+ - type: nauc_recall_at_3_std
1254
+ value: -0.7140166410605693
1255
+ - type: nauc_recall_at_5_diff1
1256
+ value: 29.375386639154865
1257
+ - type: nauc_recall_at_5_max
1258
+ value: 36.91045781164083
1259
+ - type: nauc_recall_at_5_std
1260
+ value: 4.725419050262578
1261
+ - type: ndcg_at_1
1262
+ value: 67.13900000000001
1263
+ - type: ndcg_at_10
1264
+ value: 72.392
1265
+ - type: ndcg_at_100
1266
+ value: 75.25800000000001
1267
+ - type: ndcg_at_1000
1268
+ value: 75.982
1269
+ - type: ndcg_at_20
1270
+ value: 73.783
1271
+ - type: ndcg_at_3
1272
+ value: 67.269
1273
+ - type: ndcg_at_5
1274
+ value: 69.807
1275
+ - type: precision_at_1
1276
+ value: 67.13900000000001
1277
+ - type: precision_at_10
1278
+ value: 13.327
1279
+ - type: precision_at_100
1280
+ value: 1.5559999999999998
1281
+ - type: precision_at_1000
1282
+ value: 0.164
1283
+ - type: precision_at_20
1284
+ value: 7.119000000000001
1285
+ - type: precision_at_3
1286
+ value: 35.599
1287
+ - type: precision_at_5
1288
+ value: 23.936
1289
+ - type: recall_at_1
1290
+ value: 47.370000000000005
1291
+ - type: recall_at_10
1292
+ value: 82.16
1293
+ - type: recall_at_100
1294
+ value: 93.34
1295
+ - type: recall_at_1000
1296
+ value: 98.202
1297
+ - type: recall_at_20
1298
+ value: 86.687
1299
+ - type: recall_at_3
1300
+ value: 69.319
1301
+ - type: recall_at_5
1302
+ value: 75.637
1303
+ task:
1304
+ type: Retrieval
1305
+ - dataset:
1306
+ config: default
1307
+ name: MTEB RuReviewsClassification (default)
1308
+ revision: f6d2c31f4dc6b88f468552750bfec05b4b41b05a
1309
+ split: test
1310
+ type: ai-forever/ru-reviews-classification
1311
+ metrics:
1312
+ - type: accuracy
1313
+ value: 75.0537109375
1314
+ - type: f1
1315
+ value: 74.00523205209554
1316
+ - type: f1_weighted
1317
+ value: 74.00436782840376
1318
+ - type: main_score
1319
+ value: 75.0537109375
1320
+ task:
1321
+ type: Classification
1322
+ - dataset:
1323
+ config: default
1324
+ name: MTEB RuSTSBenchmarkSTS (default)
1325
+ revision: 7cf24f325c6da6195df55bef3d86b5e0616f3018
1326
+ split: test
1327
+ type: ai-forever/ru-stsbenchmark-sts
1328
+ metrics:
1329
+ - type: cosine_pearson
1330
+ value: 81.10255413476487
1331
+ - type: cosine_spearman
1332
+ value: 81.40020843157141
1333
+ - type: euclidean_pearson
1334
+ value: 81.25155479902466
1335
+ - type: euclidean_spearman
1336
+ value: 81.40020831064922
1337
+ - type: main_score
1338
+ value: 81.40020843157141
1339
+ - type: manhattan_pearson
1340
+ value: 81.1493715249014
1341
+ - type: manhattan_spearman
1342
+ value: 81.30973667941649
1343
+ - type: pearson
1344
+ value: 81.10255413476487
1345
+ - type: spearman
1346
+ value: 81.40020843157141
1347
+ task:
1348
+ type: STS
1349
+ - dataset:
1350
+ config: default
1351
+ name: MTEB RuSciBenchGRNTIClassification (default)
1352
+ revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
1353
+ split: test
1354
+ type: ai-forever/ru-scibench-grnti-classification
1355
+ metrics:
1356
+ - type: accuracy
1357
+ value: 69.8974609375
1358
+ - type: f1
1359
+ value: 68.57837564785511
1360
+ - type: f1_weighted
1361
+ value: 68.59030489460784
1362
+ - type: main_score
1363
+ value: 69.8974609375
1364
+ task:
1365
+ type: Classification
1366
+ - dataset:
1367
+ config: default
1368
+ name: MTEB RuSciBenchGRNTIClusteringP2P (default)
1369
+ revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
1370
+ split: test
1371
+ type: ai-forever/ru-scibench-grnti-classification
1372
+ metrics:
1373
+ - type: main_score
1374
+ value: 67.03880348548029
1375
+ - type: v_measure
1376
+ value: 67.03880348548029
1377
+ - type: v_measure_std
1378
+ value: 0.6126278133139618
1379
+ task:
1380
+ type: Clustering
1381
+ - dataset:
1382
+ config: default
1383
+ name: MTEB RuSciBenchOECDClassification (default)
1384
+ revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
1385
+ split: test
1386
+ type: ai-forever/ru-scibench-oecd-classification
1387
+ metrics:
1388
+ - type: accuracy
1389
+ value: 54.63378906250001
1390
+ - type: f1
1391
+ value: 51.34306420274629
1392
+ - type: f1_weighted
1393
+ value: 51.33495867493914
1394
+ - type: main_score
1395
+ value: 54.63378906250001
1396
+ task:
1397
+ type: Classification
1398
+ - dataset:
1399
+ config: default
1400
+ name: MTEB RuSciBenchOECDClusteringP2P (default)
1401
+ revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
1402
+ split: test
1403
+ type: ai-forever/ru-scibench-oecd-classification
1404
+ metrics:
1405
+ - type: main_score
1406
+ value: 56.55947121159027
1407
+ - type: v_measure
1408
+ value: 56.55947121159027
1409
+ - type: v_measure_std
1410
+ value: 0.5498882006880662
1411
+ task:
1412
+ type: Clustering
1413
+ - dataset:
1414
+ config: ru
1415
+ name: MTEB STS22 (ru)
1416
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
1417
+ split: test
1418
+ type: mteb/sts22-crosslingual-sts
1419
+ metrics:
1420
+ - type: cosine_pearson
1421
+ value: 61.833294921667914
1422
+ - type: cosine_spearman
1423
+ value: 63.53967536726357
1424
+ - type: euclidean_pearson
1425
+ value: 60.382865218855805
1426
+ - type: euclidean_spearman
1427
+ value: 63.53967536726357
1428
+ - type: main_score
1429
+ value: 63.53967536726357
1430
+ - type: manhattan_pearson
1431
+ value: 60.24879015304578
1432
+ - type: manhattan_spearman
1433
+ value: 63.42305760430092
1434
+ - type: pearson
1435
+ value: 61.833294921667914
1436
+ - type: spearman
1437
+ value: 63.53967536726357
1438
+ task:
1439
+ type: STS
1440
+ - dataset:
1441
+ config: default
1442
+ name: MTEB SensitiveTopicsClassification (default)
1443
+ revision: 416b34a802308eac30e4192afc0ff99bb8dcc7f2
1444
+ split: test
1445
+ type: ai-forever/sensitive-topics-classification
1446
+ metrics:
1447
+ - type: accuracy
1448
+ value: 39.8193359375
1449
+ - type: f1
1450
+ value: 55.46591740935434
1451
+ - type: lrap
1452
+ value: 66.50980631510454
1453
+ - type: main_score
1454
+ value: 39.8193359375
1455
+ task:
1456
+ type: MultilabelClassification
1457
+ - dataset:
1458
+ config: default
1459
+ name: MTEB TERRa (default)
1460
+ revision: 7b58f24536063837d644aab9a023c62199b2a612
1461
+ split: dev
1462
+ type: ai-forever/terra-pairclassification
1463
+ metrics:
1464
+ - type: cosine_accuracy
1465
+ value: 66.77524429967427
1466
+ - type: cosine_accuracy_threshold
1467
+ value: 55.58975338935852
1468
+ - type: cosine_ap
1469
+ value: 66.4567219323658
1470
+ - type: cosine_f1
1471
+ value: 70.64676616915423
1472
+ - type: cosine_f1_threshold
1473
+ value: 45.55969536304474
1474
+ - type: cosine_precision
1475
+ value: 57.028112449799195
1476
+ - type: cosine_recall
1477
+ value: 92.81045751633987
1478
+ - type: dot_accuracy
1479
+ value: 66.77524429967427
1480
+ - type: dot_accuracy_threshold
1481
+ value: 55.589759349823
1482
+ - type: dot_ap
1483
+ value: 66.4567219323658
1484
+ - type: dot_f1
1485
+ value: 70.64676616915423
1486
+ - type: dot_f1_threshold
1487
+ value: 45.55969536304474
1488
+ - type: dot_precision
1489
+ value: 57.028112449799195
1490
+ - type: dot_recall
1491
+ value: 92.81045751633987
1492
+ - type: euclidean_accuracy
1493
+ value: 66.77524429967427
1494
+ - type: euclidean_accuracy_threshold
1495
+ value: 94.24455165863037
1496
+ - type: euclidean_ap
1497
+ value: 66.4567219323658
1498
+ - type: euclidean_f1
1499
+ value: 70.64676616915423
1500
+ - type: euclidean_f1_threshold
1501
+ value: 104.34587001800537
1502
+ - type: euclidean_precision
1503
+ value: 57.028112449799195
1504
+ - type: euclidean_recall
1505
+ value: 92.81045751633987
1506
+ - type: main_score
1507
+ value: 66.4567219323658
1508
+ - type: manhattan_accuracy
1509
+ value: 66.77524429967427
1510
+ - type: manhattan_accuracy_threshold
1511
+ value: 2865.5345916748047
1512
+ - type: manhattan_ap
1513
+ value: 66.26659863769075
1514
+ - type: manhattan_f1
1515
+ value: 70.8542713567839
1516
+ - type: manhattan_f1_threshold
1517
+ value: 3212.3912811279297
1518
+ - type: manhattan_precision
1519
+ value: 57.55102040816327
1520
+ - type: manhattan_recall
1521
+ value: 92.15686274509804
1522
+ - type: max_accuracy
1523
+ value: 66.77524429967427
1524
+ - type: max_ap
1525
+ value: 66.4567219323658
1526
+ - type: max_f1
1527
+ value: 70.8542713567839
1528
+ - type: max_precision
1529
+ value: 57.55102040816327
1530
+ - type: max_recall
1531
+ value: 92.81045751633987
1532
+ - type: similarity_accuracy
1533
+ value: 66.77524429967427
1534
+ - type: similarity_accuracy_threshold
1535
+ value: 55.58975338935852
1536
+ - type: similarity_ap
1537
+ value: 66.4567219323658
1538
+ - type: similarity_f1
1539
+ value: 70.64676616915423
1540
+ - type: similarity_f1_threshold
1541
+ value: 45.55969536304474
1542
+ - type: similarity_precision
1543
+ value: 57.028112449799195
1544
+ - type: similarity_recall
1545
+ value: 92.81045751633987
1546
+ task:
1547
+ type: PairClassification
1548
  license: mit
1549
  language:
1550
+ - ru
1551
+ - en
1552
+ tags:
1553
+ - mteb
1554
+ - transformers
1555
+ - sentence-transformers
1556
+ base_model: ai-forever/FRED-T5-1.7B
1557
  ---
1558
 
1559
  ---