Update README.md
Browse files
README.md
CHANGED
|
@@ -224,7 +224,7 @@ print("Generated Translations:", results_detokenized)
|
|
| 224 |
|
| 225 |
## Evaluation
|
| 226 |
|
| 227 |
-
Below are the evaluation results on Flores-200 dev and devtest compared to NLLB-3.3 ([Costa-jussà et al., 2022](https://arxiv.org/abs/2207.04672)) for CA-XX and XX-CA directions. The metrics have been computed excluding Asturian, Aranese, and Aragonese as we report them separately. The evaluation was conducted using [MT Lens](https://github.com/langtech-bsc/mt-evaluation). We report the following metrics:
|
| 228 |
|
| 229 |
<details>
|
| 230 |
<summary>Click to show metrics details</summary>
|
|
@@ -244,15 +244,183 @@ Below are the evaluation results on Flores-200 dev and devtest compared to NLLB-
|
|
| 244 |
| | Bleu ↑ | Ter ↓ | ChrF ↑ | Comet ↑ | Comet-kiwi ↑ | Bleurt ↑ |
|
| 245 |
|:-----------------------|-------:|------:|-------:|--------:|-------------:|---------:|
|
| 246 |
| **CA-XX** | | | | | | |
|
| 247 |
-
| SalamandraTA | **27.41** | **60.88** | **56.27** | 0.86 | 0.82 | 0.76 |
|
| 248 |
| nllb 3.3B | 26.84 | 61.75 | 55.7 | 0.86 | 0.82 | 0.76 |
|
| 249 |
| **XX-CA** | | | | | | |
|
| 250 |
-
| SalamandraTA | **30.75** | **57.66** | **57.6** | 0.85 | 0.81 | 0.73 |
|
| 251 |
| nllb 3.3B | 29.76 | 58.25 | 56.75 | 0.85 | **0.82** | 0.73 |
|
| 252 |
|
| 253 |
|
| 254 |
<details>
|
| 255 |
-
<summary>Click to show full table</summary>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 256 |
</details>
|
| 257 |
|
| 258 |
#### Flores200-devtest
|
|
@@ -260,18 +428,185 @@ Below are the evaluation results on Flores-200 dev and devtest compared to NLLB-
|
|
| 260 |
| | Bleu ↑ | Ter ↓ | ChrF ↑ | Comet ↑ | Comet-kiwi ↑ | Bleurt ↑ |
|
| 261 |
|:-----------------------|-------:|------:|-------:|--------:|-------------:|---------:|
|
| 262 |
| **CA-XX** | | | | | | |
|
| 263 |
-
| SalamandraTA | **27.09** | **61.06** | **56.41** | 0.86 | 0.81 | 0.75 |
|
| 264 |
| nllb 3.3B | 26.7 | 61.74 | 55.85 | 0.86 | **0.82** | **0.76** |
|
| 265 |
| **XX-CA** | | | | | | |
|
| 266 |
-
| SalamandraTA | **31** | **57.46** | **57.96** | 0.85 | 0.81 | 0.73 |
|
| 267 |
| nllb 3.3B | 30.31 | 58.26 | 57.12 | 0.85 | **0.82** | 0.73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 268 |
<details>
|
| 269 |
-
<summary>Click to show full table</summary>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 270 |
|
| 271 |
</details>
|
| 272 |
|
| 273 |
|
| 274 |
|
|
|
|
| 275 |
## Ethical Considerations and Limitations
|
| 276 |
|
| 277 |
---
|
|
|
|
| 224 |
|
| 225 |
## Evaluation
|
| 226 |
|
| 227 |
+
Below are the evaluation results on Flores-200 dev and devtest compared to NLLB-3.3 ([Costa-jussà et al., 2022](https://arxiv.org/abs/2207.04672)) for CA-XX and XX-CA directions. The metrics have been computed excluding Asturian, Aranese, and Aragonese as we report them separately. The evaluation was conducted using [MT Lens](https://github.com/langtech-bsc/mt-evaluation) following the standard setting (beam search with beam size 5, limiting the translation length to 250 tokens). We report the following metrics:
|
| 228 |
|
| 229 |
<details>
|
| 230 |
<summary>Click to show metrics details</summary>
|
|
|
|
| 244 |
| | Bleu ↑ | Ter ↓ | ChrF ↑ | Comet ↑ | Comet-kiwi ↑ | Bleurt ↑ |
|
| 245 |
|:-----------------------|-------:|------:|-------:|--------:|-------------:|---------:|
|
| 246 |
| **CA-XX** | | | | | | |
|
| 247 |
+
| SalamandraTA-2B | **27.41** | **60.88** | **56.27** | 0.86 | 0.82 | 0.76 |
|
| 248 |
| nllb 3.3B | 26.84 | 61.75 | 55.7 | 0.86 | 0.82 | 0.76 |
|
| 249 |
| **XX-CA** | | | | | | |
|
| 250 |
+
| SalamandraTA-2B | **30.75** | **57.66** | **57.6** | 0.85 | 0.81 | 0.73 |
|
| 251 |
| nllb 3.3B | 29.76 | 58.25 | 56.75 | 0.85 | **0.82** | 0.73 |
|
| 252 |
|
| 253 |
|
| 254 |
<details>
|
| 255 |
+
<summary>Click to show full table CA-XX Flores-dev</summary>
|
| 256 |
+
|
| 257 |
+
| | source | target | Bleu ↑ | Ter ↓ | ChrF ↑ | Comet ↑ | Comet-kiwi ↑ | Bleurt ↑ |
|
| 258 |
+
|:-----------------------|:---------|:---------|-------:|------:|-------:|--------:|-------------:|---------:|
|
| 259 |
+
| nllb 3.3B | ca | sv | 33.05 | 53.98 | 60.09 | 0.88 | 0.83 | 0.79 |
|
| 260 |
+
| SalamandraTA-2B | ca | sv | 30.62 | 55.4 | 57.77 | 0.87 | 0.81 | 0.78 |
|
| 261 |
+
| | | | | | | | | |
|
| 262 |
+
| SalamandraTA-2B | ca | sl | 25.74 | 63.78 | 54.29 | 0.88 | 0.83 | 0.81 |
|
| 263 |
+
| nllb 3.3B | ca | sl | 25.04 | 65.02 | 53.08 | 0.88 | 0.83 | 0.82 |
|
| 264 |
+
| | | | | | | | | |
|
| 265 |
+
| SalamandraTA-2B | ca | sk | 26.03 | 62.58 | 53.53 | 0.89 | 0.84 | 0.8 |
|
| 266 |
+
| nllb 3.3B | ca | sk | 25.59 | 63.17 | 53.28 | 0.89 | 0.84 | 0.8 |
|
| 267 |
+
| | | | | | | | | |
|
| 268 |
+
| SalamandraTA-2B | ca | ro | 33.08 | 54.36 | 59.18 | 0.89 | 0.85 | 0.8 |
|
| 269 |
+
| nllb 3.3B | ca | ro | 31.91 | 55.46 | 58.36 | 0.89 | 0.85 | 0.81 |
|
| 270 |
+
| | | | | | | | | |
|
| 271 |
+
| SalamandraTA-2B | ca | pt | 37.6 | 48.82 | 62.73 | 0.88 | 0.84 | 0.76 |
|
| 272 |
+
| nllb 3.3B | ca | pt | 36.85 | 49.56 | 62.02 | 0.88 | 0.85 | 0.76 |
|
| 273 |
+
| | | | | | | | | |
|
| 274 |
+
| nllb 3.3B | ca | pl | 17.97 | 73.06 | 47.94 | 0.88 | 0.84 | 0.78 |
|
| 275 |
+
| SalamandraTA-2B | ca | pl | 17.85 | 72.67 | 47.77 | 0.88 | 0.84 | 0.78 |
|
| 276 |
+
| | | | | | | | | |
|
| 277 |
+
| SalamandraTA-2B | ca | nl | 23.88 | 64.95 | 54.46 | 0.85 | 0.84 | 0.75 |
|
| 278 |
+
| nllb 3.3B | ca | nl | 23.26 | 66.46 | 54.17 | 0.85 | 0.85 | 0.75 |
|
| 279 |
+
| | | | | | | | | |
|
| 280 |
+
| SalamandraTA-2B | ca | mt | 25.62 | 59.08 | 60.83 | 0.69 | 0.61 | 0.43 |
|
| 281 |
+
| nllb 3.3B | ca | mt | 25.37 | 59.47 | 60.1 | 0.69 | 0.63 | 0.39 |
|
| 282 |
+
| | | | | | | | | |
|
| 283 |
+
| SalamandraTA-2B | ca | lv | 21.23 | 71.48 | 49.47 | 0.82 | 0.79 | 0.73 |
|
| 284 |
+
| nllb 3.3B | ca | lv | 20.56 | 70.88 | 50.07 | 0.85 | 0.78 | 0.77 |
|
| 285 |
+
| | | | | | | | | |
|
| 286 |
+
| SalamandraTA-2B | ca | lt | 19.92 | 71.02 | 50.88 | 0.87 | 0.8 | 0.81 |
|
| 287 |
+
| nllb 3.3B | ca | lt | 18.82 | 71.8 | 51.84 | 0.87 | 0.82 | 0.82 |
|
| 288 |
+
| | | | | | | | | |
|
| 289 |
+
| SalamandraTA-2B | ca | it | 26.76 | 60.67 | 56.3 | 0.88 | 0.85 | 0.77 |
|
| 290 |
+
| nllb 3.3B | ca | it | 26.42 | 61.47 | 55.66 | 0.87 | 0.86 | 0.77 |
|
| 291 |
+
| | | | | | | | | |
|
| 292 |
+
| SalamandraTA-2B | ca | hu | 22.8 | 66.41 | 53.41 | 0.86 | 0.82 | 0.85 |
|
| 293 |
+
| nllb 3.3B | ca | hu | 21.2 | 68.54 | 51.99 | 0.87 | 0.83 | 0.87 |
|
| 294 |
+
| | | | | | | | | |
|
| 295 |
+
| SalamandraTA-2B | ca | hr | 26.24 | 61.83 | 55.87 | 0.89 | 0.84 | 0.81 |
|
| 296 |
+
| nllb 3.3B | ca | hr | 24.04 | 64.25 | 53.79 | 0.89 | 0.85 | 0.82 |
|
| 297 |
+
| | | | | | | | | |
|
| 298 |
+
| nllb 3.3B | ca | gl | 32.85 | 51.69 | 59.33 | 0.87 | 0.85 | 0.72 |
|
| 299 |
+
| SalamandraTA-2B | ca | gl | 31.84 | 52.52 | 59.16 | 0.87 | 0.84 | 0.71 |
|
| 300 |
+
| | | | | | | | | |
|
| 301 |
+
| SalamandraTA-2B | ca | ga | 25.24 | 63.36 | 53.24 | 0.78 | 0.64 | 0.62 |
|
| 302 |
+
| nllb 3.3B | ca | ga | 23.51 | 66.54 | 51.53 | 0.77 | 0.66 | 0.62 |
|
| 303 |
+
| | | | | | | | | |
|
| 304 |
+
| SalamandraTA-2B | ca | fr | 40.14 | 48.34 | 64.24 | 0.86 | 0.84 | 0.73 |
|
| 305 |
+
| nllb 3.3B | ca | fr | 39.8 | 48.96 | 63.97 | 0.86 | 0.85 | 0.74 |
|
| 306 |
+
| | | | | | | | | |
|
| 307 |
+
| nllb 3.3B | ca | fi | 18.63 | 71.42 | 52.71 | 0.89 | 0.82 | 0.82 |
|
| 308 |
+
| SalamandraTA-2B | ca | fi | 18.49 | 71.46 | 52.09 | 0.88 | 0.8 | 0.8 |
|
| 309 |
+
| | | | | | | | | |
|
| 310 |
+
| SalamandraTA-2B | ca | eu | 18.75 | 71.09 | 57.05 | 0.87 | 0.81 | 0.8 |
|
| 311 |
+
| nllb 3.3B | ca | eu | 13.15 | 77.69 | 50.35 | 0.83 | 0.75 | 0.75 |
|
| 312 |
+
| | | | | | | | | |
|
| 313 |
+
| SalamandraTA-2B | ca | et | 22.03 | 67.55 | 54.87 | 0.88 | 0.8 | 0.79 |
|
| 314 |
+
| nllb 3.3B | ca | et | 20.07 | 70.66 | 53.19 | 0.88 | 0.81 | 0.8 |
|
| 315 |
+
| | | | | | | | | |
|
| 316 |
+
| nllb 3.3B | ca | es | 25.59 | 60.39 | 53.7 | 0.86 | 0.86 | 0.74 |
|
| 317 |
+
| SalamandraTA-2B | ca | es | 24.46 | 61.54 | 53.02 | 0.86 | 0.86 | 0.74 |
|
| 318 |
+
| | | | | | | | | |
|
| 319 |
+
| nllb 3.3B | ca | en | 49.62 | 37.33 | 71.65 | 0.89 | 0.86 | 0.8 |
|
| 320 |
+
| SalamandraTA-2B | ca | en | 46.62 | 40.03 | 70.23 | 0.88 | 0.86 | 0.79 |
|
| 321 |
+
| | | | | | | | | |
|
| 322 |
+
| SalamandraTA-2B | ca | el | 23.38 | 63 | 50.03 | 0.87 | 0.84 | 0.74 |
|
| 323 |
+
| nllb 3.3B | ca | el | 22.62 | 63.73 | 49.5 | 0.87 | 0.84 | 0.74 |
|
| 324 |
+
| | | | | | | | | |
|
| 325 |
+
| SalamandraTA-2B | ca | de | 31.89 | 57.12 | 59.07 | 0.84 | 0.83 | 0.75 |
|
| 326 |
+
| nllb 3.3B | ca | de | 31.19 | 57.87 | 58.47 | 0.85 | 0.84 | 0.76 |
|
| 327 |
+
| | | | | | | | | |
|
| 328 |
+
| SalamandraTA-2B | ca | da | 34.69 | 53.31 | 61.11 | 0.87 | 0.82 | 0.75 |
|
| 329 |
+
| nllb 3.3B | ca | da | 34.32 | 54.2 | 60.2 | 0.88 | 0.83 | 0.77 |
|
| 330 |
+
| | | | | | | | | |
|
| 331 |
+
| SalamandraTA-2B | ca | cs | 25.67 | 63.37 | 53.07 | 0.89 | 0.85 | 0.79 |
|
| 332 |
+
| nllb 3.3B | ca | cs | 25.02 | 63.59 | 52.43 | 0.89 | 0.85 | 0.79 |
|
| 333 |
+
| | | | | | | | | |
|
| 334 |
+
| SalamandraTA-2B | ca | bg | 32.09 | 57.01 | 59.4 | 0.89 | 0.85 | 0.84 |
|
| 335 |
+
| nllb 3.3B | ca | bg | 31.24 | 58.41 | 58.81 | 0.89 | 0.86 | 0.85 |
|
| 336 |
+
|
| 337 |
+
</details>
|
| 338 |
+
|
| 339 |
+
|
| 340 |
+
<details>
|
| 341 |
+
<summary>Click to show full table XX-CA Flores-dev</summary>
|
| 342 |
+
|
| 343 |
+
| | source | target | Bleu ↑ | Ter ↓ | ChrF ↑ | Comet ↑ | Comet-kiwi ↑ | Bleurt ↑ |
|
| 344 |
+
|:-----------------------|:---------|:---------|-------:|------:|-------:|--------:|-------------:|---------:|
|
| 345 |
+
| SalamandraTA-2B | sv | ca | 34.21 | 53 | 59.52 | 0.86 | 0.83 | 0.74 |
|
| 346 |
+
| nllb 3.3B | sv | ca | 33.03 | 53.42 | 59.02 | 0.86 | 0.84 | 0.75 |
|
| 347 |
+
| | | | | | | | | |
|
| 348 |
+
| SalamandraTA-2B | sl | ca | 28.98 | 59.95 | 56.24 | 0.85 | 0.82 | 0.72 |
|
| 349 |
+
| nllb 3.3B | sl | ca | 27.51 | 61.23 | 54.96 | 0.85 | 0.83 | 0.72 |
|
| 350 |
+
| | | | | | | | | |
|
| 351 |
+
| SalamandraTA-2B | sk | ca | 30.61 | 58.1 | 57.53 | 0.86 | 0.81 | 0.73 |
|
| 352 |
+
| nllb 3.3B | sk | ca | 29.24 | 58.93 | 56.29 | 0.86 | 0.83 | 0.73 |
|
| 353 |
+
| | | | | | | | | |
|
| 354 |
+
| SalamandraTA-2B | ro | ca | 33.73 | 54.23 | 60.11 | 0.87 | 0.83 | 0.75 |
|
| 355 |
+
| nllb 3.3B | ro | ca | 32.9 | 54.71 | 59.56 | 0.87 | 0.84 | 0.75 |
|
| 356 |
+
| | | | | | | | | |
|
| 357 |
+
| SalamandraTA-2B | pt | ca | 35.99 | 50.64 | 61.52 | 0.87 | 0.84 | 0.76 |
|
| 358 |
+
| nllb 3.3B | pt | ca | 34.63 | 51.15 | 60.68 | 0.87 | 0.84 | 0.76 |
|
| 359 |
+
| | | | | | | | | |
|
| 360 |
+
| SalamandraTA-2B | pl | ca | 25.77 | 64.99 | 53.46 | 0.84 | 0.82 | 0.71 |
|
| 361 |
+
| nllb 3.3B | pl | ca | 24.41 | 65.69 | 52.45 | 0.85 | 0.83 | 0.71 |
|
| 362 |
+
| | | | | | | | | |
|
| 363 |
+
| SalamandraTA-2B | nl | ca | 26.04 | 64.09 | 53.64 | 0.84 | 0.84 | 0.71 |
|
| 364 |
+
| nllb 3.3B | nl | ca | 25.35 | 64.64 | 53.15 | 0.84 | 0.85 | 0.71 |
|
| 365 |
+
| | | | | | | | | |
|
| 366 |
+
| SalamandraTA-2B | mt | ca | 37.51 | 50.18 | 62.42 | 0.79 | 0.69 | 0.75 |
|
| 367 |
+
| nllb 3.3B | mt | ca | 36.29 | 51.01 | 61.24 | 0.79 | 0.7 | 0.75 |
|
| 368 |
+
| | | | | | | | | |
|
| 369 |
+
| SalamandraTA-2B | lv | ca | 27.14 | 62.61 | 55.6 | 0.84 | 0.78 | 0.7 |
|
| 370 |
+
| nllb 3.3B | lv | ca | 27.02 | 61.12 | 54.28 | 0.84 | 0.79 | 0.71 |
|
| 371 |
+
| | | | | | | | | |
|
| 372 |
+
| SalamandraTA-2B | lt | ca | 27.76 | 61.3 | 54.52 | 0.84 | 0.76 | 0.71 |
|
| 373 |
+
| nllb 3.3B | lt | ca | 26.05 | 62.75 | 53.4 | 0.84 | 0.77 | 0.71 |
|
| 374 |
+
| | | | | | | | | |
|
| 375 |
+
| SalamandraTA-2B | it | ca | 28.44 | 61.09 | 57.12 | 0.87 | 0.85 | 0.74 |
|
| 376 |
+
| nllb 3.3B | it | ca | 27.79 | 61.42 | 56.62 | 0.87 | 0.86 | 0.74 |
|
| 377 |
+
| | | | | | | | | |
|
| 378 |
+
| SalamandraTA-2B | hu | ca | 28.15 | 60.01 | 55.29 | 0.85 | 0.81 | 0.72 |
|
| 379 |
+
| nllb 3.3B | hu | ca | 27.06 | 60.44 | 54.38 | 0.85 | 0.83 | 0.72 |
|
| 380 |
+
| | | | | | | | | |
|
| 381 |
+
| SalamandraTA-2B | hr | ca | 29.89 | 58.61 | 56.62 | 0.85 | 0.82 | 0.72 |
|
| 382 |
+
| nllb 3.3B | hr | ca | 28.23 | 59.55 | 55.37 | 0.86 | 0.84 | 0.73 |
|
| 383 |
+
| | | | | | | | | |
|
| 384 |
+
| nllb 3.3B | gl | ca | 34.28 | 52.34 | 60.86 | 0.87 | 0.85 | 0.76 |
|
| 385 |
+
| SalamandraTA-2B | gl | ca | 32.14 | 54.03 | 60.3 | 0.87 | 0.84 | 0.75 |
|
| 386 |
+
| | | | | | | | | |
|
| 387 |
+
| SalamandraTA-2B | ga | ca | 28.59 | 61.13 | 55.61 | 0.8 | 0.69 | 0.68 |
|
| 388 |
+
| nllb 3.3B | ga | ca | 28.09 | 61.12 | 54.55 | 0.8 | 0.7 | 0.68 |
|
| 389 |
+
| | | | | | | | | |
|
| 390 |
+
| SalamandraTA-2B | fr | ca | 34.53 | 52.9 | 60.38 | 0.87 | 0.83 | 0.76 |
|
| 391 |
+
| nllb 3.3B | fr | ca | 33.61 | 53.57 | 59.73 | 0.87 | 0.84 | 0.76 |
|
| 392 |
+
| | | | | | | | | |
|
| 393 |
+
| SalamandraTA-2B | fi | ca | 26.71 | 62.19 | 54.09 | 0.86 | 0.8 | 0.71 |
|
| 394 |
+
| nllb 3.3B | fi | ca | 26.31 | 62.6 | 54.06 | 0.86 | 0.82 | 0.71 |
|
| 395 |
+
| | | | | | | | | |
|
| 396 |
+
| SalamandraTA-2B | eu | ca | 27.93 | 60.26 | 55.27 | 0.87 | 0.83 | 0.73 |
|
| 397 |
+
| nllb 3.3B | eu | ca | 26.43 | 63.76 | 53.75 | 0.86 | 0.82 | 0.72 |
|
| 398 |
+
| | | | | | | | | |
|
| 399 |
+
| SalamandraTA-2B | et | ca | 30.03 | 58.25 | 56.88 | 0.86 | 0.79 | 0.72 |
|
| 400 |
+
| nllb 3.3B | et | ca | 27.56 | 59.95 | 54.92 | 0.86 | 0.8 | 0.72 |
|
| 401 |
+
| | | | | | | | | |
|
| 402 |
+
| nllb 3.3B | es | ca | 25.33 | 64.23 | 55.1 | 0.86 | 0.84 | 0.73 |
|
| 403 |
+
| SalamandraTA-2B | es | ca | 22.95 | 67.1 | 53.67 | 0.86 | 0.84 | 0.72 |
|
| 404 |
+
| | | | | | | | | |
|
| 405 |
+
| SalamandraTA-2B | en | ca | 43.55 | 42.62 | 67.03 | 0.88 | 0.85 | 0.78 |
|
| 406 |
+
| nllb 3.3B | en | ca | 42.21 | 43.63 | 65.95 | 0.88 | 0.85 | 0.78 |
|
| 407 |
+
| | | | | | | | | |
|
| 408 |
+
| SalamandraTA-2B | el | ca | 28.52 | 60.34 | 54.99 | 0.85 | 0.83 | 0.71 |
|
| 409 |
+
| nllb 3.3B | el | ca | 27.36 | 60.49 | 54.76 | 0.85 | 0.85 | 0.72 |
|
| 410 |
+
| | | | | | | | | |
|
| 411 |
+
| SalamandraTA-2B | de | ca | 33.07 | 54.46 | 59.06 | 0.85 | 0.84 | 0.74 |
|
| 412 |
+
| nllb 3.3B | de | ca | 31.43 | 56.05 | 57.95 | 0.86 | 0.85 | 0.74 |
|
| 413 |
+
| | | | | | | | | |
|
| 414 |
+
| SalamandraTA-2B | da | ca | 34.6 | 53.22 | 60.43 | 0.86 | 0.83 | 0.75 |
|
| 415 |
+
| nllb 3.3B | da | ca | 32.71 | 54.2 | 58.9 | 0.86 | 0.84 | 0.75 |
|
| 416 |
+
| | | | | | | | | |
|
| 417 |
+
| SalamandraTA-2B | cs | ca | 30.92 | 57.54 | 57.71 | 0.86 | 0.82 | 0.73 |
|
| 418 |
+
| nllb 3.3B | cs | ca | 29.02 | 58.78 | 56.44 | 0.86 | 0.83 | 0.73 |
|
| 419 |
+
| | | | | | | | | |
|
| 420 |
+
| SalamandraTA-2B | bg | ca | 31.68 | 56.32 | 58.61 | 0.85 | 0.84 | 0.73 |
|
| 421 |
+
| nllb 3.3B | bg | ca | 29.87 | 57.75 | 57.26 | 0.85 | 0.85 | 0.73 |
|
| 422 |
+
|
| 423 |
+
|
| 424 |
</details>
|
| 425 |
|
| 426 |
#### Flores200-devtest
|
|
|
|
| 428 |
| | Bleu ↑ | Ter ↓ | ChrF ↑ | Comet ↑ | Comet-kiwi ↑ | Bleurt ↑ |
|
| 429 |
|:-----------------------|-------:|------:|-------:|--------:|-------------:|---------:|
|
| 430 |
| **CA-XX** | | | | | | |
|
| 431 |
+
| SalamandraTA-2B | **27.09** | **61.06** | **56.41** | 0.86 | 0.81 | 0.75 |
|
| 432 |
| nllb 3.3B | 26.7 | 61.74 | 55.85 | 0.86 | **0.82** | **0.76** |
|
| 433 |
| **XX-CA** | | | | | | |
|
| 434 |
+
| SalamandraTA-2B | **31** | **57.46** | **57.96** | 0.85 | 0.81 | 0.73 |
|
| 435 |
| nllb 3.3B | 30.31 | 58.26 | 57.12 | 0.85 | **0.82** | 0.73 |
|
| 436 |
+
|
| 437 |
+
<details>
|
| 438 |
+
<summary>Click to show full table CA-XX Flores-devtest</summary>
|
| 439 |
+
|
| 440 |
+
| | source | target | Bleu ↑ | Ter ↓ | ChrF ↑ | Comet ↑ | Comet-kiwi ↑ | Bleurt ↑ |
|
| 441 |
+
|:-----------------------|:---------|:---------|-------:|------:|-------:|--------:|-------------:|---------:|
|
| 442 |
+
| nllb 3.3B | ca | sv | 32.49 | 55.11 | 59.93 | 0.88 | 0.82 | 0.79 |
|
| 443 |
+
| SalamandraTA-2B | ca | sv | 30.53 | 56.24 | 58.05 | 0.87 | 0.8 | 0.77 |
|
| 444 |
+
| | | | | | | | | |
|
| 445 |
+
| SalamandraTA-2B | ca | sl | 25.16 | 64.25 | 53.88 | 0.87 | 0.82 | 0.8 |
|
| 446 |
+
| nllb 3.3B | ca | sl | 24.64 | 66.02 | 52.71 | 0.88 | 0.82 | 0.81 |
|
| 447 |
+
| | | | | | | | | |
|
| 448 |
+
| SalamandraTA-2B | ca | sk | 25.64 | 63.03 | 53.55 | 0.88 | 0.83 | 0.79 |
|
| 449 |
+
| nllb 3.3B | ca | sk | 25.44 | 63.29 | 53.37 | 0.89 | 0.84 | 0.79 |
|
| 450 |
+
| | | | | | | | | |
|
| 451 |
+
| SalamandraTA-2B | ca | ro | 33.21 | 54.27 | 59.53 | 0.89 | 0.84 | 0.8 |
|
| 452 |
+
| nllb 3.3B | ca | ro | 31.29 | 56.44 | 58.16 | 0.89 | 0.85 | 0.8 |
|
| 453 |
+
| | | | | | | | | |
|
| 454 |
+
| SalamandraTA-2B | ca | pt | 37.9 | 48.95 | 63.15 | 0.88 | 0.84 | 0.75 |
|
| 455 |
+
| nllb 3.3B | ca | pt | 37.31 | 49.31 | 62.7 | 0.88 | 0.85 | 0.75 |
|
| 456 |
+
| | | | | | | | | |
|
| 457 |
+
| SalamandraTA-2B | ca | pl | 18.62 | 71.88 | 48.44 | 0.88 | 0.83 | 0.77 |
|
| 458 |
+
| nllb 3.3B | ca | pl | 18.01 | 72.23 | 48.26 | 0.88 | 0.83 | 0.77 |
|
| 459 |
+
| | | | | | | | | |
|
| 460 |
+
| SalamandraTA-2B | ca | nl | 23.4 | 65.66 | 54.55 | 0.85 | 0.84 | 0.74 |
|
| 461 |
+
| nllb 3.3B | ca | nl | 22.99 | 66.68 | 53.95 | 0.85 | 0.84 | 0.75 |
|
| 462 |
+
| | | | | | | | | |
|
| 463 |
+
| nllb 3.3B | ca | mt | 24.78 | 59.97 | 59.58 | 0.68 | 0.62 | 0.36 |
|
| 464 |
+
| SalamandraTA-2B | ca | mt | 24.35 | 60.1 | 60.51 | 0.69 | 0.6 | 0.4 |
|
| 465 |
+
| | | | | | | | | |
|
| 466 |
+
| SalamandraTA-2B | ca | lv | 20.55 | 71.85 | 50.24 | 0.82 | 0.78 | 0.74 |
|
| 467 |
+
| nllb 3.3B | ca | lv | 20.16 | 70.37 | 50.3 | 0.85 | 0.78 | 0.78 |
|
| 468 |
+
| | | | | | | | | |
|
| 469 |
+
| SalamandraTA-2B | ca | lt | 20.37 | 70.15 | 51.61 | 0.88 | 0.79 | 0.82 |
|
| 470 |
+
| nllb 3.3B | ca | lt | 19.95 | 70.47 | 52.49 | 0.88 | 0.81 | 0.81 |
|
| 471 |
+
| | | | | | | | | |
|
| 472 |
+
| SalamandraTA-2B | ca | it | 27.18 | 60.37 | 56.65 | 0.88 | 0.85 | 0.77 |
|
| 473 |
+
| nllb 3.3B | ca | it | 26.83 | 60.96 | 56.33 | 0.88 | 0.85 | 0.77 |
|
| 474 |
+
| | | | | | | | | |
|
| 475 |
+
| SalamandraTA-2B | ca | hu | 21.76 | 66.96 | 53.45 | 0.86 | 0.81 | 0.85 |
|
| 476 |
+
| nllb 3.3B | ca | hu | 20.54 | 68.28 | 52.2 | 0.87 | 0.82 | 0.87 |
|
| 477 |
+
| | | | | | | | | |
|
| 478 |
+
| SalamandraTA-2B | ca | hr | 25.41 | 62.55 | 55.65 | 0.89 | 0.84 | 0.81 |
|
| 479 |
+
| nllb 3.3B | ca | hr | 24.01 | 64.39 | 53.95 | 0.89 | 0.84 | 0.82 |
|
| 480 |
+
| | | | | | | | | |
|
| 481 |
+
| nllb 3.3B | ca | gl | 32.33 | 52.64 | 59.3 | 0.87 | 0.85 | 0.71 |
|
| 482 |
+
| SalamandraTA-2B | ca | gl | 31.97 | 52.76 | 59.48 | 0.87 | 0.84 | 0.7 |
|
| 483 |
+
| | | | | | | | | |
|
| 484 |
+
| SalamandraTA-2B | ca | ga | 23.19 | 66.3 | 51.99 | 0.77 | 0.64 | 0.6 |
|
| 485 |
+
| nllb 3.3B | ca | ga | 22.38 | 67.76 | 50.92 | 0.77 | 0.66 | 0.6 |
|
| 486 |
+
| | | | | | | | | |
|
| 487 |
+
| nllb 3.3B | ca | fr | 40.82 | 47.72 | 64.82 | 0.86 | 0.85 | 0.74 |
|
| 488 |
+
| SalamandraTA-2B | ca | fr | 40.35 | 47.79 | 64.56 | 0.86 | 0.84 | 0.73 |
|
| 489 |
+
| | | | | | | | | |
|
| 490 |
+
| nllb 3.3B | ca | fi | 18.93 | 70.8 | 53.03 | 0.89 | 0.81 | 0.82 |
|
| 491 |
+
| SalamandraTA-2B | ca | fi | 18.92 | 70.69 | 52.85 | 0.88 | 0.8 | 0.8 |
|
| 492 |
+
| | | | | | | | | |
|
| 493 |
+
| SalamandraTA-2B | ca | eu | 18.33 | 72 | 56.65 | 0.86 | 0.81 | 0.79 |
|
| 494 |
+
| nllb 3.3B | ca | eu | 12.79 | 78.69 | 50.19 | 0.83 | 0.75 | 0.75 |
|
| 495 |
+
| | | | | | | | | |
|
| 496 |
+
| SalamandraTA-2B | ca | et | 21.45 | 67.08 | 55.01 | 0.88 | 0.8 | 0.79 |
|
| 497 |
+
| nllb 3.3B | ca | et | 19.84 | 70.08 | 53.48 | 0.88 | 0.8 | 0.79 |
|
| 498 |
+
| | | | | | | | | |
|
| 499 |
+
| nllb 3.3B | ca | es | 25.87 | 59.66 | 54.06 | 0.86 | 0.86 | 0.74 |
|
| 500 |
+
| SalamandraTA-2B | ca | es | 24.73 | 60.79 | 53.48 | 0.86 | 0.86 | 0.73 |
|
| 501 |
+
| | | | | | | | | |
|
| 502 |
+
| nllb 3.3B | ca | en | 48.41 | 38.1 | 71.29 | 0.89 | 0.86 | 0.8 |
|
| 503 |
+
| SalamandraTA-2B | ca | en | 45.19 | 41.18 | 69.46 | 0.88 | 0.85 | 0.78 |
|
| 504 |
+
| | | | | | | | | |
|
| 505 |
+
| SalamandraTA-2B | ca | el | 22.78 | 63.17 | 49.97 | 0.87 | 0.83 | 0.73 |
|
| 506 |
+
| nllb 3.3B | ca | el | 22.59 | 63.8 | 49.33 | 0.87 | 0.83 | 0.73 |
|
| 507 |
+
| | | | | | | | | |
|
| 508 |
+
| SalamandraTA-2B | ca | de | 31.31 | 57.16 | 59.42 | 0.85 | 0.83 | 0.75 |
|
| 509 |
+
| nllb 3.3B | ca | de | 31.25 | 57.87 | 59.05 | 0.85 | 0.83 | 0.75 |
|
| 510 |
+
| | | | | | | | | |
|
| 511 |
+
| SalamandraTA-2B | ca | da | 34.83 | 53.16 | 61.44 | 0.88 | 0.82 | 0.75 |
|
| 512 |
+
| nllb 3.3B | ca | da | 34.43 | 53.82 | 60.73 | 0.88 | 0.83 | 0.76 |
|
| 513 |
+
| | | | | | | | | |
|
| 514 |
+
| SalamandraTA-2B | ca | cs | 24.98 | 63.45 | 53.11 | 0.89 | 0.84 | 0.77 |
|
| 515 |
+
| nllb 3.3B | ca | cs | 24.73 | 63.94 | 52.66 | 0.89 | 0.85 | 0.78 |
|
| 516 |
+
| | | | | | | | | |
|
| 517 |
+
| SalamandraTA-2B | ca | bg | 32.25 | 55.76 | 59.85 | 0.89 | 0.85 | 0.84 |
|
| 518 |
+
| nllb 3.3B | ca | bg | 31.45 | 56.93 | 59.29 | 0.89 | 0.85 | 0.85 |
|
| 519 |
+
|
| 520 |
+
</details>
|
| 521 |
+
|
| 522 |
<details>
|
| 523 |
+
<summary>Click to show full table XX-CA Flores-devtest</summary>
|
| 524 |
+
|
| 525 |
+
| | source | target | Bleu ↑ | Ter ↓ | ChrF ↑ | Comet ↑ | Comet-kiwi ↑ | Bleurt ↑ |
|
| 526 |
+
|:-----------------------|:---------|:---------|-------:|------:|-------:|--------:|-------------:|---------:|
|
| 527 |
+
| SalamandraTA-2B | sv | ca | 34.4 | 52.6 | 59.96 | 0.86 | 0.82 | 0.73 |
|
| 528 |
+
| nllb 3.3B | sv | ca | 33.4 | 53.19 | 59.29 | 0.86 | 0.83 | 0.74 |
|
| 529 |
+
| | | | | | | | | |
|
| 530 |
+
| SalamandraTA-2B | sl | ca | 29.12 | 59.26 | 56.56 | 0.85 | 0.8 | 0.71 |
|
| 531 |
+
| nllb 3.3B | sl | ca | 28.23 | 60.61 | 55.34 | 0.85 | 0.82 | 0.72 |
|
| 532 |
+
| | | | | | | | | |
|
| 533 |
+
| SalamandraTA-2B | sk | ca | 30.71 | 57.99 | 57.81 | 0.85 | 0.8 | 0.72 |
|
| 534 |
+
| nllb 3.3B | sk | ca | 29.79 | 58.99 | 56.61 | 0.85 | 0.82 | 0.73 |
|
| 535 |
+
| | | | | | | | | |
|
| 536 |
+
| SalamandraTA-2B | ro | ca | 34.79 | 53.37 | 61.22 | 0.87 | 0.83 | 0.75 |
|
| 537 |
+
| nllb 3.3B | ro | ca | 33.53 | 54.36 | 60.18 | 0.87 | 0.84 | 0.75 |
|
| 538 |
+
| | | | | | | | | |
|
| 539 |
+
| SalamandraTA-2B | pt | ca | 36.72 | 50.64 | 62.08 | 0.87 | 0.84 | 0.76 |
|
| 540 |
+
| nllb 3.3B | pt | ca | 36.11 | 50.96 | 61.33 | 0.87 | 0.84 | 0.76 |
|
| 541 |
+
| | | | | | | | | |
|
| 542 |
+
| SalamandraTA-2B | pl | ca | 25.62 | 64.15 | 53.55 | 0.85 | 0.81 | 0.71 |
|
| 543 |
+
| nllb 3.3B | pl | ca | 25.14 | 64.43 | 53.09 | 0.85 | 0.83 | 0.71 |
|
| 544 |
+
| | | | | | | | | |
|
| 545 |
+
| SalamandraTA-2B | nl | ca | 26.17 | 63.88 | 54.01 | 0.84 | 0.83 | 0.7 |
|
| 546 |
+
| nllb 3.3B | nl | ca | 25.61 | 64.26 | 53.43 | 0.84 | 0.85 | 0.71 |
|
| 547 |
+
| | | | | | | | | |
|
| 548 |
+
| SalamandraTA-2B | mt | ca | 36.97 | 50.43 | 62.69 | 0.79 | 0.68 | 0.75 |
|
| 549 |
+
| nllb 3.3B | mt | ca | 36.03 | 51.51 | 61.46 | 0.79 | 0.69 | 0.74 |
|
| 550 |
+
| | | | | | | | | |
|
| 551 |
+
| SalamandraTA-2B | lv | ca | 27.81 | 61.96 | 56.12 | 0.84 | 0.77 | 0.7 |
|
| 552 |
+
| nllb 3.3B | lv | ca | 26.83 | 63.33 | 53.93 | 0.84 | 0.78 | 0.7 |
|
| 553 |
+
| | | | | | | | | |
|
| 554 |
+
| SalamandraTA-2B | lt | ca | 27.29 | 61.15 | 54.14 | 0.84 | 0.75 | 0.7 |
|
| 555 |
+
| nllb 3.3B | lt | ca | 26.13 | 62.2 | 53.17 | 0.84 | 0.77 | 0.7 |
|
| 556 |
+
| | | | | | | | | |
|
| 557 |
+
| SalamandraTA-2B | it | ca | 29.12 | 60.95 | 57.85 | 0.87 | 0.85 | 0.74 |
|
| 558 |
+
| nllb 3.3B | it | ca | 28.06 | 61.81 | 57.06 | 0.87 | 0.85 | 0.74 |
|
| 559 |
+
| | | | | | | | | |
|
| 560 |
+
| SalamandraTA-2B | hu | ca | 28.21 | 60.54 | 55.38 | 0.85 | 0.81 | 0.71 |
|
| 561 |
+
| nllb 3.3B | hu | ca | 27.58 | 60.77 | 54.76 | 0.85 | 0.83 | 0.72 |
|
| 562 |
+
| | | | | | | | | |
|
| 563 |
+
| SalamandraTA-2B | hr | ca | 30.13 | 57.59 | 57.25 | 0.86 | 0.81 | 0.72 |
|
| 564 |
+
| nllb 3.3B | hr | ca | 29.15 | 62.59 | 56.04 | 0.86 | 0.83 | 0.72 |
|
| 565 |
+
| | | | | | | | | |
|
| 566 |
+
| nllb 3.3B | gl | ca | 34.23 | 53.25 | 61.28 | 0.88 | 0.85 | 0.76 |
|
| 567 |
+
| SalamandraTA-2B | gl | ca | 32.09 | 54.77 | 60.42 | 0.87 | 0.84 | 0.75 |
|
| 568 |
+
| | | | | | | | | |
|
| 569 |
+
| SalamandraTA-2B | ga | ca | 28.11 | 62.93 | 55.28 | 0.8 | 0.68 | 0.67 |
|
| 570 |
+
| nllb 3.3B | ga | ca | 27.73 | 62.91 | 53.93 | 0.79 | 0.69 | 0.66 |
|
| 571 |
+
| | | | | | | | | |
|
| 572 |
+
| SalamandraTA-2B | fr | ca | 35.87 | 52.28 | 61.2 | 0.87 | 0.83 | 0.75 |
|
| 573 |
+
| nllb 3.3B | fr | ca | 34.42 | 53.05 | 60.31 | 0.87 | 0.84 | 0.76 |
|
| 574 |
+
| | | | | | | | | |
|
| 575 |
+
| SalamandraTA-2B | fi | ca | 27.35 | 61.33 | 54.95 | 0.86 | 0.8 | 0.7 |
|
| 576 |
+
| nllb 3.3B | fi | ca | 27.04 | 62.35 | 54.48 | 0.86 | 0.81 | 0.71 |
|
| 577 |
+
| | | | | | | | | |
|
| 578 |
+
| SalamandraTA-2B | eu | ca | 28.02 | 60.45 | 55.44 | 0.87 | 0.82 | 0.73 |
|
| 579 |
+
| nllb 3.3B | eu | ca | 26.68 | 62.62 | 54.22 | 0.86 | 0.82 | 0.71 |
|
| 580 |
+
| | | | | | | | | |
|
| 581 |
+
| SalamandraTA-2B | et | ca | 29.84 | 58.79 | 56.74 | 0.86 | 0.78 | 0.72 |
|
| 582 |
+
| nllb 3.3B | et | ca | 28.43 | 60.01 | 55.48 | 0.86 | 0.79 | 0.72 |
|
| 583 |
+
| | | | | | | | | |
|
| 584 |
+
| nllb 3.3B | es | ca | 25.64 | 64.21 | 55.18 | 0.87 | 0.85 | 0.73 |
|
| 585 |
+
| SalamandraTA-2B | es | ca | 23.47 | 66.71 | 54.05 | 0.86 | 0.84 | 0.72 |
|
| 586 |
+
| | | | | | | | | |
|
| 587 |
+
| SalamandraTA-2B | en | ca | 43.98 | 42.35 | 67.3 | 0.87 | 0.85 | 0.77 |
|
| 588 |
+
| nllb 3.3B | en | ca | 43.24 | 43.37 | 66.58 | 0.88 | 0.85 | 0.78 |
|
| 589 |
+
| | | | | | | | | |
|
| 590 |
+
| SalamandraTA-2B | el | ca | 28.91 | 59.86 | 55.26 | 0.85 | 0.83 | 0.71 |
|
| 591 |
+
| nllb 3.3B | el | ca | 28.46 | 60.28 | 55.13 | 0.85 | 0.84 | 0.72 |
|
| 592 |
+
| | | | | | | | | |
|
| 593 |
+
| SalamandraTA-2B | de | ca | 33.71 | 54.06 | 59.79 | 0.86 | 0.83 | 0.74 |
|
| 594 |
+
| nllb 3.3B | de | ca | 32.71 | 54.91 | 58.91 | 0.86 | 0.84 | 0.74 |
|
| 595 |
+
| | | | | | | | | |
|
| 596 |
+
| SalamandraTA-2B | da | ca | 35.14 | 52.51 | 60.81 | 0.86 | 0.82 | 0.74 |
|
| 597 |
+
| nllb 3.3B | da | ca | 34.03 | 53.41 | 59.46 | 0.86 | 0.83 | 0.75 |
|
| 598 |
+
| | | | | | | | | |
|
| 599 |
+
| SalamandraTA-2B | cs | ca | 31.12 | 56.71 | 58.22 | 0.86 | 0.81 | 0.73 |
|
| 600 |
+
| nllb 3.3B | cs | ca | 29.26 | 58.38 | 56.53 | 0.86 | 0.82 | 0.73 |
|
| 601 |
+
| | | | | | | | | |
|
| 602 |
+
| SalamandraTA-2B | bg | ca | 31.33 | 56.72 | 58.75 | 0.85 | 0.84 | 0.73 |
|
| 603 |
+
| nllb 3.3B | bg | ca | 30.5 | 57.03 | 57.92 | 0.85 | 0.85 | 0.73 |
|
| 604 |
|
| 605 |
</details>
|
| 606 |
|
| 607 |
|
| 608 |
|
| 609 |
+
|
| 610 |
## Ethical Considerations and Limitations
|
| 611 |
|
| 612 |
---
|