metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:6300
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: nomic-ai/nomic-embed-text-v1.5
widget:
  - source_sentence: >-
      Where in the Annual Report can one find a description of certain legal
      matters and their impact on the company?
    sentences:
      - >-
        Apollo coordinates the delivery of new features, security updates, and
        platform configurations, ensuring the continuous operation of systems in
        any environment. It was introduced commercially in 2021.
      - >-
        In the Annual Report on Form 10-K, 'Item 1A. Risk Factors' provides a
        further description of certain legal matters and their impact on the
        company.
      - During fiscal 2022, we opened four new stores in Mexico.
  - source_sentence: How does the company assess uncertain tax positions?
    sentences:
      - >-
        We recognize tax benefits from uncertain tax positions only if we
        believe that it is more likely than not that the tax position will be
        sustained on examination by the taxing authorities based on the
        technical merits of the position.
      - >-
        CMS uses a risk-adjustment model which adjusts premiums paid to Medicare
        Advantage, or MA, plans according to health status of covered members.
        The risk-adjustment model, which CMS implemented pursuant to the
        Balanced Budget Act of 1997 (BBA) and the Benefits Improvement and
        Protection Act of 2000 (BIPA), generally pays more where a plan's
        membership has higher expected costs. Under this model, rates paid to MA
        plans are based on actuarially determined bids, which include a process
        whereby our prospective payments are based on our estimated cost of
        providing standard Medicare-covered benefits to an enrollee with a
        'national average risk profile.' That baseline payment amount is
        adjusted to account for certain demographic characteristics and health
        status of our enrolled members.
      - >-
        Walmart Inc. reported total revenues of $611,289 million for the fiscal
        year ended January 31, 2023.
  - source_sentence: >-
      When does the 364-day facility entered into in August 2023 expire, and
      what is its total amount?
    sentences:
      - In 2023, the total revenue generated by Emgality amounted to 678.3.
      - >-
        In August 2023, we entered into a new 364-day facility. The 364-day
        facility of $3.15 billion expires in August 2024.
      - >-
        Diluted EPS increased $0.09, or 2%, to $5.90 as the decrease in net
        earnings was more than fully offset by a reduction in shares
        outstanding.
  - source_sentence: >-
      What does the company believe adds significant value to its business
      regarding intellectual property?
    sentences:
      - >-
        We believe that, to varying degrees, our trademarks, trade names,
        copyrights, proprietary processes, trade secrets, trade dress, domain
        names and similar intellectual property add significant value to our
        business
      - >-
        Railroad operating revenues declined 6.9% in 2023 compared to 2022,
        reflecting an overall volume decrease of 5.7% and a decrease in average
        revenue per car/unit of 0.6%, primarily attributable to lower fuel
        surcharge revenue, partially offset by favorable price and mix.
      - >-
        Cash provided by operating activities increased from $26.413 billion in
        2022 to $28.501 billion in 2023, an increase of approximately $2.088
        billion.
  - source_sentence: >-
      How are government incentives treated in accounting according to the given
      information?
    sentences:
      - >-
        The components of 'Other income (expense), net' for the year ended
        December 30, 2023, were $197 million; for December 31, 2022, they were
        $8 million; and for December 25, 2021, they were $55 million.
      - >-
        We are entitled to certain advanced manufacturing production credits
        under the IRA, and government incentives are not accounted for or
        classified as an income tax credit. We account for government incentives
        as a reduction of expense, a reduction of the cost of the capital
        investment or other income based on the substance of the incentive
        received. Benefits are generally recorded when there is reasonable
        assurance of receipt or, as it relates with advanced manufacturing
        production credits, upon the generation of the credit.
      - >-
        Basic net income per share is computed by dividing net income
        attributable to common stock by the weighted-average number of shares of
        common stock outstanding during the period.
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: Nomic Embed Financial Matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.7185714285714285
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.87
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9014285714285715
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9357142857142857
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7185714285714285
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.29
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18028571428571427
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09357142857142857
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7185714285714285
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.87
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9014285714285715
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9357142857142857
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8337966812161252
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8004784580498868
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8030662019934727
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.7157142857142857
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8685714285714285
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9028571428571428
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9342857142857143
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7157142857142857
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2895238095238095
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18057142857142855
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09342857142857142
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7157142857142857
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8685714285714285
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9028571428571428
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9342857142857143
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8320816465681472
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7986201814058957
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8013251784905495
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.7028571428571428
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.86
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8914285714285715
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9271428571428572
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7028571428571428
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2866666666666667
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17828571428571427
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09271428571428571
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7028571428571428
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.86
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8914285714285715
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9271428571428572
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8208030315973883
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7862023809523814
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7893111186082761
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.7
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8428571428571429
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8771428571428571
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9271428571428572
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.28095238095238095
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1754285714285714
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09271428571428571
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8428571428571429
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8771428571428571
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9271428571428572
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8174548081454337
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7820821995464855
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7852661387487447
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.69
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.83
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8671428571428571
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9128571428571428
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.69
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27666666666666667
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1734285714285714
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09128571428571428
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.69
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.83
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8671428571428571
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9128571428571428
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.804303333645382
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.769315192743764
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7729055647510643
            name: Cosine Map@100
datasets:
  - philschmid/finanical-rag-embedding-dataset

Nomic Embed Financial Matryoshka

This is a sentence-transformers model finetuned from nomic-ai/nomic-embed-text-v1.5 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: nomic-ai/nomic-embed-text-v1.5
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- json
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("shail-2512/nomic-embed-financial-matryoshka")
# Run inference
sentences = [
    'How are government incentives treated in accounting according to the given information?',
    'We are entitled to certain advanced manufacturing production credits under the IRA, and government incentives are not accounted for or classified as an income tax credit. We account for government incentives as a reduction of expense, a reduction of the cost of the capital investment or other income based on the substance of the incentive received. Benefits are generally recorded when there is reasonable assurance of receipt or, as it relates with advanced manufacturing production credits, upon the generation of the credit.',
    'Basic net income per share is computed by dividing net income attributable to common stock by the weighted-average number of shares of common stock outstanding during the period.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Datasets: dim_768, dim_512, dim_256, dim_128 and dim_64
Evaluated with InformationRetrievalEvaluator

Metric	dim_768	dim_512	dim_256	dim_128	dim_64
cosine_accuracy@1	0.7186	0.7157	0.7029	0.7	0.69
cosine_accuracy@3	0.87	0.8686	0.86	0.8429	0.83
cosine_accuracy@5	0.9014	0.9029	0.8914	0.8771	0.8671
cosine_accuracy@10	0.9357	0.9343	0.9271	0.9271	0.9129
cosine_precision@1	0.7186	0.7157	0.7029	0.7	0.69
cosine_precision@3	0.29	0.2895	0.2867	0.281	0.2767
cosine_precision@5	0.1803	0.1806	0.1783	0.1754	0.1734
cosine_precision@10	0.0936	0.0934	0.0927	0.0927	0.0913
cosine_recall@1	0.7186	0.7157	0.7029	0.7	0.69
cosine_recall@3	0.87	0.8686	0.86	0.8429	0.83
cosine_recall@5	0.9014	0.9029	0.8914	0.8771	0.8671
cosine_recall@10	0.9357	0.9343	0.9271	0.9271	0.9129
cosine_ndcg@10	0.8338	0.8321	0.8208	0.8175	0.8043
cosine_mrr@10	0.8005	0.7986	0.7862	0.7821	0.7693
cosine_map@100	0.8031	0.8013	0.7893	0.7853	0.7729

Training Details

Training Dataset

json

Dataset: json
Size: 6,300 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 2 tokens
mean: 20.65 tokens
max: 45 tokens

min: 2 tokens
mean: 46.29 tokens
max: 326 tokens

	anchor	positive
type	string	string
details	min: 2 tokens mean: 20.65 tokens max: 45 tokens	min: 2 tokens mean: 46.29 tokens max: 326 tokens

Samples:

anchor	positive
`Where is the Investor Relations office of Intuit Inc. located?`	`Copies of this Annual Report on Form 10-K may also be obtained without charge by contacting Investor Relations, Intuit Inc., P.O. Box 7850, Mountain View, California 94039-7850, calling 650-944-6000, or emailing [email protected].`
`Where is the Financial Statement Schedule located in the Form 10-K?`	`The Financial Statement Schedule is found on page S-1 of the Form 10-K.`
`What factors are considered when evaluating the realization of deferred tax assets?`	`Many factors are considered when assessing whether it is more likely than not that the deferred tax assets will be realized, including recent cumulative earnings, expectations of future taxable income, carryforward periods and other relevant quantitative and qualitative factors.`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Evaluation Dataset

json

Dataset: json
Size: 700 evaluation samples
Columns: anchor and positive
Approximate statistics based on the first 700 samples:
anchor positive
type string string
details
min: 2 tokens
mean: 20.71 tokens
max: 45 tokens

min: 9 tokens
mean: 46.74 tokens
max: 248 tokens

	anchor	positive
type	string	string
details	min: 2 tokens mean: 20.71 tokens max: 45 tokens	min: 9 tokens mean: 46.74 tokens max: 248 tokens

Samples:

anchor	positive
`What fiscal changes did Garmin make in January 2023?`	`The Company announced an organization realignment in January 2023, which combined the consumer auto operating segment with the outdoor operating segment.`
`Where are the details about 'Legal Matters' and 'Government Investigations, Audits and Reviews' located in the financial statements?`	`The information required by this Item 3 is incorporated herein by reference to the information set forth under the captions 'Legal Matters' and 'Government Investigations, Audits and Reviews' in Note 12 of the Notes to the Consolidated Financial Statements included in Part II, Item 8, 'Financial Statements and Supplementary Data'.`
`Are the pages of IBM's Management’s Discussion and Analysis section in the 2023 Annual Report included in the report itself?`	`In IBM’s 2023 Annual Report, the pages containing Management’s Discussion and Analysis of Financial Condition and Results of Operations (pages 6 through 40) are incorporated by reference.`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
gradient_accumulation_steps: 8
learning_rate: 2e-05
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 8
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	dim_768_cosine_ndcg@10	dim_512_cosine_ndcg@10	dim_256_cosine_ndcg@10	dim_128_cosine_ndcg@10	dim_64_cosine_ndcg@10
0.1015	10	0.2626	-	-	-	-	-	-
0.2030	20	0.1764	-	-	-	-	-	-
0.1015	10	0.0311	-	-	-	-	-	-
0.2030	20	0.0259	-	-	-	-	-	-
0.1015	10	0.0056	-	-	-	-	-	-
0.2030	20	0.0064	-	-	-	-	-	-
0.1015	10	0.0016	-	-	-	-	-	-
0.2030	20	0.0015	-	-	-	-	-	-
0.1015	10	0.0006	-	-	-	-	-	-
0.2030	20	0.0006	-	-	-	-	-	-
0.3046	30	0.1324	-	-	-	-	-	-
0.4061	40	0.113	-	-	-	-	-	-
0.5076	50	0.128	-	-	-	-	-	-
0.6091	60	0.1134	-	-	-	-	-	-
0.7107	70	0.056	-	-	-	-	-	-
0.8122	80	0.1086	-	-	-	-	-	-
0.9137	90	0.1008	-	-	-	-	-	-
1.0	99	-	0.0771	0.8286	0.8306	0.8266	0.8197	0.7955
1.0102	100	0.0491	-	-	-	-	-	-
1.1117	110	0.0029	-	-	-	-	-	-
1.2132	120	0.0009	-	-	-	-	-	-
1.3147	130	0.0326	-	-	-	-	-	-
1.4162	140	0.0077	-	-	-	-	-	-
1.5178	150	0.0109	-	-	-	-	-	-
1.6193	160	0.0047	-	-	-	-	-	-
1.7208	170	0.004	-	-	-	-	-	-
1.8223	180	0.0122	-	-	-	-	-	-
1.9239	190	0.0043	-	-	-	-	-	-
2.0	198	-	0.0758	0.8296	0.8330	0.8222	0.8169	0.7998
2.0203	200	0.0032	-	-	-	-	-	-
2.1218	210	0.0002	-	-	-	-	-	-
2.2234	220	0.0002	-	-	-	-	-	-
2.3249	230	0.0097	-	-	-	-	-	-
2.4264	240	0.0012	-	-	-	-	-	-
2.5279	250	0.0012	-	-	-	-	-	-
2.6294	260	0.0009	-	-	-	-	-	-
2.7310	270	0.0007	-	-	-	-	-	-
2.8325	280	0.0019	-	-	-	-	-	-
2.9340	290	0.0009	-	-	-	-	-	-
2.9746	294	-	0.0744	0.8338	0.8321	0.8208	0.8175	0.8043

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.12
Sentence Transformers: 3.3.1
Transformers: 4.47.0
PyTorch: 2.5.1+cu121
Accelerate: 1.1.1
Datasets: 3.1.0
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}