wwydmanski/all-mpnet-base-v2-legal-v0.1

SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-mpnet-base-v2
• Maximum Sequence Length: 384 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: PeftModelForFeatureExtraction 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'query: Under what conditions will the legend on Series G Units cease to be required?',
    'requirement that the Series G Units contain the legend set forth in clause (j)above shall cease and terminate upon the earlier of (i)when such shares are transferred pursuant to Rule 144 under the 1933 Act or (ii)when such securities are transferred in any other transaction if the transferor delivers to the Company a written opinion of counsel (which opinion and counsel shall be reasonably satisfactory to the Company) to the effect that such legend is no longer necessary in order to protect the Company against a violation by it of the 1933 Act upon any sale or other disposition of such securities without registration thereunder. Upon the consummation of an event described in (i)or (ii)above, the Company, upon surrender of certificates containing such legend, shall, at its own',
    'Legend on Certificate. Until the eligibility conditions of this Award have been satisfied and shares of Stock have been issued in accordance with the terms of this Agreement or by action of the Committee, the Units awarded hereunder are not transferable and shall not be sold, transferred, assigned, pledged, gifted, hypothecated or otherwise disposed of or encumbered by you. Transfers of shares of Stock by you are subject to the Company’s Stock Trading Policy and applicable securities laws. Shares of Stock issued to you in certificate form or to your book entry account upon satisfaction of the vesting and other conditions of this Award may be restricted from transfer or sale by the Company and evidenced by stop-transfer instructions upon your book entry account or restricted legend(s) affixed',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

json

• Dataset: json
• Size: 10,000 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 11 tokens mean: 22.14 tokens max: 43 tokens	min: 6 tokens mean: 189.63 tokens max: 384 tokens	min: 64 tokens mean: 195.06 tokens max: 384 tokens

• Samples:

  anchor	positive	negative
  query: What is the effective date of the Fifth Amendment to the Approach Resources Inc. 2007 Stock Incentive Plan?	Exhibit 10.1 FIFTH AMENDMENT TO THE APPROACH RESOURCES INC. 2007 STOCK INCENTIVE PLAN This Fifth Amendment (the “Fifth Amendment”) to the Approach Resources Inc. 2007 Stock Incentive Plan, as amended from time to time (the “Plan”), is made effective as of June 2, 2016 (the “Amendment Effective Date”), by Approach Resources Inc., a Delaware corporation (“Approach”), subject to approval by Approach’s stockholders. W I T N E S S E T H: WHEREAS, Approach established the Plan, originally effective as of June 28, 2007 and most recently amended effective March 2, 2016, under which Approach is authorized to grant equity-based incentive awards to certain employees and service providers of Approach and its subsidiaries; WHEREAS, Section 14.1 of the Plan provides that Approach’s board of directors (the “Board”) may	Exhibit 10.39 AMENDMENT TO THE BPZ RESOURCES, INC. 2007 LONG-TERM INCENTIVE COMPENSATION PLAN WHEREAS, BPZ Resources,Inc. (the “Company”) adopted and maintains the BPZ Energy,Inc. 2007 Long-Term Incentive Compensation Plan (the “Plan”), effective as of June4, 2007, to provide an opportunity for its eligible employees and certain independent contractors to earn long term incentive awards in consideration for their services; WHEREAS, the Company now desires to amend the Plan to reserve additional shares for issuance under the Plan. NOW THEREFORE, effective as of June 20, 2014, the Plan is hereby amended by replacing Section7(a)with the following new Section7(a)that shall read as follows: “(a) Maximum Shares. Subject to adjustment as provided in this Section 7, there is hereby reserved for issuance under the Plan up to 12,000,000 shares of Stock
  query: What is the date on which the Company accepted the subscription?	to acceptance by the Company, the undersigned has completed this Subscription Agreement to evidence his/her/its sub­scrip­tion for participation in the securities of the Company, this ____th day of _________ 2013. Subscriber Printed name If an entity, on behalf of: Subscriber’s position with entity: The Company has accepted this subscription this ____ day of _________ 2012. OverNear, Inc. By Its: Printed Name: Page11 of 19 Subscription Agreement OverNear, Inc. -------------------------------------------------------------------------------- Subscription Documents - Continued OVERNEAR, INC. (THE “COMPANY”) INVESTOR APPLICATION (QUALIFICATION QUESTIONNAIRE) (CONFIDENTIAL) ALL INFORMATION CONTAINED IN THIS APPLICATIONWILL BE TREATEDCONFIDENTIALLY. The undersigned understands, however, that the Company may present this application to such parties as the Company, in his discretion, deems appropriate when called upon to establish that the proposed offer and sale of the Securities are exempt	and each Subscriber is executing and delivering this agreement in reliance upon the exemption from securities registration afforded by Section 4(2) of the Securities Act and Rule 506 of Regulation D as promulgated by the SEC under the Securities Act; and WHEREAS the subscription for the Securities will be made in accordance with and subject to the terms and conditions of this Subscription Agreement and the Company's Confidential Private Placement Memorandum dated January 28, 2014 together with all amendments thereof and supplements and exhibits thereto and as such may be amended from time to time (the "Memorandum"); and WHEREAS, the Subscriber desires to purchase such number of shares of Common Stock (together with the associated Warrants) as set forth on the signature page hereof on the terms and
  query: What percentage of common shares must an entity own to be considered an Acquiring Person under the Rights Agreement?	the mutual agreements herein set forth, the parties agree as follows: Section1. Amendment to Section1.1. Section1.1 of the Rights Agreement is amended to read in its entirety as follows: “1.1 “Acquiring Person” shall mean any Person (as such term is hereinafter defined) who or which, together with all Affiliates and Associates (as such terms are hereinafter defined) of such Person, shall be the Beneficial Owner (as such term is hereinafter defined) of 15% or more of the Common Shares of the Company then outstanding, but shall not include: (i) the Company; (ii) any Subsidiary of the Company; (iii) any employee benefit plan of the Company or of any Subsidiary of the Company or any entity holding shares of capital stock of the Company for or pursuant to the	of more than 25% of the Common Shares outstanding immediately prior to the distribution, and in making this determination the Common Shares to be issued to such Person in the distribution shall be deemed to be held by such Person but shall not be included in the aggregate number of outstanding Common Shares immediately prior to the distribution ("Exempt Acquisitions"); the acquisition of Common Shares upon the exercise of Convertible Securities received by such Person pursuant to a Permitted Bid Acquisition, an Exempt Acquisition or a Pro Rata Acquisition (as defined below) ("Convertible Security Acquisitions"); or acquisitions as a result of a stock dividend, a stock split or other event pursuant to which such Person receives or acquires Common Shares or Convertible Securities on the same pro rata

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• gradient_accumulation_steps: 8
• learning_rate: 0.0001
• num_train_epochs: 1
• lr_scheduler_type: cosine_with_restarts
• warmup_ratio: 0.1
• bf16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 8
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 0.0001
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: cosine_with_restarts
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• eval_use_gather_object: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss
0.0128	1	0.6274
0.0256	2	0.7466
0.0384	3	0.6275
0.0512	4	0.7828
0.064	5	0.8595
0.0768	6	0.7202
0.0896	7	0.8533
0.1024	8	0.8205
0.1152	9	0.5946
0.128	10	0.6259
0.1408	11	0.6942
0.1536	12	0.5639
0.1664	13	0.5801
0.1792	14	0.6961
0.192	15	0.5709
0.2048	16	0.5827
0.2176	17	0.5587
0.2304	18	0.6785
0.2432	19	0.5421
0.256	20	0.5633
0.2688	21	0.4312
0.2816	22	0.4915
0.2944	23	0.5614
0.3072	24	0.3686
0.32	25	0.4252
0.3328	26	0.4877
0.3456	27	0.4616
0.3584	28	0.3765
0.3712	29	0.4412
0.384	30	0.4257
0.3968	31	0.5719
0.4096	32	0.4199
0.4224	33	0.4739
0.4352	34	0.4306
0.448	35	0.4413
0.4608	36	0.4897
0.4736	37	0.4035
0.4864	38	0.4182
0.4992	39	0.4927
0.512	40	0.3966
0.5248	41	0.3429
0.5376	42	0.4481
0.5504	43	0.5065
0.5632	44	0.3777
0.576	45	0.3732
0.5888	46	0.3587
0.6016	47	0.3732
0.6144	48	0.3401
0.6272	49	0.3553
0.64	50	0.5326
0.6528	51	0.3874
0.6656	52	0.3768
0.6784	53	0.3759
0.6912	54	0.4088
0.704	55	0.3846
0.7168	56	0.4591
0.7296	57	0.4135
0.7424	58	0.3038
0.7552	59	0.4156
0.768	60	0.3611
0.7808	61	0.3517
0.7936	62	0.4054
0.8064	63	0.3859
0.8192	64	0.463
0.832	65	0.4033
0.8448	66	0.4706
0.8576	67	0.4508
0.8704	68	0.3988
0.8832	69	0.2877
0.896	70	0.4187
0.9088	71	0.4109
0.9216	72	0.317
0.9344	73	0.4432
0.9472	74	0.4396
0.96	75	0.4123
0.9728	76	0.3119
0.9856	77	0.3858
0.9984	78	0.4166

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 3.3.1
• Transformers: 4.44.2
• PyTorch: 2.5.1
• Accelerate: 1.2.1
• Datasets: 2.19.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}