
# SetFit with sentence-transformers/all-mpnet-base-v2

This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A [SetFitHead](huggingface.co/docs/setfit/reference/main#setfit.SetFitHead) instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

## Model Details

### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
- **Classification head:** a [SetFitHead](huggingface.co/docs/setfit/reference/main#setfit.SetFitHead) instance
- **Maximum Sequence Length:** 384 tokens
- **Number of Classes:** 18 classes
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)

## Uses

### Direct Use for Inference

First install the SetFit library:

```bash
pip install setfit
```

Then you can load this model and run inference.

```python
from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("leavoigt/vulnerability_multilabel_test")
# Run inference
preds = model("Workers in the formal sector. Formal sector workers also face economic risks. A number of them experience income instability due to contractualization, retrenchment, and firm closures. In 2014, contractual workers accounted for 22 percent of the total 4.5 million workers employed in establishments with 20 or more employees.")
```

<!--
### Downstream Use

*List how someone could finetune this model on their own dataset.*
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Set Metrics
| Training set | Min | Median  | Max |
|:-------------|:----|:--------|:----|
| Word count   | 21  | 72.7135 | 238 |

### Training Hyperparameters
- batch_size: (16, 2)
- num_epochs: (1, 0)
- max_steps: -1
- sampling_strategy: undersampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False

### Training Results
| Epoch  | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0006 | 1    | 0.1826        | -               |
| 0.6313 | 1000 | 0.0405        | 0.1171          |

### Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.3.1
- Transformers: 4.35.2
- PyTorch: 2.1.0+cu121
- Datasets: 2.3.0
- Tokenizers: 0.15.2

## Citation

### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->
{"library_name": "setfit", "tags": ["setfit", "sentence-transformers", "text-classification", "generated_from_setfit_trainer"], "metrics": ["accuracy"], "widget": [{"text": "Implementing the reform required strong support from all ministries involved. A major effort was required to present the conceptual change to car importers, politicians and the public. A great deal was also invested in public relations to describe the benefits of the tax, which by many was perceived as yet another attempt to increase tax revenues. A number of the most popular car models\u2019 prices were about to increase \u2013 mostly large family, luxury and sport cars \u2013 but for many models, the retail price was actually reduced."}, {"text": "Workers in the formal sector. Formal sector workers also face economic risks. A number of them experience income instability due to contractualization, retrenchment, and firm closures. In 2014, contractual workers accounted for 22 percent of the total 4.5 million workers employed in establishments with 20 or more employees."}, {"text": "Building additional dams and power stations to further develop energy generation potential from the same river flow as well as develop new dam sites on parallel rivers in order to maintain the baseline hydropower electricity generation capacity to levels attainable under a \u2018no-climate change\u2019 scenario. Developing and implementing climate change compatible building/construction codes for buildings, roads, airports, airfields, dry ports, railways, bridges, dams and irrigation canals that are safe for human life and minimize economic damage that is likely to result from increasing extremes in flooding."}, {"text": "Another factor that increases farmer vulnerability is the remoteness of farm villages and lack of adequate road infrastructure. Across the three regions, roads are in a poor state and unevenly distributed, with many villages lacking roads that connect them to other villages. Even the main roads are often accessible only during the dry season. The livelihood implications of this isolation are significant, as farmers have difficulties getting their products to markets as well as obtaining agricultural inputs; in addition, farmers generally have to pay higher prices for agricultural inputs in remote areas, reducing their profit margins"}, {"text": "This project aims to construct a desalination plant in the capital city in order to respond directly to drinking water supply needs. This new plant, which will have a capacity of 22,500 m3\u00a0daily, easily expandable to 45,000 m3, will be fuelled by renewable energy, which is expected to be provided by a wind farm planned for the second phase of the project. Funding:\u00a0European Union. Rural Community Development and Water Mobilization Project (PRODERMO)."}], "pipeline_tag": "text-classification", "inference": false, "base_model": "sentence-transformers/all-mpnet-base-v2"}
setfit pytorch safetensors mpnet sentence-transformers text-classification generated_from_setfit_trainer arxiv:2209.11055 base_model:sentence-transformers/all-mpnet-base-v2 region:us