
# SetFit with sentence-transformers/paraphrase-multilingual-mpnet-base-v2

This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2) as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

## Model Details

### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/paraphrase-multilingual-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2)
- **Classification head:** a OneVsRestClassifier instance
- **Maximum Sequence Length:** 128 tokens
<!-- - **Number of Classes:** Unknown -->
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)

## Evaluation

### Metrics
| Label   | Accuracy |
|:--------|:---------|
| **all** | 0.6423   |

## Uses

### Direct Use for Inference

First install the SetFit library:

```bash
pip install setfit
```

Then you can load this model and run inference.

```python
from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("furrypython/setfit-multilabel-example")
# Run inference
preds = model("診察料 薬価料")
```

<!--
### Downstream Use

*List how someone could finetune this model on their own dataset.*
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Set Metrics
| Training set | Min | Median | Max |
|:-------------|:----|:-------|:----|
| Word count   | 1   | 9.7316 | 243 |

### Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 20
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False

### Training Results
| Epoch  | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0001 | 1    | 0.2305        | -               |
| 0.0056 | 50   | 0.2494        | -               |
| 0.0113 | 100  | 0.2406        | -               |
| 0.0169 | 150  | 0.1675        | -               |
| 0.0225 | 200  | 0.23          | -               |
| 0.0281 | 250  | 0.0998        | -               |
| 0.0338 | 300  | 0.1908        | -               |
| 0.0394 | 350  | 0.1515        | -               |
| 0.0450 | 400  | 0.1056        | -               |
| 0.0506 | 450  | 0.1047        | -               |
| 0.0563 | 500  | 0.109         | -               |
| 0.0619 | 550  | 0.1025        | -               |
| 0.0675 | 600  | 0.1929        | -               |
| 0.0731 | 650  | 0.1586        | -               |
| 0.0788 | 700  | 0.24          | -               |
| 0.0844 | 750  | 0.1056        | -               |
| 0.0900 | 800  | 0.0963        | -               |
| 0.0956 | 850  | 0.1459        | -               |
| 0.1013 | 900  | 0.0516        | -               |
| 0.1069 | 950  | 0.0567        | -               |
| 0.1125 | 1000 | 0.1319        | -               |
| 0.1181 | 1050 | 0.0524        | -               |
| 0.1238 | 1100 | 0.0687        | -               |
| 0.1294 | 1150 | 0.0185        | -               |
| 0.1350 | 1200 | 0.2164        | -               |
| 0.1406 | 1250 | 0.0646        | -               |
| 0.1463 | 1300 | 0.1001        | -               |
| 0.1519 | 1350 | 0.2916        | -               |
| 0.1575 | 1400 | 0.1164        | -               |
| 0.1631 | 1450 | 0.0711        | -               |
| 0.1688 | 1500 | 0.122         | -               |
| 0.1744 | 1550 | 0.0226        | -               |
| 0.1800 | 1600 | 0.1196        | -               |
| 0.1856 | 1650 | 0.0933        | -               |
| 0.1913 | 1700 | 0.0953        | -               |
| 0.1969 | 1750 | 0.0086        | -               |
| 0.2025 | 1800 | 0.0846        | -               |
| 0.2081 | 1850 | 0.1348        | -               |
| 0.2138 | 1900 | 0.0166        | -               |
| 0.2194 | 1950 | 0.1061        | -               |
| 0.2250 | 2000 | 0.1251        | -               |
| 0.2306 | 2050 | 0.1395        | -               |
| 0.2363 | 2100 | 0.1243        | -               |
| 0.2419 | 2150 | 0.0168        | -               |
| 0.2475 | 2200 | 0.0708        | -               |
| 0.2532 | 2250 | 0.0575        | -               |
| 0.2588 | 2300 | 0.0093        | -               |
| 0.2644 | 2350 | 0.1129        | -               |
| 0.2700 | 2400 | 0.0465        | -               |
| 0.2757 | 2450 | 0.124         | -               |
| 0.2813 | 2500 | 0.0995        | -               |
| 0.2869 | 2550 | 0.0431        | -               |
| 0.2925 | 2600 | 0.0118        | -               |
| 0.2982 | 2650 | 0.0769        | -               |
| 0.3038 | 2700 | 0.024         | -               |
| 0.3094 | 2750 | 0.0756        | -               |
| 0.3150 | 2800 | 0.0461        | -               |
| 0.3207 | 2850 | 0.0792        | -               |
| 0.3263 | 2900 | 0.1671        | -               |
| 0.3319 | 2950 | 0.0186        | -               |
| 0.3375 | 3000 | 0.0562        | -               |
| 0.3432 | 3050 | 0.0414        | -               |
| 0.3488 | 3100 | 0.082         | -               |
| 0.3544 | 3150 | 0.0611        | -               |
| 0.3600 | 3200 | 0.0736        | -               |
| 0.3657 | 3250 | 0.0817        | -               |
| 0.3713 | 3300 | 0.0756        | -               |
| 0.3769 | 3350 | 0.0875        | -               |
| 0.3825 | 3400 | 0.0467        | -               |
| 0.3882 | 3450 | 0.0621        | -               |
| 0.3938 | 3500 | 0.0672        | -               |
| 0.3994 | 3550 | 0.0867        | -               |
| 0.4050 | 3600 | 0.0745        | -               |
| 0.4107 | 3650 | 0.0828        | -               |
| 0.4163 | 3700 | 0.1929        | -               |
| 0.4219 | 3750 | 0.0838        | -               |
| 0.4275 | 3800 | 0.0158        | -               |
| 0.4332 | 3850 | 0.0189        | -               |
| 0.4388 | 3900 | 0.1146        | -               |
| 0.4444 | 3950 | 0.0735        | -               |
| 0.4500 | 4000 | 0.0479        | -               |
| 0.4557 | 4050 | 0.087         | -               |
| 0.4613 | 4100 | 0.0437        | -               |
| 0.4669 | 4150 | 0.1523        | -               |
| 0.4725 | 4200 | 0.005         | -               |
| 0.4782 | 4250 | 0.0218        | -               |
| 0.4838 | 4300 | 0.0655        | -               |
| 0.4894 | 4350 | 0.0868        | -               |
| 0.4950 | 4400 | 0.0396        | -               |
| 0.5007 | 4450 | 0.073         | -               |
| 0.5063 | 4500 | 0.0108        | -               |
| 0.5119 | 4550 | 0.0334        | -               |
| 0.5176 | 4600 | 0.0234        | -               |
| 0.5232 | 4650 | 0.032         | -               |
| 0.5288 | 4700 | 0.0032        | -               |
| 0.5344 | 4750 | 0.0064        | -               |
| 0.5401 | 4800 | 0.1145        | -               |
| 0.5457 | 4850 | 0.0633        | -               |
| 0.5513 | 4900 | 0.0557        | -               |
| 0.5569 | 4950 | 0.038         | -               |
| 0.5626 | 5000 | 0.033         | -               |
| 0.5682 | 5050 | 0.0093        | -               |
| 0.5738 | 5100 | 0.0135        | -               |
| 0.5794 | 5150 | 0.0244        | -               |
| 0.5851 | 5200 | 0.0597        | -               |
| 0.5907 | 5250 | 0.0673        | -               |
| 0.5963 | 5300 | 0.0105        | -               |
| 0.6019 | 5350 | 0.0932        | -               |
| 0.6076 | 5400 | 0.0067        | -               |
| 0.6132 | 5450 | 0.0528        | -               |
| 0.6188 | 5500 | 0.0523        | -               |
| 0.6244 | 5550 | 0.1361        | -               |
| 0.6301 | 5600 | 0.0276        | -               |
| 0.6357 | 5650 | 0.0012        | -               |
| 0.6413 | 5700 | 0.0517        | -               |
| 0.6469 | 5750 | 0.0017        | -               |
| 0.6526 | 5800 | 0.0952        | -               |
| 0.6582 | 5850 | 0.0239        | -               |
| 0.6638 | 5900 | 0.109         | -               |
| 0.6694 | 5950 | 0.0032        | -               |
| 0.6751 | 6000 | 0.0264        | -               |
| 0.6807 | 6050 | 0.0143        | -               |
| 0.6863 | 6100 | 0.0016        | -               |
| 0.6919 | 6150 | 0.0327        | -               |
| 0.6976 | 6200 | 0.0114        | -               |
| 0.7032 | 6250 | 0.0134        | -               |
| 0.7088 | 6300 | 0.0956        | -               |
| 0.7144 | 6350 | 0.0009        | -               |
| 0.7201 | 6400 | 0.0759        | -               |
| 0.7257 | 6450 | 0.0051        | -               |
| 0.7313 | 6500 | 0.0432        | -               |
| 0.7369 | 6550 | 0.1171        | -               |
| 0.7426 | 6600 | 0.0636        | -               |
| 0.7482 | 6650 | 0.0112        | -               |
| 0.7538 | 6700 | 0.0034        | -               |
| 0.7595 | 6750 | 0.0045        | -               |
| 0.7651 | 6800 | 0.0743        | -               |
| 0.7707 | 6850 | 0.0051        | -               |
| 0.7763 | 6900 | 0.1074        | -               |
| 0.7820 | 6950 | 0.1028        | -               |
| 0.7876 | 7000 | 0.002         | -               |
| 0.7932 | 7050 | 0.028         | -               |
| 0.7988 | 7100 | 0.0891        | -               |
| 0.8045 | 7150 | 0.0205        | -               |
| 0.8101 | 7200 | 0.0089        | -               |
| 0.8157 | 7250 | 0.0032        | -               |
| 0.8213 | 7300 | 0.0291        | -               |
| 0.8270 | 7350 | 0.0283        | -               |
| 0.8326 | 7400 | 0.0134        | -               |
| 0.8382 | 7450 | 0.0635        | -               |
| 0.8438 | 7500 | 0.0933        | -               |
| 0.8495 | 7550 | 0.0031        | -               |
| 0.8551 | 7600 | 0.0157        | -               |
| 0.8607 | 7650 | 0.0651        | -               |
| 0.8663 | 7700 | 0.0492        | -               |
| 0.8720 | 7750 | 0.0202        | -               |
| 0.8776 | 7800 | 0.0073        | -               |
| 0.8832 | 7850 | 0.0572        | -               |
| 0.8888 | 7900 | 0.0304        | -               |
| 0.8945 | 7950 | 0.0265        | -               |
| 0.9001 | 8000 | 0.0793        | -               |
| 0.9057 | 8050 | 0.012         | -               |
| 0.9113 | 8100 | 0.0059        | -               |
| 0.9170 | 8150 | 0.0179        | -               |
| 0.9226 | 8200 | 0.014         | -               |
| 0.9282 | 8250 | 0.0024        | -               |
| 0.9338 | 8300 | 0.0046        | -               |
| 0.9395 | 8350 | 0.003         | -               |
| 0.9451 | 8400 | 0.0134        | -               |
| 0.9507 | 8450 | 0.0614        | -               |
| 0.9563 | 8500 | 0.0397        | -               |
| 0.9620 | 8550 | 0.0106        | -               |
| 0.9676 | 8600 | 0.0067        | -               |
| 0.9732 | 8650 | 0.023         | -               |
| 0.9788 | 8700 | 0.0234        | -               |
| 0.9845 | 8750 | 0.0142        | -               |
| 0.9901 | 8800 | 0.0041        | -               |
| 0.9957 | 8850 | 0.0233        | -               |

### Framework Versions
- Python: 3.10.12
- SetFit: 1.0.1
- Sentence Transformers: 2.2.2
- Transformers: 4.35.2
- PyTorch: 2.1.0+cu121
- Datasets: 2.15.0
- Tokenizers: 0.15.0

## Citation

### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->
{"library_name": "setfit", "tags": ["setfit", "sentence-transformers", "text-classification", "generated_from_setfit_trainer"], "metrics": ["accuracy"], "widget": [{"text": "\u5206\u96e2\u4e0d\u5b89 \u8a3a\u5bdf \u884c\u52d5\u8a3a\u7642 (10\u5206) \u304a\u3057\u308a\u307e\u308f\u308a\u6d17\u6d44\u6db2350ml \u5e0c\u91c8\u30dc\u30c8\u30eb\u4ed8\u304d"}, {"text": "\u518d\u8a3a\u6599 \u51e6\u65b9\u6599 \u5185\u670d\u85ac \u5c3f\u691c\u67fb"}, {"text": "\u8a3a\u5bdf\u6599 \u85ac\u4fa1\u6599"}, {"text": "\u521d\u8a3a\u6599 X\u7dda\u691c\u67fb \u80f8\u90e8 \u7d30\u80de\u8a3a\u691c\u67fb \u3008\u9662\u5185\u3009 \u819d\u90e8 \u88ab\u6bdb\u691c\u67fb(\u6bdb\u6839\u691c\u67fb \u5185: \u30a2\u30dd\u30ad\u30eb\u93203. 6mg"}, {"text": "\u81fc\u6b6f\u5185\u5074\u304b\u3089\u51fa\u8840 \u8a3a\u5bdf \u6ce8\u5c04 \u691c\u67fb \u51e6\u7f6e \u85ac \u5165\u9662\u00b7\u9810\u304b\u308a \u30d5\u30fc\u30c9 \u624b\u8853 \u305d\u306e\u4ed61 \u305d\u306e\u4ed62"}], "pipeline_tag": "text-classification", "inference": false, "base_model": "sentence-transformers/paraphrase-multilingual-mpnet-base-v2", "model-index": [{"name": "SetFit with sentence-transformers/paraphrase-multilingual-mpnet-base-v2", "results": [{"task": {"type": "text-classification", "name": "Text Classification"}, "dataset": {"name": "Unknown", "type": "unknown", "split": "test"}, "metrics": [{"type": "accuracy", "value": 0.6422947131608548, "name": "Accuracy"}]}]}]}
setfit safetensors xlm-roberta sentence-transformers text-classification generated_from_setfit_trainer arxiv:2209.11055 base_model:sentence-transformers/paraphrase-multilingual-mpnet-base-v2 model-index region:us