
# SetFit

This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

## Model Details

### Model Description
- **Model Type:** SetFit
<!-- - **Sentence Transformer:** [Unknown](https://huggingface.co/unknown) -->
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 512 tokens
- **Number of Classes:** 6 classes
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)

### Model Labels
| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
|:------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| loc3  | <ul><li>'divine'</li><li>'divine'</li><li>'reprogram'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| loc5+ | <ul><li>'path of freedom'</li><li>'able to go to sort of what would be considered by the world a potentially less functional spot or not'</li><li>'so like with location five we generally have you know a substantial persistent change in visual perception um and so that is something that can be a lot of different things um so for instance it could be um things never really being dark but just sort of being dim or it can be sort of everything or even just some things having sort of a self luminous kind of quality however one of the ways that we think about this is most of us associate light reflects off of stuff and as the sun goes down or as i turn out the lights in a room or whatever the objects get dark right and so that just seems baked into the belief systems of most people that they develop probably causally through observation as they grew up and so okay you turn on the light things get bright its because the light reflects off of the objects or whatever sun goes down things you know whatever right there seems to be a change that occurs with location five that gets beyond that where that causality is no longer sort of assumed as part of the meaning making if you will of the visual system and so what often happens is objects are not really viewed as having light reflected off them so much as just sort of being luminous in their own way almost being like their own generators of light and that has you know a wide range of effects for starters people dont necessarily realize how not dark darkness is darkness seems to be a lot more influenced by beliefs in many cases than it is by actual physiological perception'</li></ul> |
| loc2  | <ul><li>'no hearer only hearing'</li><li>'conditioning'</li><li>'its not for you its like driving right but you can imagine like if you know youre like really into some sport or something it can be like a holy crap kind of actually happened to me with location two'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| loc1  | <ul><li>'still have a full range of emotions'</li><li>'they may notice less interest in their friends coworkers and even their family members and try to get them to interact in ways that are more interesting'</li><li>'location 1 was primarily marked by the sense'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| loc4  | <ul><li>'location 4 finders either remain alone or learn how to respond appropriately so that their partner feels loved'</li><li>'location 4'</li><li>'location four'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| loc5  | <ul><li>'once location 4 or beyond is reached these finders are typically left alone to deepen and explore their new location'</li><li>'the first which will be referred to as the path of freedom pof'</li><li>'this includes very lowlevel brain processes such as dissolution and reassembly of how the senses are perceived'</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |

## Evaluation

### Metrics
| Label   | Accuracy |
|:--------|:---------|
| **all** | 0.4058   |

## Uses

### Direct Use for Inference

First install the SetFit library:

```bash
pip install setfit
```

Then you can load this model and run inference.

```python
from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("jam1599/bert-text-classification-location")
# Run inference
preds = model("nonduality")
```

<!--
### Downstream Use

*List how someone could finetune this model on their own dataset.*
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Set Metrics
| Training set | Min | Median  | Max |
|:-------------|:----|:--------|:----|
| Word count   | 1   | 21.5667 | 283 |

| Label | Training Sample Count |
|:------|:----------------------|
| loc1  | 10                    |
| loc2  | 10                    |
| loc3  | 10                    |
| loc4  | 10                    |
| loc5  | 10                    |
| loc5+ | 10                    |

### Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (4, 4)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True

### Training Results
| Epoch   | Step    | Training Loss | Validation Loss |
|:-------:|:-------:|:-------------:|:---------------:|
| 0.0053  | 1       | 0.313         | -               |
| 0.2660  | 50      | 0.2641        | -               |
| 0.5319  | 100     | 0.0982        | -               |
| 0.7979  | 150     | 0.0445        | -               |
| **1.0** | **188** | **-**         | **0.2658**      |
| 1.0638  | 200     | 0.0222        | -               |
| 1.3298  | 250     | 0.0032        | -               |
| 1.5957  | 300     | 0.0013        | -               |
| 1.8617  | 350     | 0.0011        | -               |
| 2.0     | 376     | -             | 0.2883          |
| 2.1277  | 400     | 0.0013        | -               |
| 2.3936  | 450     | 0.0009        | -               |
| 2.6596  | 500     | 0.0003        | -               |
| 2.9255  | 550     | 0.0007        | -               |
| 3.0     | 564     | -             | 0.2913          |
| 3.1915  | 600     | 0.0006        | -               |
| 3.4574  | 650     | 0.0004        | -               |
| 3.7234  | 700     | 0.0012        | -               |
| 3.9894  | 750     | 0.0009        | -               |
| 4.0     | 752     | -             | 0.2943          |

* The bold row denotes the saved checkpoint.
### Framework Versions
- Python: 3.10.12
- SetFit: 1.0.2
- Sentence Transformers: 2.2.2
- Transformers: 4.35.2
- PyTorch: 2.1.0+cu121
- Datasets: 2.16.1
- Tokenizers: 0.15.0

## Citation

### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->
{"library_name": "setfit", "tags": ["setfit", "sentence-transformers", "text-classification", "generated_from_setfit_trainer"], "metrics": ["accuracy"], "widget": [{"text": "there s a lot of joy arising in location two it s one of the distinctive factors and i dont tend to notice i have days where im joyful and days where im pissed off and angry like ill have the whole spectrum of emotions and there s no obvious propensity towards joy that i notice"}, {"text": "location 4 finders who experience this in committed relationships are equally fortunate"}, {"text": "i can kind of provide a kind of resource for this human being whos you know trying so hard all the time you know like crying all the time and and when i and when i im able to have that awareness then the intentions the devotion that sense of devotion that that all boom"}, {"text": "and are often surprised by the words that come out of their mouths"}, {"text": "nonduality"}], "pipeline_tag": "text-classification", "inference": true, "model-index": [{"name": "SetFit", "results": [{"task": {"type": "text-classification", "name": "Text Classification"}, "dataset": {"name": "Unknown", "type": "unknown", "split": "test"}, "metrics": [{"type": "accuracy", "value": 0.4057971014492754, "name": "Accuracy"}]}]}]}
setfit safetensors bert sentence-transformers text-classification generated_from_setfit_trainer arxiv:2209.11055 model-index has_space region:us