
# SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a [SetFit](https://github.com/huggingface/setfit) model trained on the [Kevinger/hub-report-dataset](https://huggingface.co/datasets/Kevinger/hub-report-dataset) dataset that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

## Model Details

### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
- **Classification head:** a OneVsRestClassifier instance
- **Maximum Sequence Length:** 512 tokens
<!-- - **Number of Classes:** Unknown -->
- **Training Dataset:** [Kevinger/hub-report-dataset](https://huggingface.co/datasets/Kevinger/hub-report-dataset)
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)

## Evaluation

### Metrics
| Label   | Accuracy |
|:--------|:---------|
| **all** | 0.6529   |

## Uses

### Direct Use for Inference

First install the SetFit library:

```bash
pip install setfit
```

Then you can load this model and run inference.

```python
from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Kevinger/setfit-hub-multilabel-example")
# Run inference
preds = model("LEVERETT — Dakin Humane Society announced Wednesday that it has sold its former animal shelter at 63 Montague Road in Leverett to Better Together Dog Rescue.

The news release didn’t include a sales price for the 3,480-square-foot building on 5 acres of land.

But records at the Franklin County Registry of Deeds show the sale was for $575,000.")
```

<!--
### Downstream Use

*List how someone could finetune this model on their own dataset.*
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Set Metrics
| Training set | Min | Median   | Max  |
|:-------------|:----|:---------|:-----|
| Word count   | 53  | 386.3906 | 2161 |

### Training Hyperparameters
- batch_size: (8, 8)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 75
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False

### Training Results
| Epoch  | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0008 | 1    | 0.1304        | -               |
| 0.0417 | 50   | 0.1596        | -               |
| 0.0833 | 100  | 0.132         | -               |
| 0.125  | 150  | 0.0064        | -               |
| 0.1667 | 200  | 0.0017        | -               |
| 0.2083 | 250  | 0.0004        | -               |
| 0.25   | 300  | 0.0001        | -               |
| 0.2917 | 350  | 0.0002        | -               |
| 0.3333 | 400  | 0.0003        | -               |
| 0.375  | 450  | 0.0002        | -               |
| 0.4167 | 500  | 0.0001        | -               |
| 0.4583 | 550  | 0.0002        | -               |
| 0.5    | 600  | 0.0002        | -               |
| 0.5417 | 650  | 0.0002        | -               |
| 0.5833 | 700  | 0.0001        | -               |
| 0.625  | 750  | 0.0001        | -               |
| 0.6667 | 800  | 0.0001        | -               |
| 0.7083 | 850  | 0.0001        | -               |
| 0.75   | 900  | 0.0           | -               |
| 0.7917 | 950  | 0.0001        | -               |
| 0.8333 | 1000 | 0.0001        | -               |
| 0.875  | 1050 | 0.0001        | -               |
| 0.9167 | 1100 | 0.0001        | -               |
| 0.9583 | 1150 | 0.0           | -               |
| 1.0    | 1200 | 0.0001        | -               |

### Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.3.1
- Transformers: 4.35.2
- PyTorch: 2.1.0+cu121
- Datasets: 2.17.0
- Tokenizers: 0.15.2

## Citation

### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->
{"library_name": "setfit", "tags": ["setfit", "sentence-transformers", "text-classification", "generated_from_setfit_trainer"], "datasets": ["Kevinger/hub-report-dataset"], "metrics": ["accuracy"], "widget": [{"text": "A 16-acre property once home to the long-shuttered Foxborough State Hospital will soon provide housing for 141 low-income senior households.\n\nWalnut Street, an affordable housing project being developed by the Affordable Housing Services Collaborative and Onyx, will turn land that has been vacant for decades into much-needed affordable housing.\n\n\u201cHousing is empowering. No matter our age, it is a comfort not to worry about whether we can afford a place,\u201d Onyx CEO Chanda Smart said at a press conference Thursday. \u201cSenior housing for the town of Foxborough means that seniors who worked and raised their families here in Foxborough still have the opportunity to remain here.\u201d\n\nFoxborough State Hospital opened in 1889 as the Massachusetts Hospital for Dipsomaniacs and Inebriates for treatment of alcoholism, according to the National Park Service, and was later converted to a standard psychiatric hospital. It closed in 1975, and parts of the property have already been redeveloped over the years.\n\nThe Foxborough Housing Authority first began working on the project back in 2011. The land was transferred to the agency from the state in 2017 to be used for affordable housing.\n\nActing Town Manager Paige Duncan told MassLive that the town held a number of community meetings to decide what to build on the property.\n\n\u201cIt was controversial, but what came out was a clear support for senior housing,\u201d she said. \u201cWe really tried to address the needs of the community and we came up with a project that was sensitive to the area. We didn\u2019t want a big block of buildings that towered over the neighborhood.\u201d\n\nAfter that, she said, there was overwhelming support for the project. The permits were filed in February and approved by April, an almost unheard-of timeline.\n\nThe finished project will provide 141 new apartments for residents age 55 and over. Of those, 35 will be reserved for people making 30% or less of the area median income, and 85 will be for those making 60% AMI. Foxborough residents will be given preference for 70% of the units.\n\nA second phase of the project once this one is complete will add approximately 60 more units.\n\nGreg Spiers, chairman of the Housing Authority, said the new senior housing was badly needed, noting there are about 5,500 elderly and disabled people on public housing waiting lists in Massachusetts.\n\n\u201cWith 195 of those on that list Foxborough residents, that 70% local preference for first-time rentals is one of our goals,\u201d he said. \u201cThe need is so great for affordable housing in our area and the entire state.\u201d\n\nHousing and Livable Communities Secretary Ed Augustus praised the town for its dedication to creating more affordable housing, even though more than 10% of its total housing units qualify as affordable. The 10% threshold is the state requirement to stop projects being filed under Chapter 40B, a law which allows affordable housing developments to bypass certain local permitting requirements.\n\n\u201cYou know that that is just an arbitrary number, but the real needs are significantly more than that,\u201d Augustus said. \u201cWe need more communities to take note of what Foxborough is doing.\u201d\n\nLt. Gov. Kim Driscoll said the project is a good example of the use of surplus state land for housing. Gov. Maura Healey\u2019s housing bond bill filed in October included a proposed $30 million that would support similar projects to use underutilized state property for housing. Healey also issued an executive order requesting state agencies to conduct an audit of their property to find land any surplus land suitable for this purpose.\n\n\u201cConverting state-owned land to another entity can be a little bit of a torturous pathway. We know that building all the resources you need takes time,\u201d Driscoll said Thursday. \u201cHow do we leverage the cost of land, which is one of the reasons housing is so expensive, to build the type of housing we need, but do it in a shorter timetable? That\u2019s what this (project) is all about.\u201d\n\nThe project has received more than $25 million in state and federal funding, including through American Rescue Plan Act rental funds and state and federal Low Income Housing Tax Credits. Work on the site has not yet started."}, {"text": "\u201cI was on my co-op last year for, like, a straight year, so coming back to campus feels kind of nerve-wracking,\u201d said Jasmine Rodriguez, 21. \u201cBut I feel more experienced than I did in my first year. I had a lot of anxiety in my first year, but now it\u2019s been really chill.\u201d\n\nAs about a dozen Northeastern University students went around a conference table talking about their college experiences, voices were soft and answers halting, at least initially. Gradually, though, the students at this check-in meeting last fall began to open up and speak candidly about the challenges and adjustments of college life.\n\nAdvertisement\n\nThe students were Black, Latino, and Asian American and ranged from first-years to seniors, mostly from neighborhoods across Boston; the majority were the first generation of their family to attend college. Most were their high schools\u2019 valedictorians \u2014 hardworking, smart students who excelled despite lacking the advantages of many peers.\n\nThat\u2019s where The Valedictorian Project came in.\n\nThe Boston-based nonprofit was founded in 2020 in response to the Boston Globe\u2019s award-winning 2019 investigative series, The Valedictorians Project, which found that the city\u2019s best and brightest public school students often encounter major obstacles to their academic and professional goals. (The Globe is not involved with the organization.)\n\nThe Valedictorian Project matches participating high school graduates with peer mentors close to their age and a senior mentor who is an experienced professional in their intended line of work. It also provides a $500 stipend for books and other necessities, and supplemental support through partnerships with other organizations to help students navigate their new lives on campus and choose career paths.\n\n\u201cMany of our mentors are first-gen college students themselves,\u201d cofounder and executive director Amy McDermott said in an interview. \u201cMany navigated very similar personal backgrounds to our mentees. I hear often in our mentor interviews, they want to be that person that they wish they had in navigating college.\u201d\n\nAdvertisement\n\nThis academic year marks a milestone for the organization, as its first cohort of college freshmen are now seniors.\n\nMcDermott said the organization began by inviting Boston valedictorians to participate in its first year, then added students from Lawrence in year two, Brockton and Worcester in 2022, and Chelsea last spring.\n\nJasmine Rodriguez took part in a roundtable discussion at Northeastern University for students participating in The Valedictorian Project. Jonathan Wiggs/Globe Staff\n\nMentor John Marley, 30, of Taunton, said the organization helps level the playing field for young people who don\u2019t come from privileged backgrounds.\n\n\u201cStudents from wealthier families have always had these mentorship relationships, always had these connections, and those things are just unseen,\u201d said Marley, an attorney whose family came to the United States from Jamaica when he was 5. \u201cUnfairly or not ... it\u2019s always advantaged a particular group and class of students over another. And I think they do a good job addressing that.\u201d\n\nThis academic year, The Valedictorian Project is supporting 140 students, of whom about three-quarters are first-generation college students and roughly 85 percent are people of color, according to McDermott. Besides Northeastern, students in the program attend Boston University, Harvard, MIT, Tufts, Brown, Yale, Stanford, and other colleges around the country, she said.\n\nAs a student of color at an expensive private university, Rodriguez said, \u201cYou have to physically go out and try to find people that look like you. And I feel like for everyone else, it\u2019s very easy. They find them in their classes. But it\u2019s like, in my classes there\u2019ll be like one other Black or Hispanic person.\u201d\n\nAdvertisement\n\nRodriguez, a Dorchester native majoring in communications and sociology, recently spent a year as a social media co-op for an organization that supports domestic violence victims. She is drawn to work that will help others, she said, because she saw people in need in her neighborhood and her own family as she grew up.\n\n\u201cI saw a lot of people that look like me struggle and go through a lot of things,\u201d she said. \u201cMy mom is an immigrant. \u2026 We grew up on Section 8 [housing assistance]; we grew up on food stamps and stuff like that.\u201d\n\nCiana Omnis participated in a Northeastern University roundtable discussion for students participating in The Valedictorian Project. Jonathan Wiggs/Globe Staff\n\nCiana Omnis, 20, a third-year industrial engineering major who grew up in Florida, moved to Dorchester at age 14, and was the 2021 valedictorian at Brighton High School. She is the eldest of three children, so she can\u2019t lean on older siblings for advice, she said.\n\nHer father, a truck driver who immigrated to the United States from Haiti, didn\u2019t complete high school, she said, while her mother, a health care administrator, completed an associate\u2019s degree but doesn\u2019t yet have her bachelor\u2019s.\n\n\u201cI\u2019ve met a lot of people in college who have parents who have done four-year degrees or whatnot, or even other kinds of higher education, so they\u2019re able to get advice from their parents,\u201d Omnis said. \u201cFor me, it\u2019s been a bit harder, because I have to kind of figure out certain things on my own.\u201d\n\nAdvertisement\n\nHer mentors help fill that gap, she said, and the program helps her \u201cmeet other people who have the same background as me.\u201d\n\nAfter they met through a Valedictorian Project event, John Le, who was the 2022 valedictorian at East Boston High School, became friends with Connor Lashley, the 2022 valedictorian at Jeremiah E. Burke High School in Dorchester.\n\n\u201cOne of the issues is socializing, like making a friend group, because from my experience, from each class you kind of like meet people there, but if you\u2019re not in the same major, you might not be able to maintain a relationship with them,\u201d said Le, 20.\n\nThe Valedictorian Project, he added, \u201chas really been helpful to meet people at Northeastern and ... find people with similar interests.\u201d\n\nLashley, 19, said his mentors have helped him learn how to network with others in his field and steered him toward scholarship opportunities, and he can count on their support whenever he needs it.\n\n\u201cThey\u2019re pretty much available the same day if stuff comes up,\u201d he said.\n\nConnor Lashley (left) and John Le took part in a roundtable discussion at Northeastern University for students participating in The Valedictorian Project. Jonathan Wiggs/Globe Staff\n\nJeremy C. Fox can be reached at jeremy.fox@globe.com. Follow him @jeremycfox."}, {"text": "LEVERETT \u2014 Dakin Humane Society announced Wednesday that it has sold its former animal shelter at 63 Montague Road in Leverett to Better Together Dog Rescue.\n\nThe news release didn\u2019t include a sales price for the 3,480-square-foot building on 5 acres of land.\n\nBut records at the Franklin County Registry of Deeds show the sale was for $575,000."}, {"text": "Joan Acocella, a cultural critic whose elegant, erudite essays about dance and literature appeared in The New Yorker and The New York Review of Books for more than four decades, died on Sunday at her home in Manhattan. She was 78.\n\nHer son, Bartholomew Acocella, said the cause was cancer.\n\nMs. Acocella (pronounced ack-ah-CHELL-uh) wrote deeply about dancers and choreographers, including Mikhail Baryshnikov, Suzanne Farrell and George Balanchine. She scrutinized the vicissitudes of the New York City Ballet as well as the feats of the ballroom-dancing pros and celebrity oafs of the popular TV series \u201cDancing With the Stars.\u201d\n\nShe was The New Yorker\u2019s dance critic from 1998 to 2019 and freelanced for The Review for 33 years. Her final articles for The Review were a two-part commentary in May on the biography \u201cMr. B: George Balanchine\u2019s 20th Century,\u201d by Jennifer Homans, her successor as The New Yorker\u2019s dance critic.\n\n\u201cWhat she wrote for us,\u201d Emily Greenhouse, the editor of The Review, said in an email, \u201cwas often mischievous and always delicious \u2014 on crotch shots and cuss words, on Neapolitan hand gestures and Isadora Duncan\u2019s emphasis on the solar plexus.\u201d"}, {"text": "StreetsblogMASS relies on the generous support of readers like you. Help us meet our year-end fundraising goals \u2013 give today!\n\nLast week, the labor union that represents most Boston police officers ratified a new contract that will introduce a number of reforms \u2013 including one that will start allowing civilians to take unwanted traffic detail shifts at construction sites.\n\nUnder the former contract, Boston Police officers were the only people allowed to direct traffic for events and at construction sites. And they got paid extremely handsomely to do so: Boston police working as flaggers take home $60 an hour.\n\nIn spite of that lucrative pay, Boston has a lot of construction sites, and fewer and fewer people who want to wear a police uniform.\n\nBoston City Councilor Kendra Lara told StreetsblogMASS earlier this year that over 40 percent of requests for police details at construction sites were going unfilled.\n\nThe new labor contract removes a key barrier to reforming this system. But there is still a city ordinance on the books that requires at least one Boston Police officer at every city construction site \"to protect the safety and general welfare of the public and to preserve the free circulation of traffic.\"\n\nA press spokesperson for Mayor Michelle Wu told StreetsblogMASS last week that their office is aware of the ordinance and has \"identified multiple legal paths to implementing the new collective bargaining agreement.\"\n\nOld rules created absurd delays for street projects\n\nCouncilor Lara also told StreetsblogMASS that many privately-run construction sites will simply ignore the law and do their work without a flagger if nobody responds to their requests for a detail.\n\nBut construction firms who are sticklers for the rules can end up waiting months before a cop shows up to let them get their work done.\n\nThat's what happened earlier this year in Oak Square, where the MBTA waited a full year for a police detail to show up so that they could paint some new crosswalks on Washington Street in Oak Square.\n\nNeighbors report that those crosswalks finally got painted in August \u2013 after a year-long wait.\n\nNew contract hikes pay, allows civilian flaggers\n\nFor all these reasons, allowing civilian flaggers at construction sites had been one of the city's key points of negotiation for a new collective bargaining agreement with its police union.\n\nPolice details will still be required at \"high-priority\" events and construction sites, which involve major streets, busy intersections, or major events that anticipate over 5,000 attendees. The new contract would also pay cops who work those high-priority details \"the highest overtime rate of the most senior officer.\"\n\nAt other worksites, such as those along quiet neighborhood streets, Boston Police would still get the right of first refusal to fill traffic details. But if no Boston Police are interested, the work can be offered to other non-BPD certified officers, including campus police and retired Boston cops. If people with those qualifications still aren't interested, construction contractors can then offer the job to civilian workers.\n\nThe agreement further specifies that anyone directing traffic in those lower-priority sites will earn $60 per hour.\n\nThe new agreement will also ban cops from double-booking their shifts, which allowed some to get paid twice for the same period of time when one detail ended early.\n\nIncredibly, the police department is still using a labor-intensive paper-based system to assign details in each police district. The new agreement will allow for a citywide electronic scheduling system."}], "pipeline_tag": "text-classification", "inference": false, "base_model": "sentence-transformers/paraphrase-mpnet-base-v2", "model-index": [{"name": "SetFit with sentence-transformers/paraphrase-mpnet-base-v2", "results": [{"task": {"type": "text-classification", "name": "Text Classification"}, "dataset": {"name": "Kevinger/hub-report-dataset", "type": "Kevinger/hub-report-dataset", "split": "test"}, "metrics": [{"type": "accuracy", "value": 0.6529242569511026, "name": "Accuracy"}]}]}]}
setfit safetensors mpnet sentence-transformers text-classification generated_from_setfit_trainer dataset:Kevinger/hub-report-dataset arxiv:2209.11055 base_model:sentence-transformers/paraphrase-mpnet-base-v2 model-index region:us