
# SpanMarker

This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model that can be used for Named Entity Recognition.

## Model Details

### Model Description
- **Model Type:** SpanMarker
<!-- - **Encoder:** [Unknown](https://huggingface.co/unknown) -->
- **Maximum Sequence Length:** 512 tokens
- **Maximum Entity Length:** 8 words
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER)
- **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf)

### Model Labels
| Label           | Examples                                                                                                         |
|:----------------|:-----------------------------------------------------------------------------------------------------------------|
| BUSINESS        | "Merrill Lynch & Co. Inc.", "FormCap Advisors LLC", "FormCap Advisory LLC"                                       |
| COURT           | "New York City", "U.S.", "Delaware"                                                                              |
| GOVERNMENT      | "FASB ASC", "Advisers Act", "FASB Interpretation"                                                                |
| LEGISLATION/ACT | "non-small cell lung cancer", "NSCLC", "Stimuvax"                                                                |
| LOCATION        | "New York District Court", "Ninth Circuit", "United States District Court for the Southern District of New York" |
| MISCELLANEOUS   | "Scott Smith B .", "Dan Kelly", "James Altucher"                                                                 |
| PERSON          | "Securities and Exchange Commission", "Public Company Accounting Oversight Board ( United States )", "SEC"       |

## Uses

### Direct Use for Inference

```python
from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("lambdavi/span-luke-ener")
# Run inference
entities = model.predict("In October 2009 the FASB issued Accounting Standards Update 2009-15 Accounting for Own-Share Lending Arrangements in Contemplation of Convertible Debt Issuance or Other Financing.")
```

### Downstream Use
You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

```python
from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("lambdavi/span-luke-ener")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("lambdavi/span-luke-ener-finetuned")
```
</details>

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Set Metrics
| Training set          | Min | Median  | Max |
|:----------------------|:----|:--------|:----|
| Sentence length       | 1   | 32.8120 | 655 |
| Entities per sentence | 0   | 0.7204  | 61  |

### Training Hyperparameters
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.06
- num_epochs: 5

### Training Results
| Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy |
|:-----:|:----:|:---------------:|:--------------------:|:-----------------:|:-------------:|:-------------------:|
| 1.0   | 1572 | 0.0055          | 0.8175               | 0.6339            | 0.7141        | 0.9782              |
| 2.0   | 3144 | 0.0034          | 0.8722               | 0.8212            | 0.8459        | 0.9897              |
| 3.0   | 4716 | 0.0025          | 0.9056               | 0.8780            | 0.8916        | 0.9917              |
| 4.0   | 6288 | 0.0023          | 0.8965               | 0.9178            | 0.9070        | 0.9932              |
| 5.0   | 7860 | 0.0022          | 0.9083               | 0.9229            | 0.9155        | 0.9938              |

### Framework Versions
- Python: 3.10.13
- SpanMarker: 1.5.0
- Transformers: 4.37.0
- PyTorch: 2.1.2
- Datasets: 2.17.0
- Tokenizers: 0.15.1

## Citation

### BibTeX
```
@software{Aarsen_SpanMarker,
    author = {Aarsen, Tom},
    license = {Apache-2.0},
    title = {{SpanMarker for Named Entity Recognition}},
    url = {https://github.com/tomaarsen/SpanMarkerNER}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->
{"library_name": "span-marker", "tags": ["span-marker", "token-classification", "ner", "named-entity-recognition", "generated_from_span_marker_trainer"], "metrics": ["precision", "recall", "f1"], "widget": [{"text": "JPMDS a Delaware corporation is registered as a broker-dealer under the Exchange Act and serves and may in the future serve as principal underwriter to various Funds."}, {"text": "In October 2009 the FASB issued Accounting Standards Update 2009-15 Accounting for Own-Share Lending Arrangements in Contemplation of Convertible Debt Issuance or Other Financing."}, {"text": "Revenue Recognition In the fourth quarter of 2010 we adopted retrospective to the beginning of the year the provisions of Accounting Standards Update No. 2009-13 Revenue Recognition\u0097Multiple\u0097Deliverable Revenue Arrangements (ASU 2009-13) and Accounting Standards Update 2009-14 Software\u0097Certain Revenue Arrangements that Include Software Elements (ASU 2009-14)."}, {"text": "Bafetinib also has been granted Orphan Drug Status for the treatment of Philadelphia chromosome-positive (Ph+) CML by the U.S. Food and Drug Administration (FDA)."}, {"text": "The Fund s Forms N-Q may also be reviewed and copied at the Commission s Public Reference Room in Washington DC; information on the operation of the Public Reference Room may be obtained by calling (800) SEC-0330."}], "pipeline_tag": "token-classification", "model-index": [{"name": "SpanMarker", "results": [{"task": {"type": "token-classification", "name": "Named Entity Recognition"}, "dataset": {"name": "Unknown", "type": "unknown", "split": "eval"}, "metrics": [{"type": "f1", "value": 0.9155107187894074, "name": "F1"}, {"type": "precision", "value": 0.908256880733945, "name": "Precision"}, {"type": "recall", "value": 0.9228813559322034, "name": "Recall"}]}]}]}
span-marker safetensors token-classification ner named-entity-recognition generated_from_span_marker_trainer model-index region:us