transformers safetensors longformer token-classification generated_from_trainer dataset:essays_su_g base_model:allenai/longformer-base-4096 license:apache-2.0 model-index autotrain_compatible endpoints_compatible region:us
---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-simple
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: simple
      split: train[0%:20%]
      args: simple
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8399468193904684
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-simple

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 1.0670
- Claim: {'precision': 0.5894120517199317, 'recall': 0.5668700140778977, 'f1-score': 0.5779213012797513, 'support': 4262.0}
- Majorclaim: {'precision': 0.7822390174775626, 'recall': 0.7648960739030023, 'f1-score': 0.7734703409621673, 'support': 2165.0}
- O: {'precision': 0.9166666666666666, 'recall': 0.882853668423186, 'f1-score': 0.8994424943217014, 'support': 9868.0}
- Premise: {'precision': 0.8707947700896136, 'recall': 0.9091954904517218, 'f1-score': 0.889580910216486, 'support': 13039.0}
- Accuracy: 0.8399
- Macro avg: {'precision': 0.7897781264884436, 'recall': 0.780953811713952, 'f1-score': 0.7851037616950265, 'support': 29334.0}
- Weighted avg: {'precision': 0.8388075717984049, 'recall': 0.8399468193904684, 'f1-score': 0.8390471090378641, 'support': 29334.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                               | Majorclaim                                                                                                          | O                                                                                                                  | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 81   | 0.5489          | {'precision': 0.43257049448304047, 'recall': 0.496715157203191, 'f1-score': 0.46242900830056793, 'support': 4262.0} | {'precision': 0.6522533495736906, 'recall': 0.49468822170900695, 'f1-score': 0.5626477541371159, 'support': 2165.0} | {'precision': 0.911293908403735, 'recall': 0.8307661126874747, 'f1-score': 0.8691687871077185, 'support': 9868.0}  | {'precision': 0.8377046804810897, 'recall': 0.88672444205844, 'f1-score': 0.8615178272046495, 'support': 13039.0}   | 0.7823   | {'precision': 0.708455608235389, 'recall': 0.6772234834145282, 'f1-score': 0.6889408441875129, 'support': 29334.0}  | {'precision': 0.7899101236188295, 'recall': 0.7823004022635849, 'f1-score': 0.7840489998358311, 'support': 29334.0} |
| No log        | 2.0   | 162  | 0.5075          | {'precision': 0.5107383923092657, 'recall': 0.5858751759737213, 'f1-score': 0.5457327068079991, 'support': 4262.0}  | {'precision': 0.5687340153452686, 'recall': 0.8217090069284064, 'f1-score': 0.6722085773663329, 'support': 2165.0}  | {'precision': 0.9344863131370977, 'recall': 0.8268139440616133, 'f1-score': 0.8773589977955805, 'support': 9868.0} | {'precision': 0.8834419195931988, 'recall': 0.852749443975765, 'f1-score': 0.8678243902439025, 'support': 13039.0}  | 0.8030   | {'precision': 0.7243501600962077, 'recall': 0.7717868927348766, 'f1-score': 0.7407811680534537, 'support': 29334.0} | {'precision': 0.8232353684753937, 'recall': 0.8029590236585532, 'f1-score': 0.8097969994222006, 'support': 29334.0} |
| No log        | 3.0   | 243  | 0.5219          | {'precision': 0.5516552511415526, 'recall': 0.45354293758798686, 'f1-score': 0.4978109708987896, 'support': 4262.0} | {'precision': 0.8036105032822757, 'recall': 0.6785219399538106, 'f1-score': 0.7357876283496118, 'support': 2165.0}  | {'precision': 0.9331699710403207, 'recall': 0.8490068909606809, 'f1-score': 0.8891011355194736, 'support': 9868.0} | {'precision': 0.8191560170394037, 'recall': 0.9438607255157604, 'f1-score': 0.8770979581655561, 'support': 13039.0} | 0.8211   | {'precision': 0.7768979356258882, 'recall': 0.7312331235045597, 'f1-score': 0.7499494232333578, 'support': 29334.0} | {'precision': 0.8174973750724106, 'recall': 0.821129065248517, 'f1-score': 0.815598992812927, 'support': 29334.0}   |
| No log        | 4.0   | 324  | 0.4725          | {'precision': 0.5851569933396765, 'recall': 0.5771938057250118, 'f1-score': 0.5811481218993623, 'support': 4262.0}  | {'precision': 0.7335329341317365, 'recall': 0.792147806004619, 'f1-score': 0.761714412613813, 'support': 2165.0}    | {'precision': 0.9085720215857203, 'recall': 0.8872111876773409, 'f1-score': 0.8977645611156685, 'support': 9868.0} | {'precision': 0.8851474612344178, 'recall': 0.8930899608865711, 'f1-score': 0.8891009734682191, 'support': 13039.0} | 0.8378   | {'precision': 0.7781023525728877, 'recall': 0.7874106900733857, 'f1-score': 0.7824320172742658, 'support': 29334.0} | {'precision': 0.8382513248807655, 'recall': 0.8377650507943001, 'f1-score': 0.8378705011585708, 'support': 29334.0} |
| No log        | 5.0   | 405  | 0.5539          | {'precision': 0.5784176029962547, 'recall': 0.5797747536367902, 'f1-score': 0.5790953831731895, 'support': 4262.0}  | {'precision': 0.8008497079129049, 'recall': 0.6965357967667436, 'f1-score': 0.7450592885375493, 'support': 2165.0}  | {'precision': 0.9040794979079498, 'recall': 0.8758613700851237, 'f1-score': 0.8897467572575665, 'support': 9868.0} | {'precision': 0.8672442910639547, 'recall': 0.905820998542833, 'f1-score': 0.8861129867206842, 'support': 13039.0}  | 0.8329   | {'precision': 0.787647774970266, 'recall': 0.7644982297578727, 'f1-score': 0.7750036039222473, 'support': 29334.0}  | {'precision': 0.8327711951367025, 'recall': 0.8329242517215518, 'f1-score': 0.8323176558681596, 'support': 29334.0} |
| No log        | 6.0   | 486  | 0.5790          | {'precision': 0.5325670498084292, 'recall': 0.6848897231346786, 'f1-score': 0.5991994252283692, 'support': 4262.0}  | {'precision': 0.7400087834870444, 'recall': 0.7782909930715936, 'f1-score': 0.7586672669968483, 'support': 2165.0}  | {'precision': 0.9212211784799317, 'recall': 0.8745439805431698, 'f1-score': 0.8972759409440632, 'support': 9868.0} | {'precision': 0.9062909567496723, 'recall': 0.8485313290896541, 'f1-score': 0.8764605695726225, 'support': 13039.0} | 0.8283   | {'precision': 0.7750219921312693, 'recall': 0.796564006459774, 'f1-score': 0.7829008006854759, 'support': 29334.0}  | {'precision': 0.8447418748493872, 'recall': 0.8283220835890094, 'f1-score': 0.8344852708551485, 'support': 29334.0} |
| 0.3561        | 7.0   | 567  | 0.7058          | {'precision': 0.5955997904662127, 'recall': 0.5335523228531206, 'f1-score': 0.5628712871287128, 'support': 4262.0}  | {'precision': 0.8422198041349293, 'recall': 0.7150115473441109, 'f1-score': 0.7734199350487135, 'support': 2165.0}  | {'precision': 0.9193378321383383, 'recall': 0.8835630320226996, 'f1-score': 0.9010954940057875, 'support': 9868.0} | {'precision': 0.848316189939411, 'recall': 0.9234603880665695, 'f1-score': 0.8842947894099071, 'support': 13039.0}  | 0.8380   | {'precision': 0.8013684041697228, 'recall': 0.7638968225716252, 'f1-score': 0.7804203763982802, 'support': 29334.0} | {'precision': 0.8350403187795808, 'recall': 0.838003681734506, 'f1-score': 0.835063123988816, 'support': 29334.0}   |
| 0.3561        | 8.0   | 648  | 0.6876          | {'precision': 0.5858841386288894, 'recall': 0.5434068512435476, 'f1-score': 0.5638466220328667, 'support': 4262.0}  | {'precision': 0.8025641025641026, 'recall': 0.7228637413394919, 'f1-score': 0.7606318347509112, 'support': 2165.0}  | {'precision': 0.8864763037874281, 'recall': 0.9060599918929875, 'f1-score': 0.896161170692593, 'support': 9868.0}  | {'precision': 0.8807043836642937, 'recall': 0.9013728046629342, 'f1-score': 0.8909187386294724, 'support': 13039.0} | 0.8378   | {'precision': 0.7889072321611785, 'recall': 0.7684258472847404, 'f1-score': 0.7778895915264609, 'support': 29334.0} | {'precision': 0.8340438435010799, 'recall': 0.8377650507943001, 'f1-score': 0.8355454452418354, 'support': 29334.0} |
| 0.3561        | 9.0   | 729  | 0.6963          | {'precision': 0.5602836879432624, 'recall': 0.6302205537306429, 'f1-score': 0.5931978798586572, 'support': 4262.0}  | {'precision': 0.8248823836905385, 'recall': 0.7288683602771363, 'f1-score': 0.7739087788131438, 'support': 2165.0}  | {'precision': 0.927653083460449, 'recall': 0.8627888123226591, 'f1-score': 0.8940459939094823, 'support': 9868.0}  | {'precision': 0.8708454160160607, 'recall': 0.8982283917478334, 'f1-score': 0.8843249773482331, 'support': 13039.0} | 0.8349   | {'precision': 0.7959161427775777, 'recall': 0.7800265295195679, 'f1-score': 0.786369407482379, 'support': 29334.0}  | {'precision': 0.8414411074427396, 'recall': 0.8348673893775141, 'f1-score': 0.8371473756606818, 'support': 29334.0} |
| 0.3561        | 10.0  | 810  | 0.7715          | {'precision': 0.5701775147928994, 'recall': 0.5652275926794932, 'f1-score': 0.5676917638741604, 'support': 4262.0}  | {'precision': 0.7745940783190067, 'recall': 0.7491916859122402, 'f1-score': 0.7616811458088754, 'support': 2165.0}  | {'precision': 0.9296124365756234, 'recall': 0.8726185650587759, 'f1-score': 0.9002143118498772, 'support': 9868.0} | {'precision': 0.8669284467713787, 'recall': 0.9143339213129841, 'f1-score': 0.8900003732596766, 'support': 13039.0} | 0.8374   | {'precision': 0.785328119114727, 'recall': 0.7753429412408733, 'f1-score': 0.7798968986981474, 'support': 29334.0}  | {'precision': 0.8380850988337167, 'recall': 0.8373900593168337, 'f1-score': 0.8371368267053725, 'support': 29334.0} |
| 0.3561        | 11.0  | 891  | 0.7798          | {'precision': 0.5522299306243805, 'recall': 0.6536837165649929, 'f1-score': 0.5986891586977544, 'support': 4262.0}  | {'precision': 0.7361830742659758, 'recall': 0.7875288683602771, 'f1-score': 0.760990850256639, 'support': 2165.0}   | {'precision': 0.9179415855354659, 'recall': 0.8694770976895014, 'f1-score': 0.8930523028883683, 'support': 9868.0} | {'precision': 0.8987802946301283, 'recall': 0.8703121405015722, 'f1-score': 0.8843171634521723, 'support': 13039.0} | 0.8324   | {'precision': 0.7762837212639876, 'recall': 0.795250455779086, 'f1-score': 0.7842623688237336, 'support': 29334.0}  | {'precision': 0.8428746215263233, 'recall': 0.83244698984114, 'f1-score': 0.8366540534646059, 'support': 29334.0}   |
| 0.3561        | 12.0  | 972  | 0.8434          | {'precision': 0.5933806146572104, 'recall': 0.5889253871421868, 'f1-score': 0.5911446066886481, 'support': 4262.0}  | {'precision': 0.7994011976047904, 'recall': 0.7399538106235566, 'f1-score': 0.7685296234108899, 'support': 2165.0}  | {'precision': 0.9005203550658096, 'recall': 0.8944061613295501, 'f1-score': 0.8974528445777621, 'support': 9868.0} | {'precision': 0.8794646214001053, 'recall': 0.8970013037809648, 'f1-score': 0.8881464044346572, 'support': 13039.0} | 0.8398   | {'precision': 0.7931916971819789, 'recall': 0.7800716657190646, 'f1-score': 0.7863183697779894, 'support': 29334.0} | {'precision': 0.8390729472526346, 'recall': 0.8397763687188927, 'f1-score': 0.8392967405095946, 'support': 29334.0} |
| 0.0633        | 13.0  | 1053 | 0.9362          | {'precision': 0.5771842462652784, 'recall': 0.5983106522759268, 'f1-score': 0.5875576036866359, 'support': 4262.0}  | {'precision': 0.7786116322701688, 'recall': 0.766743648960739, 'f1-score': 0.7726320688852689, 'support': 2165.0}   | {'precision': 0.9130388953304522, 'recall': 0.8777867855695176, 'f1-score': 0.8950658744510462, 'support': 9868.0} | {'precision': 0.8741821463488004, 'recall': 0.891479407930056, 'f1-score': 0.8827460510328068, 'support': 13039.0}  | 0.8351   | {'precision': 0.785754230053675, 'recall': 0.7835801236840598, 'f1-score': 0.7845003995139396, 'support': 29334.0}  | {'precision': 0.8370485534468686, 'recall': 0.835071930183405, 'f1-score': 0.83587491458883, 'support': 29334.0}    |
| 0.0633        | 14.0  | 1134 | 1.0311          | {'precision': 0.6124338624338624, 'recall': 0.5431722196152041, 'f1-score': 0.5757274309873166, 'support': 4262.0}  | {'precision': 0.7854137447405329, 'recall': 0.7759815242494227, 'f1-score': 0.7806691449814126, 'support': 2165.0}  | {'precision': 0.916153682869879, 'recall': 0.8747466558573166, 'f1-score': 0.894971487817522, 'support': 9868.0}   | {'precision': 0.8590723933395269, 'recall': 0.9219265281079837, 'f1-score': 0.8893903521751998, 'support': 13039.0} | 0.8403   | {'precision': 0.7932684208459503, 'recall': 0.7789567319574817, 'f1-score': 0.7851896039903627, 'support': 29334.0} | {'precision': 0.8370035916809992, 'recall': 0.8402536305993046, 'f1-score': 0.8376709093048489, 'support': 29334.0} |
| 0.0633        | 15.0  | 1215 | 1.0063          | {'precision': 0.5736224028906955, 'recall': 0.5959643359924918, 'f1-score': 0.5845799769850402, 'support': 4262.0}  | {'precision': 0.8367459878251245, 'recall': 0.6983833718244804, 'f1-score': 0.7613293051359518, 'support': 2165.0}  | {'precision': 0.9253507550605119, 'recall': 0.8755573571139035, 'f1-score': 0.8997656860192659, 'support': 9868.0} | {'precision': 0.8641912512716174, 'recall': 0.9121098243730348, 'f1-score': 0.8875041976045669, 'support': 13039.0} | 0.8381   | {'precision': 0.7999775992619873, 'recall': 0.7705037223259776, 'f1-score': 0.7832947914362062, 'support': 29334.0} | {'precision': 0.8405224217982303, 'recall': 0.8381059521374514, 'f1-score': 0.8383041122838222, 'support': 29334.0} |
| 0.0633        | 16.0  | 1296 | 0.9864          | {'precision': 0.6114068441064638, 'recall': 0.5659314875645237, 'f1-score': 0.587790910198611, 'support': 4262.0}   | {'precision': 0.8076540755467196, 'recall': 0.7505773672055427, 'f1-score': 0.7780703854440987, 'support': 2165.0}  | {'precision': 0.9003660024400163, 'recall': 0.8974462910417511, 'f1-score': 0.8989037758830695, 'support': 9868.0} | {'precision': 0.8745292075917583, 'recall': 0.908198481478641, 'f1-score': 0.89104589917231, 'support': 13039.0}    | 0.8432   | {'precision': 0.7984890324212395, 'recall': 0.7805384068226147, 'f1-score': 0.7889527426745223, 'support': 29334.0} | {'precision': 0.8400553996388973, 'recall': 0.8432194722847208, 'f1-score': 0.8412905564694496, 'support': 29334.0} |
| 0.0633        | 17.0  | 1377 | 1.0474          | {'precision': 0.5777574788764558, 'recall': 0.5936180197090568, 'f1-score': 0.5855803726420553, 'support': 4262.0}  | {'precision': 0.7891123099558607, 'recall': 0.7431870669745958, 'f1-score': 0.7654614652711702, 'support': 2165.0}  | {'precision': 0.9293800539083558, 'recall': 0.8735306039724362, 'f1-score': 0.9005902941022829, 'support': 9868.0} | {'precision': 0.869437724507001, 'recall': 0.9095789554413682, 'f1-score': 0.8890554722638682, 'support': 13039.0}  | 0.8393   | {'precision': 0.7914218918119184, 'recall': 0.7799786615243642, 'f1-score': 0.7851719010698441, 'support': 29334.0} | {'precision': 0.8412951315142951, 'recall': 0.8392650167041659, 'f1-score': 0.8397213794764584, 'support': 29334.0} |
| 0.0633        | 18.0  | 1458 | 1.0609          | {'precision': 0.5889078083191438, 'recall': 0.5680431722196152, 'f1-score': 0.5782873522035114, 'support': 4262.0}  | {'precision': 0.7991159135559921, 'recall': 0.751501154734411, 'f1-score': 0.7745774815520113, 'support': 2165.0}   | {'precision': 0.9093172857439302, 'recall': 0.8881232265910012, 'f1-score': 0.8985953040090229, 'support': 9868.0} | {'precision': 0.8732747804265998, 'recall': 0.9074315514993481, 'f1-score': 0.8900255754475703, 'support': 13039.0} | 0.8401   | {'precision': 0.7926539470114164, 'recall': 0.7787747762610939, 'f1-score': 0.7853714283030289, 'support': 29334.0} | {'precision': 0.8386099362381009, 'recall': 0.8401172700620441, 'f1-score': 0.8390946642419506, 'support': 29334.0} |
| 0.0173        | 19.0  | 1539 | 1.0791          | {'precision': 0.586526726873322, 'recall': 0.5638198029094322, 'f1-score': 0.5749491565976791, 'support': 4262.0}   | {'precision': 0.7679227941176471, 'recall': 0.771824480369515, 'f1-score': 0.7698686938493434, 'support': 2165.0}   | {'precision': 0.9245000534702171, 'recall': 0.8760640453992704, 'f1-score': 0.8996305739112337, 'support': 9868.0} | {'precision': 0.8675419401896426, 'recall': 0.912186517370964, 'f1-score': 0.8893042730569367, 'support': 13039.0}  | 0.8391   | {'precision': 0.7866228786627072, 'recall': 0.7809737115122954, 'f1-score': 0.7834381743537983, 'support': 29334.0} | {'precision': 0.8385210215100449, 'recall': 0.839060475898275, 'f1-score': 0.8382897643467849, 'support': 29334.0}  |
| 0.0173        | 20.0  | 1620 | 1.0670          | {'precision': 0.5894120517199317, 'recall': 0.5668700140778977, 'f1-score': 0.5779213012797513, 'support': 4262.0}  | {'precision': 0.7822390174775626, 'recall': 0.7648960739030023, 'f1-score': 0.7734703409621673, 'support': 2165.0}  | {'precision': 0.9166666666666666, 'recall': 0.882853668423186, 'f1-score': 0.8994424943217014, 'support': 9868.0}  | {'precision': 0.8707947700896136, 'recall': 0.9091954904517218, 'f1-score': 0.889580910216486, 'support': 13039.0}  | 0.8399   | {'precision': 0.7897781264884436, 'recall': 0.780953811713952, 'f1-score': 0.7851037616950265, 'support': 29334.0}  | {'precision': 0.8388075717984049, 'recall': 0.8399468193904684, 'f1-score': 0.8390471090378641, 'support': 29334.0} |


### Framework versions

- Transformers 4.38.2
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
