transformers pytorch bert fill-mask generated_from_trainer base_model:nlpaueb/legal-bert-base-uncased license:cc-by-sa-4.0 autotrain_compatible endpoints_compatible region:us
---
license: cc-by-sa-4.0
base_model: nlpaueb/legal-bert-base-uncased
tags:
- generated_from_trainer
model-index:
- name: ms-coa-legal-bert-base-uncased
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ms-coa-legal-bert-base-uncased

This model is a fine-tuned version of [nlpaueb/legal-bert-base-uncased](https://huggingface.co/nlpaueb/legal-bert-base-uncased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1135

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 1.5686        | 0.06  | 323   | 1.3832          |
| 1.461         | 0.12  | 646   | 1.3481          |
| 1.4377        | 0.19  | 969   | 1.3334          |
| 1.4151        | 0.25  | 1292  | 1.3130          |
| 1.3889        | 0.31  | 1615  | 1.2913          |
| 1.385         | 0.37  | 1938  | 1.2890          |
| 1.369         | 0.44  | 2261  | 1.2747          |
| 1.3597        | 0.5   | 2584  | 1.2654          |
| 1.367         | 0.56  | 2907  | 1.2632          |
| 1.3519        | 0.62  | 3230  | 1.2561          |
| 1.3294        | 0.69  | 3553  | 1.2455          |
| 1.326         | 0.75  | 3876  | 1.2428          |
| 1.3275        | 0.81  | 4199  | 1.2332          |
| 1.3174        | 0.87  | 4522  | 1.2276          |
| 1.3079        | 0.94  | 4845  | 1.2261          |
| 1.2978        | 1.0   | 5168  | 1.2142          |
| 1.2863        | 1.06  | 5491  | 1.2089          |
| 1.2832        | 1.12  | 5814  | 1.2022          |
| 1.2774        | 1.19  | 6137  | 1.2020          |
| 1.2851        | 1.25  | 6460  | 1.1975          |
| 1.2628        | 1.31  | 6783  | 1.1924          |
| 1.2756        | 1.37  | 7106  | 1.1918          |
| 1.2643        | 1.44  | 7429  | 1.1905          |
| 1.2578        | 1.5   | 7752  | 1.1850          |
| 1.2606        | 1.56  | 8075  | 1.1812          |
| 1.251         | 1.62  | 8398  | 1.1779          |
| 1.2477        | 1.69  | 8721  | 1.1731          |
| 1.2518        | 1.75  | 9044  | 1.1721          |
| 1.2428        | 1.81  | 9367  | 1.1723          |
| 1.2294        | 1.87  | 9690  | 1.1684          |
| 1.2357        | 1.94  | 10013 | 1.1630          |
| 1.2456        | 2.0   | 10336 | 1.1610          |
| 1.2196        | 2.06  | 10659 | 1.1588          |
| 1.2234        | 2.12  | 10982 | 1.1557          |
| 1.2353        | 2.19  | 11305 | 1.1530          |
| 1.2205        | 2.25  | 11628 | 1.1483          |
| 1.2243        | 2.31  | 11951 | 1.1539          |
| 1.226         | 2.37  | 12274 | 1.1519          |
| 1.2193        | 2.44  | 12597 | 1.1514          |
| 1.2152        | 2.5   | 12920 | 1.1472          |
| 1.2137        | 2.56  | 13243 | 1.1410          |
| 1.211         | 2.62  | 13566 | 1.1389          |
| 1.2113        | 2.69  | 13889 | 1.1443          |
| 1.206         | 2.75  | 14212 | 1.1448          |
| 1.213         | 2.81  | 14535 | 1.1375          |
| 1.1941        | 2.87  | 14858 | 1.1323          |
| 1.2089        | 2.94  | 15181 | 1.1364          |
| 1.2           | 3.0   | 15504 | 1.1320          |
| 1.1935        | 3.06  | 15827 | 1.1321          |
| 1.1874        | 3.12  | 16150 | 1.1296          |
| 1.1934        | 3.19  | 16473 | 1.1315          |
| 1.1928        | 3.25  | 16796 | 1.1271          |
| 1.1853        | 3.31  | 17119 | 1.1304          |
| 1.1931        | 3.37  | 17442 | 1.1266          |
| 1.1841        | 3.44  | 17765 | 1.1232          |
| 1.1734        | 3.5   | 18088 | 1.1300          |
| 1.18          | 3.56  | 18411 | 1.1254          |
| 1.174         | 3.62  | 18734 | 1.1236          |
| 1.1822        | 3.69  | 19057 | 1.1206          |
| 1.177         | 3.75  | 19380 | 1.1187          |
| 1.1756        | 3.81  | 19703 | 1.1183          |
| 1.1748        | 3.87  | 20026 | 1.1134          |
| 1.1693        | 3.94  | 20349 | 1.1138          |
| 1.1768        | 4.0   | 20672 | 1.1135          |


### Framework versions

- Transformers 4.34.1
- Pytorch 1.12.1+cu113
- Datasets 2.8.0
- Tokenizers 0.14.1
