
# Dataset Card for "LegalLAMA"

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Dataset Specifications](#supported-tasks-and-leaderboards)
 
## Dataset Description
- **Homepage:** https://github.com/coastalcph/lexlms
- **Repository:** https://github.com/coastalcph/lexlms
- **Paper:** https://arxiv.org/abs/2305.07507
- **Point of Contact:** [Ilias Chalkidis](mailto:ilias.chalkidis@di.ku.dk)

### Dataset Summary

LegalLAMA is a diverse probing benchmark suite comprising 8 sub-tasks that aims to assess the acquaintance of legal knowledge that PLMs acquired in pre-training.

### Dataset Specifications

| Corpus                               | Corpus alias         | Examples  | Avg. Tokens | Labels |
|--------------------------------------|----------------------|-----------|-------------|--------|
| Criminal Code Sections (Canada)      | `canadian_sections`  | 321       | 72          | 144    |
| Legal Terminology (EU)               | `cjeu_term`          | 2,127     | 164         | 23     |
| Contractual Section Titles (US)      | `contract_sections`  | 1,527     | 85          | 20     |
| Contract Types (US)                  | `contract_types`     | 1,089     | 150         | 15     |
| ECHR Articles (CoE)                  | `ecthr_articles`     | 5,072     | 69          | 13     |
| Legal Terminology (CoE)              | `ecthr_terms`        | 6,803     | 97          | 250    |
| Crime Charges (US)                   | `us_crimes`          | 4,518     | 118         | 59     |
| Legal Terminology (US)               | `us_terms`           | 5,829     | 308         | 7      |

### Usage

Load a specific sub-corpus, given the corpus alias, as presented above.

```python
from datasets import load_dataset
dataset = load_dataset('lexlms/legal_lama', name='ecthr_terms')
```

### Citation

[*Ilias Chalkidis\*, Nicolas Garneau\*, Catalina E.C. Goanta, Daniel Martin Katz, and Anders Søgaard.*
*LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development.*
*2022. In the Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics. Toronto, Canada.*](https://aclanthology.org/2023.acl-long.865/)
```
@inproceedings{chalkidis-etal-2023-lexfiles,
    title = "{L}e{XF}iles and {L}egal{LAMA}: Facilitating {E}nglish Multinational Legal Language Model Development",
    author = "Chalkidis, Ilias  and
      Garneau, Nicolas  and
      Goanta, Catalina  and
      Katz, Daniel  and
      S{\o}gaard, Anders",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.865",
    pages = "15513--15535",
}

```
task_categories:text-generation task_categories:fill-mask task_ids:masked-language-modeling annotations_creators:no-annotation language_creators:found multilinguality:monolingual size_categories:1K<n<10K source_datasets:extended language:en license:cc-by-nc-sa-4.0 legal law arxiv:2305.07507 region:us
{"annotations_creators": ["no-annotation"], "language_creators": ["found"], "language": ["en"], "license": ["cc-by-nc-sa-4.0"], "multilinguality": ["monolingual"], "size_categories": ["1K<n<10K"], "source_datasets": ["extended"], "task_categories": ["text-generation", "fill-mask"], "task_ids": ["masked-language-modeling"], "pretty_name": "LegalLAMA", "tags": ["legal", "law"]}
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-found #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-extended #language-English #license-cc-by-nc-sa-4.0 #legal #law #arxiv-2305.07507 #region-us 

Dataset Card for "LegalLAMA"
============================


Table of Contents
-----------------


* Dataset Description
	+ Dataset Summary
	+ Dataset Specifications


Dataset Description
-------------------


* Homepage: URL
* Repository: URL
* Paper: URL
* Point of Contact: Ilias Chalkidis


### Dataset Summary


LegalLAMA is a diverse probing benchmark suite comprising 8 sub-tasks that aims to assess the acquaintance of legal knowledge that PLMs acquired in pre-training.


### Dataset Specifications



### Usage


Load a specific sub-corpus, given the corpus alias, as presented above.


*Ilias Chalkidis\*, Nicolas Garneau\*, Catalina E.C. Goanta, Daniel Martin Katz, and Anders Søgaard.*
*LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development.*
*2022. In the Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics. Toronto, Canada.*



### Dataset Summary


LegalLAMA is a diverse probing benchmark suite comprising 8 sub-tasks that aims to assess the acquaintance of legal knowledge that PLMs acquired in pre-training. ### Dataset Specifications ### Usage


Load a specific sub-corpus, given the corpus alias, as presented above.


*Ilias Chalkidis\*, Nicolas Garneau\*, Catalina E.C. Goanta, Daniel Martin Katz, and Anders Søgaard.*
*LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development.*
*2022. In the Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics. Toronto, Canada.*
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-found #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-extended #language-English #license-cc-by-nc-sa-4.0 #legal #law #arxiv-2305.07507 #region-us 
 ### Dataset Summary


LegalLAMA is a diverse probing benchmark suite comprising 8 sub-tasks that aims to assess the acquaintance of legal knowledge that PLMs acquired in pre-training. ### Dataset Specifications ### Usage


Load a specific sub-corpus, given the corpus alias, as presented above.


*Ilias Chalkidis\*, Nicolas Garneau\*, Catalina E.C. Goanta, Daniel Martin Katz, and Anders Søgaard.*
*LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development.*
*2022. In the Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics. Toronto, Canada.*
passage: TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-found #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-extended #language-English #license-cc-by-nc-sa-4.0 #legal #law #arxiv-2305.07507 #region-us 
### Dataset Summary


LegalLAMA is a diverse probing benchmark suite comprising 8 sub-tasks that aims to assess the acquaintance of legal knowledge that PLMs acquired in pre-training.### Dataset Specifications### Usage


Load a specific sub-corpus, given the corpus alias, as presented above.


*Ilias Chalkidis\*, Nicolas Garneau\*, Catalina E.C. Goanta, Daniel Martin Katz, and Anders Søgaard.*
*LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development.*
*2022. In the Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics. Toronto, Canada.*