# Dataset Card for PIEs corpus

### Dataset Summary

This corpus is a collection of 57170 potentially idiomatic expressions (PIEs) based on the British National Corpus, prepaired for NER task.
Each of the objects is comes with a contextual set of tokens, BIO tags and boolean label.
The data sources are:
* [MAGPIE corpus](https://github.com/hslh/magpie-corpus)
* [PIE corpus](https://github.com/hslh/pie-annotation)

Detailed data preparation pipeline can be found [here](https://github.com/Gooogr/Idioms_spotter)
### Supported Tasks and Leaderboards

Token classification (NER)

### Languages

English

## Dataset Structure

### Data Instances
For each instance there is a string with target idiom, tokenized by word text with context of idiom usage, corresponded BIO tags 
and boolean label `is_pie`. This tag determines whether or not a collocation is considered an idiom in a given context. 
For a PIE dataset the choice was determined by the original PIE_label. For MAGPIE a threshold of 0.75 confidence coefficient was chosen.

An example from the train set looks like the following:
```
{'idiom': "go public"
'is_pie': True
'tokens': [ "Private", "dealers", "in", "the", "States", "go", "public" ]
'ner_tags': [ 0, 0, 0, 0, 0, 1, 2 ]
}
```
Where NER tags is {0: 'O', 1: 'B-PIE', 2: 'I-PIE'}

### Data Fields

* idiom: a string containg original PIE
* is_pie: a boolean label determining whether a PIE can be considered an idiom in a given context
* tokens: sequence of word tkenized string with PIE usage context
* ner_tags: corresponded BIO tags for word tokens

### Data Splits

The SNLI dataset has 3 splits: _train_, _validation_, and _test_. 

| Dataset Split | Number of Instances in Split |
| ------------- |----------------------------- |
| Train         | 45,736                       |
| Validation    | 5,717                        |
| Test          | 5,717                        |

## Dataset Creation

### Source Data

#### Initial Data Collection and Normalization

* [MAGPIE corpus](https://github.com/hslh/magpie-corpus)
* [PIE English corpus](https://github.com/hslh/pie-annotation)


## Additional Information

### Licensing Information

Corpus and it's sources are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

### Citation Information
[PIE Corpus](https://github.com/hslh/pie-annotation) (Haagsma, H. (Creator), Bos, J. (Contributor), Plank, B. (Contributor), University of Groningen.)<br>
[MAGPIE: A Large Corpus of Potentially Idiomatic Expressions](https://aclanthology.org/2020.lrec-1.35) (Haagsma et al., LREC 2020)
task_categories:token-classification size_categories:10K<n<100K language:en license:cc-by-4.0 PIE idioms region:us
{"language": ["en"], "license": "cc-by-4.0", "size_categories": ["10K<n<100K"], "task_categories": ["token-classification"], "pretty_name": "Corpus of potentially idiomatic expressions (PIEs)", "dataset_info": {"features": [{"name": "idiom", "dtype": "string"}, {"name": "is_pie", "dtype": "bool"}, {"name": "tokens", "sequence": "string"}, {"name": "ner_tags", "sequence": {"class_label": {"names": {"0": "O", "1": "B-PIE", "2": "I-PIE"}}}}], "splits": [{"name": "train", "num_bytes": 82950018, "num_examples": 46090}, {"name": "validation", "num_bytes": 10420303, "num_examples": 5761}, {"name": "test", "num_bytes": 10376839, "num_examples": 5762}], "download_size": 19258913, "dataset_size": 103747160}, "tags": ["PIE", "idioms"]}
TAGS
#task_categories-token-classification #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #PIE #idioms #region-us 

Dataset Card for PIEs corpus
============================


### Dataset Summary


This corpus is a collection of 57170 potentially idiomatic expressions (PIEs) based on the British National Corpus, prepaired for NER task.
Each of the objects is comes with a contextual set of tokens, BIO tags and boolean label.
The data sources are:


* MAGPIE corpus
* PIE corpus


Detailed data preparation pipeline can be found here


### Supported Tasks and Leaderboards


Token classification (NER)


### Languages


English


Dataset Structure
-----------------


### Data Instances


For each instance there is a string with target idiom, tokenized by word text with context of idiom usage, corresponded BIO tags
and boolean label 'is\_pie'. This tag determines whether or not a collocation is considered an idiom in a given context.
For a PIE dataset the choice was determined by the original PIE\_label. For MAGPIE a threshold of 0.75 confidence coefficient was chosen.


An example from the train set looks like the following:


Where NER tags is {0: 'O', 1: 'B-PIE', 2: 'I-PIE'}


### Data Fields


* idiom: a string containg original PIE
* is\_pie: a boolean label determining whether a PIE can be considered an idiom in a given context
* tokens: sequence of word tkenized string with PIE usage context
* ner\_tags: corresponded BIO tags for word tokens


### Data Splits


The SNLI dataset has 3 splits: *train*, *validation*, and *test*.



Dataset Creation
----------------


### Source Data


#### Initial Data Collection and Normalization


* MAGPIE corpus
* PIE English corpus


Additional Information
----------------------


### Licensing Information


Corpus and it's sources are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.


PIE Corpus (Haagsma, H. (Creator), Bos, J. (Contributor), Plank, B. (Contributor), University of Groningen.)  

MAGPIE: A Large Corpus of Potentially Idiomatic Expressions (Haagsma et al., LREC 2020)



### Dataset Summary


This corpus is a collection of 57170 potentially idiomatic expressions (PIEs) based on the British National Corpus, prepaired for NER task.
Each of the objects is comes with a contextual set of tokens, BIO tags and boolean label.
The data sources are:


* MAGPIE corpus
* PIE corpus


Detailed data preparation pipeline can be found here ### Supported Tasks and Leaderboards


Token classification (NER) ### Languages


English


Dataset Structure
----------------- ### Data Instances


For each instance there is a string with target idiom, tokenized by word text with context of idiom usage, corresponded BIO tags
and boolean label 'is\_pie'. This tag determines whether or not a collocation is considered an idiom in a given context.
For a PIE dataset the choice was determined by the original PIE\_label. For MAGPIE a threshold of 0.75 confidence coefficient was chosen.


An example from the train set looks like the following:


Where NER tags is {0: 'O', 1: 'B-PIE', 2: 'I-PIE'} ### Data Fields


* idiom: a string containg original PIE
* is\_pie: a boolean label determining whether a PIE can be considered an idiom in a given context
* tokens: sequence of word tkenized string with PIE usage context
* ner\_tags: corresponded BIO tags for word tokens ### Data Splits


The SNLI dataset has 3 splits: *train*, *validation*, and *test*.



Dataset Creation
---------------- ### Source Data #### Initial Data Collection and Normalization


* MAGPIE corpus
* PIE English corpus


Additional Information
---------------------- ### Licensing Information


Corpus and it's sources are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.


PIE Corpus (Haagsma, H. (Creator), Bos, J. (Contributor), Plank, B. (Contributor), University of Groningen.)  

MAGPIE: A Large Corpus of Potentially Idiomatic Expressions (Haagsma et al., LREC 2020)
TAGS
#task_categories-token-classification #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #PIE #idioms #region-us 
 ### Dataset Summary


This corpus is a collection of 57170 potentially idiomatic expressions (PIEs) based on the British National Corpus, prepaired for NER task.
Each of the objects is comes with a contextual set of tokens, BIO tags and boolean label.
The data sources are:


* MAGPIE corpus
* PIE corpus


Detailed data preparation pipeline can be found here ### Supported Tasks and Leaderboards


Token classification (NER) ### Languages


English


Dataset Structure
----------------- ### Data Instances


For each instance there is a string with target idiom, tokenized by word text with context of idiom usage, corresponded BIO tags
and boolean label 'is\_pie'. This tag determines whether or not a collocation is considered an idiom in a given context.
For a PIE dataset the choice was determined by the original PIE\_label. For MAGPIE a threshold of 0.75 confidence coefficient was chosen.


An example from the train set looks like the following:


Where NER tags is {0: 'O', 1: 'B-PIE', 2: 'I-PIE'} ### Data Fields


* idiom: a string containg original PIE
* is\_pie: a boolean label determining whether a PIE can be considered an idiom in a given context
* tokens: sequence of word tkenized string with PIE usage context
* ner\_tags: corresponded BIO tags for word tokens ### Data Splits


The SNLI dataset has 3 splits: *train*, *validation*, and *test*.



Dataset Creation
---------------- ### Source Data #### Initial Data Collection and Normalization


* MAGPIE corpus
* PIE English corpus


Additional Information
---------------------- ### Licensing Information


Corpus and it's sources are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.


PIE Corpus (Haagsma, H. (Creator), Bos, J. (Contributor), Plank, B. (Contributor), University of Groningen.)  

MAGPIE: A Large Corpus of Potentially Idiomatic Expressions (Haagsma et al., LREC 2020)
passage: TAGS
#task_categories-token-classification #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #PIE #idioms #region-us 
### Dataset Summary


This corpus is a collection of 57170 potentially idiomatic expressions (PIEs) based on the British National Corpus, prepaired for NER task.
Each of the objects is comes with a contextual set of tokens, BIO tags and boolean label.
The data sources are:


* MAGPIE corpus
* PIE corpus


Detailed data preparation pipeline can be found here### Supported Tasks and Leaderboards


Token classification (NER)### Languages


English


Dataset Structure
-----------------### Data Instances


For each instance there is a string with target idiom, tokenized by word text with context of idiom usage, corresponded BIO tags
and boolean label 'is\_pie'. This tag determines whether or not a collocation is considered an idiom in a given context.
For a PIE dataset the choice was determined by the original PIE\_label. For MAGPIE a threshold of 0.75 confidence coefficient was chosen.


An example from the train set looks like the following:


Where NER tags is {0: 'O', 1: 'B-PIE', 2: 'I-PIE'}### Data Fields


* idiom: a string containg original PIE
* is\_pie: a boolean label determining whether a PIE can be considered an idiom in a given context
* tokens: sequence of word tkenized string with PIE usage context
* ner\_tags: corresponded BIO tags for word tokens### Data Splits


The SNLI dataset has 3 splits: *train*, *validation*, and *test*.



Dataset Creation
----------------### Source Data#### Initial Data Collection and Normalization


* MAGPIE corpus
* PIE English corpus


Additional Information
----------------------