
# Function Of Citation in Astrophysics Literature (FOCAL): Dataset and Task
*Can you explain why the authors made a given citation?*

This dataset was created as a [shared task](https://ui.adsabs.harvard.edu/WIESP/2023/shared_task_1) for [WIESP @ AACL-IJCNLP 2023](https://ui.adsabs.harvard.edu/WIESP/2023/).

## Dataset Description
Datasets are in JSON Lines format (each line is a json dictionary).  

Each entry consists of a dictionary with the following keys:
- `"Identifier"`: unique string to identify the entry
- `"Paragraph"`: text string from an astrophysics paper 
- `"Citation Text"`: list of strings forming the citation (most often a single string, but sometimes the citation text is split up)
- `"Citation Start End"`: list of integer pairs denoting where the citation starts and end in `"Paragraph"` (most often a single pair, sometimes the citation text is split up, if so follows the order in `"Citation Text"`)
- `"Functions Text"`: list of strings highlighting parts of the paragraph that explain the function of the citation
- `"Functions Label"`: list of strings with the label for each text element in `"Functions Text"` (in same order)
- `"Functions Start End"`: list of integer pairs denoting where the elements in `"Functions Text"` start and end in `"Paragraph"`(in same order)

start and end are defined by the character position in the `"Paragraph"` string.


## Instructions for Workshop Participants:
How to load the data using the Huggingface library:
```python
from datasets import load_dataset
dataset = load_dataset("adsabs/FOCAL")
```

How to load the data if you cloned the repository locally:  
(assuming `./FOCAL-TRAINING.jsonl` is in the current directory, change as needed)
- python (as list of dictionaries):
```python
import json
with open("./FOCAL-TRAINING.jsonl", 'r') as f:
    focal_training_from_json = [json.loads(l) for l in list(f)]
```
 - into Huggingface (as a Huggingface Dataset):
```python
from datasets import Dataset
focal_training_from_json = Dataset.from_json(path_or_paths="./FOCAL-TRAINING.jsonl")
```

## File List
``` 
├── FOCAL-TRAINING.jsonl (2421 samples for training)
├── FOCAL-VALIDATION.jsonl (606 samples for validating your training methods)
├── FOCAL-TESTING.jsonl (821 samples for testing)
├── FOCAL-VALIDATION-NO-LABELS.jsonl (606 samples for validation without the labels. Used during the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/)  
├── FOCAL-TESTING-NO-LABELS.jsonl (821 samples for testing without the labels. Used during the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/)
├── /scoring_scripts/score_focal_seqeval.py (scoring scripts used during the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/)
├── /scoring_scripts/score_focal_labels_only.py (scoring scripts used during the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/)
├── /data/*.parquet (files used when loading the dataset through Huggingface's API)
├── README.MD (this file)  
└──  
```

Maintainer: Felix Grezes (ORCID: 0000-0001-8714-7774)  
Data annotator: Tom Allen (ORCID: 0000-0002-5532-4809)
task_categories:token-classification annotations_creators:expert-generated multilinguality:monolingual size_categories:1K<n<10K language:en license:cc-by-4.0 astronomy region:us
{"annotations_creators": ["expert-generated"], "language": ["en"], "license": "cc-by-4.0", "multilinguality": ["monolingual"], "size_categories": ["1K<n<10K"], "task_categories": ["token-classification"], "tags": ["astronomy"], "dataset_info": {"features": [{"name": "Identifier", "dtype": "string"}, {"name": "Paragraph", "dtype": "string"}, {"name": "Citation Text", "sequence": "string"}, {"name": "Functions Text", "sequence": "string"}, {"name": "Functions Label", "sequence": "string"}, {"name": "Citation Start End", "sequence": {"sequence": "int64"}}, {"name": "Functions Start End", "sequence": {"sequence": "int64"}}], "splits": [{"name": "train", "num_bytes": 7096500, "num_examples": 2421}, {"name": "validation", "num_bytes": 1761751, "num_examples": 606}, {"name": "test", "num_bytes": 2512022, "num_examples": 821}], "download_size": 5649484, "dataset_size": 11370273}}
TAGS
#task_categories-token-classification #annotations_creators-expert-generated #multilinguality-monolingual #size_categories-1K<n<10K #language-English #license-cc-by-4.0 #astronomy #region-us 


# Function Of Citation in Astrophysics Literature (FOCAL): Dataset and Task
*Can you explain why the authors made a given citation?*

This dataset was created as a shared task for WIESP @ AACL-IJCNLP 2023.

## Dataset Description
Datasets are in JSON Lines format (each line is a json dictionary).  

Each entry consists of a dictionary with the following keys:
- '"Identifier"': unique string to identify the entry
- '"Paragraph"': text string from an astrophysics paper 
- '"Citation Text"': list of strings forming the citation (most often a single string, but sometimes the citation text is split up)
- '"Citation Start End"': list of integer pairs denoting where the citation starts and end in '"Paragraph"' (most often a single pair, sometimes the citation text is split up, if so follows the order in '"Citation Text"')
- '"Functions Text"': list of strings highlighting parts of the paragraph that explain the function of the citation
- '"Functions Label"': list of strings with the label for each text element in '"Functions Text"' (in same order)
- '"Functions Start End"': list of integer pairs denoting where the elements in '"Functions Text"' start and end in '"Paragraph"'(in same order)

start and end are defined by the character position in the '"Paragraph"' string.


## Instructions for Workshop Participants:
How to load the data using the Huggingface library:


How to load the data if you cloned the repository locally:  
(assuming './URL' is in the current directory, change as needed)
- python (as list of dictionaries):

 - into Huggingface (as a Huggingface Dataset):


## File List


Maintainer: Felix Grezes (ORCID: 0000-0001-8714-7774)  
Data annotator: Tom Allen (ORCID: 0000-0002-5532-4809)
# Function Of Citation in Astrophysics Literature (FOCAL): Dataset and Task
*Can you explain why the authors made a given citation?*

This dataset was created as a shared task for WIESP @ AACL-IJCNLP 2023. ## Dataset Description
Datasets are in JSON Lines format (each line is a json dictionary).  

Each entry consists of a dictionary with the following keys:
- '"Identifier"': unique string to identify the entry
- '"Paragraph"': text string from an astrophysics paper 
- '"Citation Text"': list of strings forming the citation (most often a single string, but sometimes the citation text is split up)
- '"Citation Start End"': list of integer pairs denoting where the citation starts and end in '"Paragraph"' (most often a single pair, sometimes the citation text is split up, if so follows the order in '"Citation Text"')
- '"Functions Text"': list of strings highlighting parts of the paragraph that explain the function of the citation
- '"Functions Label"': list of strings with the label for each text element in '"Functions Text"' (in same order)
- '"Functions Start End"': list of integer pairs denoting where the elements in '"Functions Text"' start and end in '"Paragraph"'(in same order)

start and end are defined by the character position in the '"Paragraph"' string. ## Instructions for Workshop Participants:
How to load the data using the Huggingface library:


How to load the data if you cloned the repository locally:  
(assuming './URL' is in the current directory, change as needed)
- python (as list of dictionaries):

 - into Huggingface (as a Huggingface Dataset): ## File List


Maintainer: Felix Grezes (ORCID: 0000-0001-8714-7774)  
Data annotator: Tom Allen (ORCID: 0000-0002-5532-4809)
TAGS
#task_categories-token-classification #annotations_creators-expert-generated #multilinguality-monolingual #size_categories-1K<n<10K #language-English #license-cc-by-4.0 #astronomy #region-us 
 # Function Of Citation in Astrophysics Literature (FOCAL): Dataset and Task
*Can you explain why the authors made a given citation?*

This dataset was created as a shared task for WIESP @ AACL-IJCNLP 2023. ## Dataset Description
Datasets are in JSON Lines format (each line is a json dictionary).  

Each entry consists of a dictionary with the following keys:
- '"Identifier"': unique string to identify the entry
- '"Paragraph"': text string from an astrophysics paper 
- '"Citation Text"': list of strings forming the citation (most often a single string, but sometimes the citation text is split up)
- '"Citation Start End"': list of integer pairs denoting where the citation starts and end in '"Paragraph"' (most often a single pair, sometimes the citation text is split up, if so follows the order in '"Citation Text"')
- '"Functions Text"': list of strings highlighting parts of the paragraph that explain the function of the citation
- '"Functions Label"': list of strings with the label for each text element in '"Functions Text"' (in same order)
- '"Functions Start End"': list of integer pairs denoting where the elements in '"Functions Text"' start and end in '"Paragraph"'(in same order)

start and end are defined by the character position in the '"Paragraph"' string. ## Instructions for Workshop Participants:
How to load the data using the Huggingface library:


How to load the data if you cloned the repository locally:  
(assuming './URL' is in the current directory, change as needed)
- python (as list of dictionaries):

 - into Huggingface (as a Huggingface Dataset): ## File List


Maintainer: Felix Grezes (ORCID: 0000-0001-8714-7774)  
Data annotator: Tom Allen (ORCID: 0000-0002-5532-4809)