

# Dataset Card for [covid_qa_castorini]

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** http://covidqa.ai
- **Repository:** https://github.com/castorini/pygaggle
- **Paper:** https://arxiv.org/abs/2004.11339
- **Point of Contact:** [Castorini research group @UWaterloo](https://github.com/castorini/)

### Dataset Summary

CovidQA is a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered 
from Kaggle’s COVID-19 Open Research Dataset Challenge.
The dataset comprises 156 question-article pairs with 27 questions (topics) and 85 unique articles.

### Supported Tasks and Leaderboards

[More Information Needed]

### Languages

The text in the dataset is in English.

## Dataset Structure

### Data Instances

**What do the instances that comprise the dataset represent?**
Each represents a question, a context (document passage from the CORD19 dataset) and an answer.

**How many instances are there in total?**

**What data does each instance consist of?**
Each instance is a query (natural language question and keyword-based), a set of answers, and a document id with its title associated with each answer.

[More Information Needed]

### Data Fields

The data was annotated in SQuAD style fashion, where each row contains:

* **question_query**: Natural language question query
* **keyword_query**: Keyword-based query
* **category_name**: Category in which the queries are part of
* **answers**: List of answers
  * **id**: The document ID the answer is found on
  * **title**: Title of the document of the answer
  * **exact_answer**: Text (string) of the exact answer

### Data Splits

**data/kaggle-lit-review-0.2.json**: 156 question-article pairs with 27 questions (topics) and 85 unique articles from
CORD-19.

[More Information Needed]

## Dataset Creation

The dataset aims to help for guiding research until more substantial evaluation resources become available. Being a smaller dataset,
it can be helpful for evaluating the zero-shot or transfer capabilities of existing models on topics specifically related to COVID-19.

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

#### Who are the source language producers?

[More Information Needed]

### Annotations

Five of the co-authors participated in this annotation effort, applying the aforementioned approach, with one lead 
annotator responsible for approving topics and answering technical questions from the other annotators. Two annotators are
undergraduate students majoring in computer science, one is a science alumna, another is a computer science professor, 
and the lead annotator is a graduate student in computer science—all affiliated with the University of Waterloo.

#### Annotation process

#### Who are the annotators?

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

The dataset was intended as a stopgap measure for guiding research until more substantial evaluation resources become available.

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

While this dataset, comprising 124 question–article pairs as of the present version 0.1 release, does not have sufficient
examples for supervised machine learning, it can be helpful for evaluating the zero-shot or transfer capabilities
of existing models on topics specifically related to COVID-19.

## Additional Information

The listed authors in the homepage are maintaining/supporting the dataset. 

### Dataset Curators

[More Information Needed]

### Licensing Information

The dataset is licensed under the [Apache License 2.0](https://github.com/castorini/pygaggle/blob/master/LICENSE).

### Citation Information

```
@article{tang2020rapidly,
  title={Rapidly Bootstrapping a Question Answering Dataset for COVID-19},
  author={Tang, Raphael and Nogueira, Rodrigo and Zhang, Edwin and Gupta, Nikhil and Cam, Phuong and Cho, Kyunghyun and Lin, Jimmy},
  journal={arXiv preprint arXiv:2004.11339},
  year={2020}
}
```

### Contributions

Thanks to [@olinguyen](https://github.com/olinguyen) for adding this dataset.
task_categories:question-answering task_ids:open-domain-qa task_ids:extractive-qa annotations_creators:expert-generated language_creators:found multilinguality:monolingual size_categories:1K<n<10K source_datasets:original language:en license:apache-2.0 arxiv:2004.11339 region:us
{"annotations_creators": ["expert-generated"], "language_creators": ["found"], "language": ["en"], "license": ["apache-2.0"], "multilinguality": ["monolingual"], "size_categories": ["1K<n<10K"], "source_datasets": ["original"], "task_categories": ["question-answering"], "task_ids": ["open-domain-qa", "extractive-qa"], "paperswithcode_id": "covidqa", "pretty_name": "CovidQaCastorini", "dataset_info": [{"config_name": "covid_qa_deepset", "features": [{"name": "document_id", "dtype": "int32"}, {"name": "context", "dtype": "string"}, {"name": "question", "dtype": "string"}, {"name": "is_impossible", "dtype": "bool"}, {"name": "id", "dtype": "int32"}, {"name": "answers", "sequence": [{"name": "text", "dtype": "string"}, {"name": "answer_start", "dtype": "int32"}]}], "splits": [{"name": "train", "num_bytes": 65151262, "num_examples": 2019}], "download_size": 4418117, "dataset_size": 65151262}, {"config_name": "covidqa", "features": [{"name": "category_name", "dtype": "string"}, {"name": "question_query", "dtype": "string"}, {"name": "keyword_query", "dtype": "string"}, {"name": "answers", "sequence": [{"name": "id", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "exact_answer", "dtype": "string"}]}], "splits": [{"name": "train", "num_bytes": 33757, "num_examples": 27}], "download_size": 51438, "dataset_size": 33757}, {"config_name": "covid_qa_castorini", "features": [{"name": "category_name", "dtype": "string"}, {"name": "question_query", "dtype": "string"}, {"name": "keyword_query", "dtype": "string"}, {"name": "answers", "sequence": [{"name": "id", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "exact_answer", "dtype": "string"}]}], "splits": [{"name": "train", "num_bytes": 33757, "num_examples": 27}], "download_size": 51438, "dataset_size": 33757}]}
TAGS
#task_categories-question-answering #task_ids-open-domain-qa #task_ids-extractive-qa #annotations_creators-expert-generated #language_creators-found #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #license-apache-2.0 #arxiv-2004.11339 #region-us 



# Dataset Card for [covid_qa_castorini]

## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions

## Dataset Description

- Homepage: URL
- Repository: URL
- Paper: URL
- Point of Contact: Castorini research group @UWaterloo

### Dataset Summary

CovidQA is a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered 
from Kaggle’s COVID-19 Open Research Dataset Challenge.
The dataset comprises 156 question-article pairs with 27 questions (topics) and 85 unique articles.

### Supported Tasks and Leaderboards



### Languages

The text in the dataset is in English.

## Dataset Structure

### Data Instances

What do the instances that comprise the dataset represent?
Each represents a question, a context (document passage from the CORD19 dataset) and an answer.

How many instances are there in total?

What data does each instance consist of?
Each instance is a query (natural language question and keyword-based), a set of answers, and a document id with its title associated with each answer.



### Data Fields

The data was annotated in SQuAD style fashion, where each row contains:

* question_query: Natural language question query
* keyword_query: Keyword-based query
* category_name: Category in which the queries are part of
* answers: List of answers
  * id: The document ID the answer is found on
  * title: Title of the document of the answer
  * exact_answer: Text (string) of the exact answer

### Data Splits

data/kaggle-lit-review-0.2.json: 156 question-article pairs with 27 questions (topics) and 85 unique articles from
CORD-19.



## Dataset Creation

The dataset aims to help for guiding research until more substantial evaluation resources become available. Being a smaller dataset,
it can be helpful for evaluating the zero-shot or transfer capabilities of existing models on topics specifically related to COVID-19.

### Curation Rationale



### Source Data

#### Initial Data Collection and Normalization

#### Who are the source language producers?



### Annotations

Five of the co-authors participated in this annotation effort, applying the aforementioned approach, with one lead 
annotator responsible for approving topics and answering technical questions from the other annotators. Two annotators are
undergraduate students majoring in computer science, one is a science alumna, another is a computer science professor, 
and the lead annotator is a graduate student in computer science—all affiliated with the University of Waterloo.

#### Annotation process

#### Who are the annotators?

### Personal and Sensitive Information



## Considerations for Using the Data

### Social Impact of Dataset

The dataset was intended as a stopgap measure for guiding research until more substantial evaluation resources become available.

### Discussion of Biases



### Other Known Limitations

While this dataset, comprising 124 question–article pairs as of the present version 0.1 release, does not have sufficient
examples for supervised machine learning, it can be helpful for evaluating the zero-shot or transfer capabilities
of existing models on topics specifically related to COVID-19.

## Additional Information

The listed authors in the homepage are maintaining/supporting the dataset. 

### Dataset Curators



### Licensing Information

The dataset is licensed under the Apache License 2.0.





### Contributions

Thanks to @olinguyen for adding this dataset.
# Dataset Card for [covid_qa_castorini] ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions ## Dataset Description

- Homepage: URL
- Repository: URL
- Paper: URL
- Point of Contact: Castorini research group @UWaterloo ### Dataset Summary

CovidQA is a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered 
from Kaggle’s COVID-19 Open Research Dataset Challenge.
The dataset comprises 156 question-article pairs with 27 questions (topics) and 85 unique articles. ### Supported Tasks and Leaderboards ### Languages

The text in the dataset is in English. ## Dataset Structure ### Data Instances

What do the instances that comprise the dataset represent?
Each represents a question, a context (document passage from the CORD19 dataset) and an answer.

How many instances are there in total?

What data does each instance consist of?
Each instance is a query (natural language question and keyword-based), a set of answers, and a document id with its title associated with each answer. ### Data Fields

The data was annotated in SQuAD style fashion, where each row contains:

* question_query: Natural language question query
* keyword_query: Keyword-based query
* category_name: Category in which the queries are part of
* answers: List of answers
  * id: The document ID the answer is found on
  * title: Title of the document of the answer
  * exact_answer: Text (string) of the exact answer ### Data Splits

data/kaggle-lit-review-0.2.json: 156 question-article pairs with 27 questions (topics) and 85 unique articles from
CORD-19. ## Dataset Creation

The dataset aims to help for guiding research until more substantial evaluation resources become available. Being a smaller dataset,
it can be helpful for evaluating the zero-shot or transfer capabilities of existing models on topics specifically related to COVID-19. ### Curation Rationale ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations

Five of the co-authors participated in this annotation effort, applying the aforementioned approach, with one lead 
annotator responsible for approving topics and answering technical questions from the other annotators. Two annotators are
undergraduate students majoring in computer science, one is a science alumna, another is a computer science professor, 
and the lead annotator is a graduate student in computer science—all affiliated with the University of Waterloo. #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset

The dataset was intended as a stopgap measure for guiding research until more substantial evaluation resources become available. ### Discussion of Biases ### Other Known Limitations

While this dataset, comprising 124 question–article pairs as of the present version 0.1 release, does not have sufficient
examples for supervised machine learning, it can be helpful for evaluating the zero-shot or transfer capabilities
of existing models on topics specifically related to COVID-19. ## Additional Information

The listed authors in the homepage are maintaining/supporting the dataset. ### Dataset Curators ### Licensing Information

The dataset is licensed under the Apache License 2.0. ### Contributions

Thanks to @olinguyen for adding this dataset.
TAGS
#task_categories-question-answering #task_ids-open-domain-qa #task_ids-extractive-qa #annotations_creators-expert-generated #language_creators-found #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #license-apache-2.0 #arxiv-2004.11339 #region-us 
 # Dataset Card for [covid_qa_castorini] ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions ## Dataset Description

- Homepage: URL
- Repository: URL
- Paper: URL
- Point of Contact: Castorini research group @UWaterloo ### Dataset Summary

CovidQA is a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered 
from Kaggle’s COVID-19 Open Research Dataset Challenge.
The dataset comprises 156 question-article pairs with 27 questions (topics) and 85 unique articles. ### Supported Tasks and Leaderboards ### Languages

The text in the dataset is in English. ## Dataset Structure ### Data Instances

What do the instances that comprise the dataset represent?
Each represents a question, a context (document passage from the CORD19 dataset) and an answer.

How many instances are there in total?

What data does each instance consist of?
Each instance is a query (natural language question and keyword-based), a set of answers, and a document id with its title associated with each answer. ### Data Fields

The data was annotated in SQuAD style fashion, where each row contains:

* question_query: Natural language question query
* keyword_query: Keyword-based query
* category_name: Category in which the queries are part of
* answers: List of answers
  * id: The document ID the answer is found on
  * title: Title of the document of the answer
  * exact_answer: Text (string) of the exact answer ### Data Splits

data/kaggle-lit-review-0.2.json: 156 question-article pairs with 27 questions (topics) and 85 unique articles from
CORD-19. ## Dataset Creation

The dataset aims to help for guiding research until more substantial evaluation resources become available. Being a smaller dataset,
it can be helpful for evaluating the zero-shot or transfer capabilities of existing models on topics specifically related to COVID-19. ### Curation Rationale ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations

Five of the co-authors participated in this annotation effort, applying the aforementioned approach, with one lead 
annotator responsible for approving topics and answering technical questions from the other annotators. Two annotators are
undergraduate students majoring in computer science, one is a science alumna, another is a computer science professor, 
and the lead annotator is a graduate student in computer science—all affiliated with the University of Waterloo. #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset

The dataset was intended as a stopgap measure for guiding research until more substantial evaluation resources become available. ### Discussion of Biases ### Other Known Limitations

While this dataset, comprising 124 question–article pairs as of the present version 0.1 release, does not have sufficient
examples for supervised machine learning, it can be helpful for evaluating the zero-shot or transfer capabilities
of existing models on topics specifically related to COVID-19. ## Additional Information

The listed authors in the homepage are maintaining/supporting the dataset. ### Dataset Curators ### Licensing Information

The dataset is licensed under the Apache License 2.0. ### Contributions

Thanks to @olinguyen for adding this dataset.