
ConcurrentQA is a textual multi-hop QA benchmark to require concurrent retrieval over multiple data-distributions (i.e. Wikipedia and email data). This dataset was constructed by researchers at Stanford and FAIR, following the data collection process and schema of HotpotQA.  This benchmark can be used to study generalization in retrieval as well as privacy when reasoning across multiple privacy scopes --- i.e. public Wikipedia documents and private emails.

This dataset is for the Question-Answering task. The dataset for the Retrieval task can be found here: https://huggingface.co/datasets/simarora/ConcurrentQA-Retrieval

The corpora of documents (Wikipedia and Emails) over which a system would need to retrieve information and answer questions can be downloaded using the following commands:
```
cd ..
mkdir corpora
cd corpora

wget https://dl.fbaipublicfiles.com/concurrentqa/corpora/enron_only_corpus.json
wget https://dl.fbaipublicfiles.com/concurrentqa/corpora/combined_corpus.json
wget https://dl.fbaipublicfiles.com/concurrentqa/corpora/wiki_only_corpus.json
wget https://dl.fbaipublicfiles.com/concurrentqa/corpora/title2sent_map.json
```

The repo https://github.com/facebookresearch/concurrentqa contains model training and result analysis code.

If you find this resource useful, consider citing the paper:
```
@article{arora2023reasoning,
         title={Reasoning over Public and Private Data in Retrieval-Based Systems}, 
         author={Simran Arora and Patrick Lewis and Angela Fan and Jacob Kahn and Christopher Ré},
         year={2023},
	 url={https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00556/116046/Aggretriever-A-Simple-Approach-to-Aggregate},
         journal={Transactions of the Association for Computational Linguistics},
}
```

Please reach out at ```simran@cs.stanford.edu``` with questions or feedback!
task_categories:question-answering language:en license:mit region:us
{"language": ["en"], "license": "mit", "task_categories": ["question-answering"]}
TAGS
#task_categories-question-answering #language-English #license-mit #region-us 


ConcurrentQA is a textual multi-hop QA benchmark to require concurrent retrieval over multiple data-distributions (i.e. Wikipedia and email data). This dataset was constructed by researchers at Stanford and FAIR, following the data collection process and schema of HotpotQA.  This benchmark can be used to study generalization in retrieval as well as privacy when reasoning across multiple privacy scopes --- i.e. public Wikipedia documents and private emails.

This dataset is for the Question-Answering task. The dataset for the Retrieval task can be found here: URL

The corpora of documents (Wikipedia and Emails) over which a system would need to retrieve information and answer questions can be downloaded using the following commands:


The repo URL contains model training and result analysis code.

If you find this resource useful, consider citing the paper:


Please reach out at  with questions or feedback!
TAGS
#task_categories-question-answering #language-English #license-mit #region-us 
