arxiv:2003.13624 region:us
# TREC Conversational Assistance Track (CAsT) 

There are currently few datasets appropriate for training and evaluating models for Conversational Information Seeking (CIS). The main aim of TREC CAsT is to advance research on conversational search systems. The goal of the track is to create a reusable benchmark for open-domain information centric conversational dialogues. 

# Year 1 (TREC 2019)
* Read the [TREC 2019 Overview](https://arxiv.org/abs/2003.13624) paper. 

## 2019 Data
### Topics
 * [Training topics] - 30 example training topics
 * [Training judgments] - The judgments are graded on a three point scale (2 very relevant, 1 relevant, and 0 not relevant). 
 * [Evaluation topics]- 50 evaluation topics

 ### Sample of Dataset 
 * Number: 1
 * Title: US Judicial history
 * Description: Judicial history in the US including key court cases and what they established.
 * Prompts:
  1. What are the most important US Supreme Court cases?
  2. What did plessy v. ferguson establish?
  3. How about marbury vs madison?
  4. Was it unanimous?
  5. What was the implication of roe vs wade?
  6. What were the main arguments?
  7. What was the point of the brown v board of education?
  8. What were the main arguments?
  9. Why is it important today?
      

### Collection
 * The corpus is a combination of three standard TREC collections: MARCO Ranking passages, Wikipedia (TREC CAR), and News (Washington Post)
 * The [MS MARCO Passage Ranking collection](https://msmarco.blob.core.windows.net/msmarcoranking/collection.tar.gz) - This file only includes the passage id and passage text.  For convenience, we also provide a passage id -> URL mapping file in TSV format [pid to URL file](http://boston.lti.cs.cmu.edu/vaibhav2/cast/marco_pas_url.tsv). 
 * The [TREC CAR paragraph collection v2.0](http://trec-car.cs.unh.edu/datareleases/v2.0/paragraphCorpus.v2.0.tar.xz)
 * The [TREC Washington Post Corpus version 2](https://ir.nist.gov/wapo/WashingtonPost.v2.tar.gz): Note this is behind a password and requires an organizational agreement, to obtain it see: https://ir.nist.gov/wapo/


### Document ID format
 * The document id format is `[collection_id_paragraph_id]` with collection id and paragraph id separated by an underscore.
 * The collection ids are in the set: `{MARCO, CAR, WAPO}`. 
 * The paragraph ids are: standard provided by MARCO and CAR. For WAPO the paragraph ID is `[article_id-paragraph_index]` where the paragraph_index is the *starting from 1-based* index of the paragraph using the provided paragraph markup separated by a single dash. 
 * Example WaPo combined document id: `[WAPO_903cc1eab726b829294d1abdd755d5ab-1]`, or CAR: `[CAR_6869dee46ab12f0f7060874f7fc7b1c57d53144a]`
 
 
## Code and tools
* [TREC-CAsT Tools](https://github.com/gla-ial/trec-cast-tools) repository with code and scripts for processing data. 
* The tools contain scripts for parsing the collection into standard indexing formats. It also provides APIs for working with the topics (in text, json, and protocol buffer formats).

