
# Dataset Card for Taskmaster-2

- **Repository:** https://github.com/google-research-datasets/Taskmaster/tree/master/TM-2-2020
- **Paper:** https://arxiv.org/pdf/1909.05358.pdf
- **Leaderboard:** None
- **Who transforms the dataset:** Qi Zhu(zhuq96 at gmail dot com)

To use this dataset, you need to install [ConvLab-3](https://github.com/ConvLab/ConvLab-3) platform first. Then you can load the dataset via:
```
from convlab.util import load_dataset, load_ontology, load_database

dataset = load_dataset('tm2')
ontology = load_ontology('tm2')
database = load_database('tm2')
```
For more usage please refer to [here](https://github.com/ConvLab/ConvLab-3/tree/master/data/unified_datasets).

### Dataset Summary

The Taskmaster-2 dataset consists of 17,289 dialogs in the seven domains. Unlike Taskmaster-1, which includes both written "self-dialogs" and spoken two-person dialogs, Taskmaster-2 consists entirely of spoken two-person dialogs. In addition, while Taskmaster-1 is almost exclusively task-based, Taskmaster-2 contains a good number of search- and recommendation-oriented dialogs, as seen for example in the restaurants, flights, hotels, and movies verticals. The music browsing and sports conversations are almost exclusively search- and recommendation-based. All dialogs in this release were created using a Wizard of Oz (WOz) methodology in which crowdsourced workers played the role of a 'user' and trained call center operators played the role of the 'assistant'. In this way, users were led to believe they were interacting with an automated system that “spoke” using text-to-speech (TTS) even though it was in fact a human behind the scenes. As a result, users could express themselves however they chose in the context of an automated interface.

- **How to get the transformed data from original data:** 
  - Download [master.zip](https://github.com/google-research-datasets/Taskmaster/archive/refs/heads/master.zip).
  - Run `python preprocess.py` in the current directory.
- **Main changes of the transformation:**
  - Remove dialogs that are empty or only contain one speaker.
  - Split each domain dialogs into train/validation/test randomly (8:1:1).
  - Merge continuous turns by the same speaker (ignore repeated turns).
  - Annotate `dialogue acts` according to the original segment annotations. Add `intent` annotation (`==inform`). The type of `dialogue act` is set to `non-categorical` if the `slot` is not in `anno2slot` in `preprocess.py`). Otherwise, the type is set to `binary` (and the `value` is empty). If there are multiple spans overlapping, we only keep the shortest one, since we found that this simple strategy can reduce the noise in annotation.
  - Add `domain`, `intent`, and `slot` descriptions.
  - Add `state` by accumulate `non-categorical dialogue acts` in the order that they appear.
  - Keep the first annotation since each conversation was annotated by two workers.
- **Annotations:**
  - dialogue acts, state.

### Supported Tasks and Leaderboards

NLU, DST, Policy, NLG

### Languages

English

### Data Splits

| split      |   dialogues |   utterances |   avg_utt |   avg_tokens |   avg_domains | cat slot match(state)   | cat slot match(goal)   | cat slot match(dialogue act)   |   non-cat slot span(dialogue act) |
|------------|-------------|--------------|-----------|--------------|---------------|-------------------------|------------------------|--------------------------------|-----------------------------------|
| train      |       13838 |       234321 |     16.93 |         9.1  |             1 | -                       | -                      | -                              |                               100 |
| validation |        1731 |        29349 |     16.95 |         9.15 |             1 | -                       | -                      | -                              |                               100 |
| test       |        1734 |        29447 |     16.98 |         9.07 |             1 | -                       | -                      | -                              |                               100 |
| all        |       17303 |       293117 |     16.94 |         9.1  |             1 | -                       | -                      | -                              |                               100 |

7 domains: ['flights', 'food-ordering', 'hotels', 'movies', 'music', 'restaurant-search', 'sports']
- **cat slot match**: how many values of categorical slots are in the possible values of ontology in percentage.
- **non-cat slot span**: how many values of non-categorical slots have span annotation in percentage.

### Citation

```
@inproceedings{byrne-etal-2019-taskmaster,
  title = {Taskmaster-1:Toward a Realistic and Diverse Dialog Dataset},
  author = {Bill Byrne and Karthik Krishnamoorthi and Chinnadhurai Sankar and Arvind Neelakantan and Daniel Duckworth and Semih Yavuz and Ben Goodrich and Amit Dubey and Kyu-Young Kim and Andy Cedilnik},
  booktitle = {2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing},
  address = {Hong Kong}, 
  year = {2019} 
}
```

### Licensing Information

[**CC BY 4.0**](https://creativecommons.org/licenses/by/4.0/)
task_categories:conversational multilinguality:monolingual size_categories:10K<n<100K language:en license:cc-by-4.0 arxiv:1909.05358 region:us
{"language": ["en"], "license": ["cc-by-4.0"], "multilinguality": ["monolingual"], "size_categories": ["10K<n<100K"], "task_categories": ["conversational"], "pretty_name": "Taskmaster-2"}
TAGS
#task_categories-conversational #multilinguality-monolingual #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #arxiv-1909.05358 #region-us 

Dataset Card for Taskmaster-2
=============================


* Repository: URL
* Paper: URL
* Leaderboard: None
* Who transforms the dataset: Qi Zhu(zhuq96 at gmail dot com)


To use this dataset, you need to install ConvLab-3 platform first. Then you can load the dataset via:


For more usage please refer to here.


### Dataset Summary


The Taskmaster-2 dataset consists of 17,289 dialogs in the seven domains. Unlike Taskmaster-1, which includes both written "self-dialogs" and spoken two-person dialogs, Taskmaster-2 consists entirely of spoken two-person dialogs. In addition, while Taskmaster-1 is almost exclusively task-based, Taskmaster-2 contains a good number of search- and recommendation-oriented dialogs, as seen for example in the restaurants, flights, hotels, and movies verticals. The music browsing and sports conversations are almost exclusively search- and recommendation-based. All dialogs in this release were created using a Wizard of Oz (WOz) methodology in which crowdsourced workers played the role of a 'user' and trained call center operators played the role of the 'assistant'. In this way, users were led to believe they were interacting with an automated system that “spoke” using text-to-speech (TTS) even though it was in fact a human behind the scenes. As a result, users could express themselves however they chose in the context of an automated interface.


* How to get the transformed data from original data:
	+ Download URL.
	+ Run 'python URL' in the current directory.
* Main changes of the transformation:
	+ Remove dialogs that are empty or only contain one speaker.
	+ Split each domain dialogs into train/validation/test randomly (8:1:1).
	+ Merge continuous turns by the same speaker (ignore repeated turns).
	+ Annotate 'dialogue acts' according to the original segment annotations. Add 'intent' annotation ('==inform'). The type of 'dialogue act' is set to 'non-categorical' if the 'slot' is not in 'anno2slot' in 'URL'). Otherwise, the type is set to 'binary' (and the 'value' is empty). If there are multiple spans overlapping, we only keep the shortest one, since we found that this simple strategy can reduce the noise in annotation.
	+ Add 'domain', 'intent', and 'slot' descriptions.
	+ Add 'state' by accumulate 'non-categorical dialogue acts' in the order that they appear.
	+ Keep the first annotation since each conversation was annotated by two workers.
* Annotations:
	+ dialogue acts, state.


### Supported Tasks and Leaderboards


NLU, DST, Policy, NLG


### Languages


English


### Data Splits



7 domains: ['flights', 'food-ordering', 'hotels', 'movies', 'music', 'restaurant-search', 'sports']


* cat slot match: how many values of categorical slots are in the possible values of ontology in percentage.
* non-cat slot span: how many values of non-categorical slots have span annotation in percentage.


### Licensing Information


CC BY 4.0



### Dataset Summary


The Taskmaster-2 dataset consists of 17,289 dialogs in the seven domains. Unlike Taskmaster-1, which includes both written "self-dialogs" and spoken two-person dialogs, Taskmaster-2 consists entirely of spoken two-person dialogs. In addition, while Taskmaster-1 is almost exclusively task-based, Taskmaster-2 contains a good number of search- and recommendation-oriented dialogs, as seen for example in the restaurants, flights, hotels, and movies verticals. The music browsing and sports conversations are almost exclusively search- and recommendation-based. All dialogs in this release were created using a Wizard of Oz (WOz) methodology in which crowdsourced workers played the role of a 'user' and trained call center operators played the role of the 'assistant'. In this way, users were led to believe they were interacting with an automated system that “spoke” using text-to-speech (TTS) even though it was in fact a human behind the scenes. As a result, users could express themselves however they chose in the context of an automated interface.


* How to get the transformed data from original data:
	+ Download URL.
	+ Run 'python URL' in the current directory.
* Main changes of the transformation:
	+ Remove dialogs that are empty or only contain one speaker.
	+ Split each domain dialogs into train/validation/test randomly (8:1:1).
	+ Merge continuous turns by the same speaker (ignore repeated turns).
	+ Annotate 'dialogue acts' according to the original segment annotations. Add 'intent' annotation ('==inform'). The type of 'dialogue act' is set to 'non-categorical' if the 'slot' is not in 'anno2slot' in 'URL'). Otherwise, the type is set to 'binary' (and the 'value' is empty). If there are multiple spans overlapping, we only keep the shortest one, since we found that this simple strategy can reduce the noise in annotation.
	+ Add 'domain', 'intent', and 'slot' descriptions.
	+ Add 'state' by accumulate 'non-categorical dialogue acts' in the order that they appear.
	+ Keep the first annotation since each conversation was annotated by two workers.
* Annotations:
	+ dialogue acts, state. ### Supported Tasks and Leaderboards


NLU, DST, Policy, NLG ### Languages


English ### Data Splits



7 domains: ['flights', 'food-ordering', 'hotels', 'movies', 'music', 'restaurant-search', 'sports']


* cat slot match: how many values of categorical slots are in the possible values of ontology in percentage.
* non-cat slot span: how many values of non-categorical slots have span annotation in percentage. ### Licensing Information


CC BY 4.0
TAGS
#task_categories-conversational #multilinguality-monolingual #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #arxiv-1909.05358 #region-us 
 ### Dataset Summary


The Taskmaster-2 dataset consists of 17,289 dialogs in the seven domains. Unlike Taskmaster-1, which includes both written "self-dialogs" and spoken two-person dialogs, Taskmaster-2 consists entirely of spoken two-person dialogs. In addition, while Taskmaster-1 is almost exclusively task-based, Taskmaster-2 contains a good number of search- and recommendation-oriented dialogs, as seen for example in the restaurants, flights, hotels, and movies verticals. The music browsing and sports conversations are almost exclusively search- and recommendation-based. All dialogs in this release were created using a Wizard of Oz (WOz) methodology in which crowdsourced workers played the role of a 'user' and trained call center operators played the role of the 'assistant'. In this way, users were led to believe they were interacting with an automated system that “spoke” using text-to-speech (TTS) even though it was in fact a human behind the scenes. As a result, users could express themselves however they chose in the context of an automated interface.


* How to get the transformed data from original data:
	+ Download URL.
	+ Run 'python URL' in the current directory.
* Main changes of the transformation:
	+ Remove dialogs that are empty or only contain one speaker.
	+ Split each domain dialogs into train/validation/test randomly (8:1:1).
	+ Merge continuous turns by the same speaker (ignore repeated turns).
	+ Annotate 'dialogue acts' according to the original segment annotations. Add 'intent' annotation ('==inform'). The type of 'dialogue act' is set to 'non-categorical' if the 'slot' is not in 'anno2slot' in 'URL'). Otherwise, the type is set to 'binary' (and the 'value' is empty). If there are multiple spans overlapping, we only keep the shortest one, since we found that this simple strategy can reduce the noise in annotation.
	+ Add 'domain', 'intent', and 'slot' descriptions.
	+ Add 'state' by accumulate 'non-categorical dialogue acts' in the order that they appear.
	+ Keep the first annotation since each conversation was annotated by two workers.
* Annotations:
	+ dialogue acts, state. ### Supported Tasks and Leaderboards


NLU, DST, Policy, NLG ### Languages


English ### Data Splits



7 domains: ['flights', 'food-ordering', 'hotels', 'movies', 'music', 'restaurant-search', 'sports']


* cat slot match: how many values of categorical slots are in the possible values of ontology in percentage.
* non-cat slot span: how many values of non-categorical slots have span annotation in percentage. ### Licensing Information


CC BY 4.0
passage: TAGS
#task_categories-conversational #multilinguality-monolingual #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #arxiv-1909.05358 #region-us 
