
# Dataset Card for MedDialog

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Repository:** https://github.com/UCSD-AI4H/Medical-Dialogue-System
- **Paper:** [MedDialog: Two Large-scale Medical Dialogue Datasets](https://arxiv.org/abs/2004.03329)
- **Point of Contact:** [Pengtao Xie](mailto:pengtaoxie2008@gmail.com)

### Dataset Summary

The MedDialog dataset (Chinese) contains conversations (in Chinese) between doctors and patients. It has 1.1 million dialogues and 4 million utterances. The data is continuously growing and more dialogues will be added. The raw dialogues are from haodf.com. All copyrights of the data belong to haodf.com.

The MedDialog dataset (English) contains conversations (in English) between doctors and patients. It has 0.26 million dialogues. The data is continuously growing and more dialogues will be added. The raw dialogues are from healthcaremagic.com and icliniq.com. All copyrights of the data belong to healthcaremagic.com and icliniq.com.

Directions for using the pre-trained model using BERT using PyTorch is available in the Homepage.


### Supported Tasks and Leaderboards

Closed domain qa

### Languages

Monolingual. The datasets are in English (EN) and Chinese (ZH)

## Dataset Structure

### Data Instances

There are 4 configurations:
- Raw data:
  - en
  - zh
- Processed data:
  - processed.en
  - processed.zh

#### en

Each consultation consists of the below:
- ID
- URL
- Description of patient’s medical condition
- Dialogue

The dataset is built from [icliniq.com](https://www.icliniq.com/), [healthcaremagic.com](https://www.healthcaremagic.com/), [healthtap.com](https://www.healthtap.com/) and all copyrights of the data belong to these websites.

#### zh

Each consultation consists of the below:
- ID
- URL
- Description of patient’s medical condition
- Dialogue
- (Optional) Diagnosis and suggestions.

The dataset is built from [Haodf.com](https://www.haodf.com/) and all copyrights of the data belong to [Haodf.com](https://www.haodf.com/).

One example for chinese is

```
{
{'dialogue_id': 2,
  'dialogue_turns': [{'speaker': '病人',
    'utterance': '孩子哭闹时，鸡鸡旁边会肿起，情绪平静时肿块会消失，去一个私人诊所看过，说是疝气.如果确定是疝气，是不是一定要手术治疗？我孩子只有1岁10月，自愈的可能性大吗？如果一定要手术，这么小的孩子风险大吗？术后的恢复困难吗？谢谢.'},
   {'speaker': '医生', 'utterance': '南方医的B超说得不清楚，可能是鞘膜积液，可到我医院复查一个B超。'}],
  'dialogue_url': 'https://www.haodf.com/doctorteam/flow_team_6477251152.htm',
  'file_name': '2020.txt'},
}
```

#### processed.en
```
{
  'description': 'throat a bit sore and want to get a good imune booster, especially in light of the virus. please advise. have not been in contact with nyone with the virus.',
  'utterances': [
    'patient: throat a bit sore and want to get a good imune booster, especially in light of the virus. please advise. have not been in contact with nyone with the virus.',
    "doctor: during this pandemic. throat pain can be from a strep throat infection (antibiotics needed), a cold or influenza or other virus, or from some other cause such as allergies or irritants. usually, a person sees the doctor (call first) if the sore throat is bothersome, recurrent, or doesn't go away quickly. covid-19 infections tend to have cough, whereas strep throat usually lacks cough but has more throat pain. (3/21/20)"
  ]
}
```

#### processed.zh
```
{
  'utterances': [
    '病人：强制性脊柱炎，晚上睡觉翻身时腰骶骨区域疼痛，其他身体任何部位均不疼痛。', 
    '医生：应该没有问题，但最好把图像上传看看。'
  ]
}
```

### Data Fields

For generating the QA only the below fields have been considered:
- ID : Consultatation Identifier (restarts for each file)
- URL: The url link of the extracted conversation
- Dialogue : The conversation between the doctor and the patient.

These are arranged as below in the prepared dataset. Each item will be represented with these parameters.

- "file_name": string - signifies the file from which the conversation was extracted
- "dialogue_id": int32 - the dialogue id
- "dialogue_url": string - url of the conversation
- "dialogue_turns": datasets.Sequence - sequence of dialogues between patient and the doctor.Consists ClassLabel(names=["病人", "医生"]), and "utterance"(string) for each turn. (ClassLable(names=["Patient", "Doctor"]) for english)

#### processed.en
- `description` (str): Description of the dialog.
- `utterances` (list of str): Dialog utterances between patient and doctor.

#### processed.zh
- `utterances` (list of str): Dialog utterances between patient and doctor.

### Data Splits

There are no data splits on the original raw data. The "train" split for each language contains:
- en: 229674 examples
- zh: 1921127 examples

For processed configurations, data is split into train, validation and test, with the following number of examples:

|              |   train | validation |   test |
|--------------|--------:|-----------:|-------:|
| processed.en |     482 |         60 |     61 |
| processed.zh | 2725989 |     340748 | 340754 |

## Dataset Creation

### Curation Rationale

Medical dialogue systems are promising in assisting in telemedicine to increase access to healthcare services, improve the quality of patient care, and reduce medical costs. 

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

The authors claim that:
- They scraped the data from the following websites:
  - MedDialog-EN: data was crawled from https://www.icliniq.com/ and https://www.healthcaremagic.com/
  - MedDialog-CN: data was crawled from https://www.haodf.com/
- All copyrights of the data belong to the corresponding websites

The [terms and conditions](https://www.icliniq.com/p/terms) (last updated on: 11th April 2022) of www.icliniq.com website state:
> No person (including a User, Doctor, Alternative Medicine Practitioner, or Wellness Professional) shall copy, transfer, download, republish, sell, duplicate, or "scrape", for commercial or any other purpose whatsoever, the contents or information made available on the Platform including Directory Listing Services, academic articles, and queries, in whole or in part, in any medium whatsoever.

The [terms and conditions](https://www.healthcaremagic.com/tc) (last updated: August 17, 2012) of www.healthcaremagic.com website stipulate:
> You are prohibited from republishing, selling, duplicating, or "scraping" for commercial or any other purpose whatsoever any of the data or other information contained therein, in whole or in part, in any medium whatsoever.


### Citation Information
```
@article{chen2020meddiag,
  title={MedDialog: a large-scale medical dialogue dataset},
  author={Chen, Shu and Ju, Zeqian and Dong, Xiangyu and Fang, Hongchao and Wang, Sicheng and Yang, Yue and Zeng, Jiaqi and Zhang, Ruisi and Zhang, Ruoyu and Zhou, Meng and Zhu, Penghui and Xie, Pengtao},
  journal={arXiv preprint arXiv:2004.03329}, 
  year={2020}
}
```

### Contributions

Thanks to [@vrindaprabhu](https://github.com/vrindaprabhu) for adding this dataset.
task_categories:question-answering task_ids:closed-domain-qa annotations_creators:found language_creators:expert-generated language_creators:found multilinguality:monolingual size_categories:1M<n<10M source_datasets:original language:en language:zh license:unknown arxiv:2004.03329 region:us
{"annotations_creators": ["found"], "language_creators": ["expert-generated", "found"], "language": ["en", "zh"], "license": ["unknown"], "multilinguality": ["monolingual"], "size_categories": ["1M<n<10M"], "source_datasets": ["original"], "task_categories": ["question-answering"], "task_ids": ["closed-domain-qa"], "paperswithcode_id": "meddialog", "pretty_name": "MedDialog", "config_names": ["en", "zh"], "dataset_info": [{"config_name": "en", "features": [{"name": "file_name", "dtype": "string"}, {"name": "dialogue_id", "dtype": "int32"}, {"name": "dialogue_url", "dtype": "string"}, {"name": "dialogue_turns", "sequence": [{"name": "speaker", "dtype": {"class_label": {"names": {"0": "Patient", "1": "Doctor"}}}}, {"name": "utterance", "dtype": "string"}]}], "splits": [{"name": "train", "num_bytes": 290274759, "num_examples": 229674}], "download_size": 0, "dataset_size": 290274759}, {"config_name": "zh", "features": [{"name": "file_name", "dtype": "string"}, {"name": "dialogue_id", "dtype": "int32"}, {"name": "dialogue_url", "dtype": "string"}, {"name": "dialogue_turns", "sequence": [{"name": "speaker", "dtype": {"class_label": {"names": {"0": "\u75c5\u4eba", "1": "\u533b\u751f"}}}}, {"name": "utterance", "dtype": "string"}]}], "splits": [{"name": "train", "num_bytes": 1092063621, "num_examples": 1921127}], "download_size": 0, "dataset_size": 1092063621}, {"config_name": "processed.en", "features": [{"name": "description", "dtype": "string"}, {"name": "utterances", "sequence": "string"}], "splits": [{"name": "train", "num_bytes": 370745, "num_examples": 482}, {"name": "validation", "num_bytes": 52145, "num_examples": 60}, {"name": "test", "num_bytes": 46514, "num_examples": 61}], "download_size": 524214, "dataset_size": 469404}, {"config_name": "processed.zh", "features": [{"name": "utterances", "sequence": "string"}], "splits": [{"name": "train", "num_bytes": 1571262099, "num_examples": 2725989}, {"name": "validation", "num_bytes": 197117565, "num_examples": 340748}, {"name": "test", "num_bytes": 196526738, "num_examples": 340754}], "download_size": 2082354155, "dataset_size": 1964906402}], "viewer": false}
TAGS
#task_categories-question-answering #task_ids-closed-domain-qa #annotations_creators-found #language_creators-expert-generated #language_creators-found #multilinguality-monolingual #size_categories-1M<n<10M #source_datasets-original #language-English #language-Chinese #license-unknown #arxiv-2004.03329 #region-us 

Dataset Card for MedDialog
==========================


Table of Contents
-----------------


* Dataset Description
	+ Dataset Summary
	+ Supported Tasks and Leaderboards
	+ Languages
* Dataset Structure
	+ Data Instances
	+ Data Fields
	+ Data Splits
* Dataset Creation
	+ Curation Rationale
	+ Source Data
	+ Annotations
	+ Personal and Sensitive Information
* Considerations for Using the Data
	+ Social Impact of Dataset
	+ Discussion of Biases
	+ Other Known Limitations
* Additional Information
	+ Dataset Curators
	+ Licensing Information
	+ Citation Information
	+ Contributions


Dataset Description
-------------------


* Repository: URL
* Paper: MedDialog: Two Large-scale Medical Dialogue Datasets
* Point of Contact: Pengtao Xie


### Dataset Summary


The MedDialog dataset (Chinese) contains conversations (in Chinese) between doctors and patients. It has 1.1 million dialogues and 4 million utterances. The data is continuously growing and more dialogues will be added. The raw dialogues are from URL. All copyrights of the data belong to URL.


The MedDialog dataset (English) contains conversations (in English) between doctors and patients. It has 0.26 million dialogues. The data is continuously growing and more dialogues will be added. The raw dialogues are from URL and URL. All copyrights of the data belong to URL and URL.


Directions for using the pre-trained model using BERT using PyTorch is available in the Homepage.


### Supported Tasks and Leaderboards


Closed domain qa


### Languages


Monolingual. The datasets are in English (EN) and Chinese (ZH)


Dataset Structure
-----------------


### Data Instances


There are 4 configurations:


* Raw data:
	+ en
	+ zh
* Processed data:
	+ URL
	+ URL


#### en


Each consultation consists of the below:


* ID
* URL
* Description of patient’s medical condition
* Dialogue


The dataset is built from URL, URL, URL and all copyrights of the data belong to these websites.


#### zh


Each consultation consists of the below:


* ID
* URL
* Description of patient’s medical condition
* Dialogue
* (Optional) Diagnosis and suggestions.


The dataset is built from URL and all copyrights of the data belong to URL.


One example for chinese is


#### URL


#### URL


### Data Fields


For generating the QA only the below fields have been considered:


* ID : Consultatation Identifier (restarts for each file)
* URL: The url link of the extracted conversation
* Dialogue : The conversation between the doctor and the patient.


These are arranged as below in the prepared dataset. Each item will be represented with these parameters.


* "file\_name": string - signifies the file from which the conversation was extracted
* "dialogue\_id": int32 - the dialogue id
* "dialogue\_url": string - url of the conversation
* "dialogue\_turns": datasets.Sequence - sequence of dialogues between patient and the doctor.Consists ClassLabel(names=["病人", "医生"]), and "utterance"(string) for each turn. (ClassLable(names=["Patient", "Doctor"]) for english)


#### URL


* 'description' (str): Description of the dialog.
* 'utterances' (list of str): Dialog utterances between patient and doctor.


#### URL


* 'utterances' (list of str): Dialog utterances between patient and doctor.


### Data Splits


There are no data splits on the original raw data. The "train" split for each language contains:


* en: 229674 examples
* zh: 1921127 examples


For processed configurations, data is split into train, validation and test, with the following number of examples:



Dataset Creation
----------------


### Curation Rationale


Medical dialogue systems are promising in assisting in telemedicine to increase access to healthcare services, improve the quality of patient care, and reduce medical costs.


### Source Data


#### Initial Data Collection and Normalization


#### Who are the source language producers?


### Annotations


#### Annotation process


#### Who are the annotators?


### Personal and Sensitive Information


Considerations for Using the Data
---------------------------------


### Social Impact of Dataset


### Discussion of Biases


### Other Known Limitations


Additional Information
----------------------


### Dataset Curators


### Licensing Information


The authors claim that:


* They scraped the data from the following websites:
	+ MedDialog-EN: data was crawled from URL and URL
	+ MedDialog-CN: data was crawled from URL
* All copyrights of the data belong to the corresponding websites


The terms and conditions (last updated on: 11th April 2022) of URL website state:



> 
> No person (including a User, Doctor, Alternative Medicine Practitioner, or Wellness Professional) shall copy, transfer, download, republish, sell, duplicate, or "scrape", for commercial or any other purpose whatsoever, the contents or information made available on the Platform including Directory Listing Services, academic articles, and queries, in whole or in part, in any medium whatsoever.
> 
> 
> 


The terms and conditions (last updated: August 17, 2012) of URL website stipulate:



> 
> You are prohibited from republishing, selling, duplicating, or "scraping" for commercial or any other purpose whatsoever any of the data or other information contained therein, in whole or in part, in any medium whatsoever.
> 
> 
> 


### Contributions


Thanks to @vrindaprabhu for adding this dataset.



### Dataset Summary


The MedDialog dataset (Chinese) contains conversations (in Chinese) between doctors and patients. It has 1.1 million dialogues and 4 million utterances. The data is continuously growing and more dialogues will be added. The raw dialogues are from URL. All copyrights of the data belong to URL.


The MedDialog dataset (English) contains conversations (in English) between doctors and patients. It has 0.26 million dialogues. The data is continuously growing and more dialogues will be added. The raw dialogues are from URL and URL. All copyrights of the data belong to URL and URL.


Directions for using the pre-trained model using BERT using PyTorch is available in the Homepage. ### Supported Tasks and Leaderboards


Closed domain qa ### Languages


Monolingual. The datasets are in English (EN) and Chinese (ZH)


Dataset Structure
----------------- ### Data Instances


There are 4 configurations:


* Raw data:
	+ en
	+ zh
* Processed data:
	+ URL
	+ URL #### en


Each consultation consists of the below:


* ID
* URL
* Description of patient’s medical condition
* Dialogue


The dataset is built from URL, URL, URL and all copyrights of the data belong to these websites. #### zh


Each consultation consists of the below:


* ID
* URL
* Description of patient’s medical condition
* Dialogue
* (Optional) Diagnosis and suggestions.


The dataset is built from URL and all copyrights of the data belong to URL.


One example for chinese is #### URL #### URL ### Data Fields


For generating the QA only the below fields have been considered:


* ID : Consultatation Identifier (restarts for each file)
* URL: The url link of the extracted conversation
* Dialogue : The conversation between the doctor and the patient.


These are arranged as below in the prepared dataset. Each item will be represented with these parameters.


* "file\_name": string - signifies the file from which the conversation was extracted
* "dialogue\_id": int32 - the dialogue id
* "dialogue\_url": string - url of the conversation
* "dialogue\_turns": datasets.Sequence - sequence of dialogues between patient and the doctor.Consists ClassLabel(names=["病人", "医生"]), and "utterance"(string) for each turn. (ClassLable(names=["Patient", "Doctor"]) for english) #### URL


* 'description' (str): Description of the dialog.
* 'utterances' (list of str): Dialog utterances between patient and doctor. #### URL


* 'utterances' (list of str): Dialog utterances between patient and doctor. ### Data Splits


There are no data splits on the original raw data. The "train" split for each language contains:


* en: 229674 examples
* zh: 1921127 examples


For processed configurations, data is split into train, validation and test, with the following number of examples:



Dataset Creation
---------------- ### Curation Rationale


Medical dialogue systems are promising in assisting in telemedicine to increase access to healthcare services, improve the quality of patient care, and reduce medical costs. ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators ### Licensing Information


The authors claim that:


* They scraped the data from the following websites:
	+ MedDialog-EN: data was crawled from URL and URL
	+ MedDialog-CN: data was crawled from URL
* All copyrights of the data belong to the corresponding websites


The terms and conditions (last updated on: 11th April 2022) of URL website state:



> 
> No person (including a User, Doctor, Alternative Medicine Practitioner, or Wellness Professional) shall copy, transfer, download, republish, sell, duplicate, or "scrape", for commercial or any other purpose whatsoever, the contents or information made available on the Platform including Directory Listing Services, academic articles, and queries, in whole or in part, in any medium whatsoever.
> 
> 
> 


The terms and conditions (last updated: August 17, 2012) of URL website stipulate:



> 
> You are prohibited from republishing, selling, duplicating, or "scraping" for commercial or any other purpose whatsoever any of the data or other information contained therein, in whole or in part, in any medium whatsoever.
> 
> 
> ### Contributions


Thanks to @vrindaprabhu for adding this dataset.
TAGS
#task_categories-question-answering #task_ids-closed-domain-qa #annotations_creators-found #language_creators-expert-generated #language_creators-found #multilinguality-monolingual #size_categories-1M<n<10M #source_datasets-original #language-English #language-Chinese #license-unknown #arxiv-2004.03329 #region-us 
 ### Dataset Summary


The MedDialog dataset (Chinese) contains conversations (in Chinese) between doctors and patients. It has 1.1 million dialogues and 4 million utterances. The data is continuously growing and more dialogues will be added. The raw dialogues are from URL. All copyrights of the data belong to URL.


The MedDialog dataset (English) contains conversations (in English) between doctors and patients. It has 0.26 million dialogues. The data is continuously growing and more dialogues will be added. The raw dialogues are from URL and URL. All copyrights of the data belong to URL and URL.


Directions for using the pre-trained model using BERT using PyTorch is available in the Homepage. ### Supported Tasks and Leaderboards


Closed domain qa ### Languages


Monolingual. The datasets are in English (EN) and Chinese (ZH)


Dataset Structure
----------------- ### Data Instances


There are 4 configurations:


* Raw data:
	+ en
	+ zh
* Processed data:
	+ URL
	+ URL #### en


Each consultation consists of the below:


* ID
* URL
* Description of patient’s medical condition
* Dialogue


The dataset is built from URL, URL, URL and all copyrights of the data belong to these websites. #### zh


Each consultation consists of the below:


* ID
* URL
* Description of patient’s medical condition
* Dialogue
* (Optional) Diagnosis and suggestions.


The dataset is built from URL and all copyrights of the data belong to URL.


One example for chinese is #### URL #### URL ### Data Fields


For generating the QA only the below fields have been considered:


* ID : Consultatation Identifier (restarts for each file)
* URL: The url link of the extracted conversation
* Dialogue : The conversation between the doctor and the patient.


These are arranged as below in the prepared dataset. Each item will be represented with these parameters.


* "file\_name": string - signifies the file from which the conversation was extracted
* "dialogue\_id": int32 - the dialogue id
* "dialogue\_url": string - url of the conversation
* "dialogue\_turns": datasets.Sequence - sequence of dialogues between patient and the doctor.Consists ClassLabel(names=["病人", "医生"]), and "utterance"(string) for each turn. (ClassLable(names=["Patient", "Doctor"]) for english) #### URL


* 'description' (str): Description of the dialog.
* 'utterances' (list of str): Dialog utterances between patient and doctor. #### URL


* 'utterances' (list of str): Dialog utterances between patient and doctor. ### Data Splits


There are no data splits on the original raw data. The "train" split for each language contains:


* en: 229674 examples
* zh: 1921127 examples


For processed configurations, data is split into train, validation and test, with the following number of examples:



Dataset Creation
---------------- ### Curation Rationale


Medical dialogue systems are promising in assisting in telemedicine to increase access to healthcare services, improve the quality of patient care, and reduce medical costs. ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators ### Licensing Information


The authors claim that:


* They scraped the data from the following websites:
	+ MedDialog-EN: data was crawled from URL and URL
	+ MedDialog-CN: data was crawled from URL
* All copyrights of the data belong to the corresponding websites


The terms and conditions (last updated on: 11th April 2022) of URL website state:



> 
> No person (including a User, Doctor, Alternative Medicine Practitioner, or Wellness Professional) shall copy, transfer, download, republish, sell, duplicate, or "scrape", for commercial or any other purpose whatsoever, the contents or information made available on the Platform including Directory Listing Services, academic articles, and queries, in whole or in part, in any medium whatsoever.
> 
> 
> 


The terms and conditions (last updated: August 17, 2012) of URL website stipulate:



> 
> You are prohibited from republishing, selling, duplicating, or "scraping" for commercial or any other purpose whatsoever any of the data or other information contained therein, in whole or in part, in any medium whatsoever.
> 
> 
> ### Contributions


Thanks to @vrindaprabhu for adding this dataset.
passage: TAGS
#task_categories-question-answering #task_ids-closed-domain-qa #annotations_creators-found #language_creators-expert-generated #language_creators-found #multilinguality-monolingual #size_categories-1M<n<10M #source_datasets-original #language-English #language-Chinese #license-unknown #arxiv-2004.03329 #region-us 
### Dataset Summary


The MedDialog dataset (Chinese) contains conversations (in Chinese) between doctors and patients. It has 1.1 million dialogues and 4 million utterances. The data is continuously growing and more dialogues will be added. The raw dialogues are from URL. All copyrights of the data belong to URL.


The MedDialog dataset (English) contains conversations (in English) between doctors and patients. It has 0.26 million dialogues. The data is continuously growing and more dialogues will be added. The raw dialogues are from URL and URL. All copyrights of the data belong to URL and URL.


Directions for using the pre-trained model using BERT using PyTorch is available in the Homepage.### Supported Tasks and Leaderboards


Closed domain qa### Languages


Monolingual. The datasets are in English (EN) and Chinese (ZH)


Dataset Structure
-----------------### Data Instances


There are 4 configurations:


* Raw data:
	+ en
	+ zh
* Processed data:
	+ URL
	+ URL#### en


Each consultation consists of the below:


* ID
* URL
* Description of patient’s medical condition
* Dialogue


The dataset is built from URL, URL, URL and all copyrights of the data belong to these websites.#### zh


Each consultation consists of the below:


* ID
* URL
* Description of patient’s medical condition
* Dialogue
* (Optional) Diagnosis and suggestions.


The dataset is built from URL and all copyrights of the data belong to URL.


One example for chinese is#### URL#### URL passage: ### Data Fields


For generating the QA only the below fields have been considered:


* ID : Consultatation Identifier (restarts for each file)
* URL: The url link of the extracted conversation
* Dialogue : The conversation between the doctor and the patient.


These are arranged as below in the prepared dataset. Each item will be represented with these parameters.


* "file\_name": string - signifies the file from which the conversation was extracted
* "dialogue\_id": int32 - the dialogue id
* "dialogue\_url": string - url of the conversation
* "dialogue\_turns": datasets.Sequence - sequence of dialogues between patient and the doctor.Consists ClassLabel(names=["病人", "医生"]), and "utterance"(string) for each turn. (ClassLable(names=["Patient", "Doctor"]) for english)#### URL


* 'description' (str): Description of the dialog.
* 'utterances' (list of str): Dialog utterances between patient and doctor.#### URL


* 'utterances' (list of str): Dialog utterances between patient and doctor.### Data Splits


There are no data splits on the original raw data. The "train" split for each language contains:


* en: 229674 examples
* zh: 1921127 examples


For processed configurations, data is split into train, validation and test, with the following number of examples:



Dataset Creation
----------------### Curation Rationale


Medical dialogue systems are promising in assisting in telemedicine to increase access to healthcare services, improve the quality of patient care, and reduce medical costs.### Source Data#### Initial Data Collection and Normalization#### Who are the source language producers?### Annotations#### Annotation process#### Who are the annotators?### Personal and Sensitive Information


Considerations for Using the Data
---------------------------------### Social Impact of Dataset### Discussion of Biases### Other Known Limitations


Additional Information
----------------------### Dataset Curators