 Dataset card for the dataset used in :
## Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?

Paper: https://gitlab.inria.fr/wantoun/robust-chatgpt-detection/-/raw/main/towards_chatgpt_detection.pdf

Source Code: https://gitlab.inria.fr/wantoun/robust-chatgpt-detection
## Dataset Summary

#### overview:
This dataset is made of two parts:
- First, an extension of the [Human ChatGPT Comparison Corpus (HC3) dataset](https://huggingface.co/datasets/Hello-SimpleAI/HC3) with French data automatically translated from the English source.
- Second, out-of-domain and adversarial French data set have been gathereed (Human adversarial, BingGPT, Native French ChatGPT responses).



#### Details:  
- We first format the data into three subsets: `sentence`, `question` and `full` following the original paper.
- We then extend the data by translating the English questions and answers to French.
- We provide native French ChatGPT responses to a sample of the translated questions.
- We added a subset with QA pairs from BingGPT
- We included an adversarial subset with human-written answers in the style of conversational LLMs like Bing/ChatGPT.



## Available Subsets

### Out-of-domain:
- `hc3_fr_qa_chatgpt`: Translated French questions and native French ChatGPT answers pairs from HC3. This is the `ChatGPT-Native` subset from the paper.
    - Features: `id`, `question`, `answer`, `chatgpt_answer`, `label`, `source`
    - Size:
        - test: `113` examples, `25592` words
- `qa_fr_binggpt`: French questions and BingGPT answers pairs. This is the `BingGPT` subset from the paper.
    - Features: `id`, `question`, `answer`, `label`, `deleted_clues`, `deleted_sources`, `remarks`
    - Size:
        - test: `106` examples, `26291` words
- `qa_fr_binglikehuman`: French questions and human written BingGPT-like answers pairs. This is the `Adversarial` subset from the paper.
    - Features: `id`, `question`, `answer`, `label`, `source`
    - Size:
        - test: `61` examples, `17328` words
-  `faq_fr_gouv`: French FAQ questions and answers pairs from domain ending with `.gouv` from the MQA dataset (subset 'fr-faq-page'). https://huggingface.co/datasets/clips/mqa. This is the `FAQ-Gouv` subset from the paper.
    - Features: `id`, `page_id`, `question_id`, `answer_id`, `bucket`, `domain`, `question`, `answer`, `label`
    - Size:
        - test: `235` examples, `22336` words
- `faq_fr_random`: French FAQ questions and answers pairs from random domain from the MQA dataset (subset 'fr-faq-page'). https://huggingface.co/datasets/clips/mqa. This is the `FAQ-Rand` subset from the paper.
    - Features: `id`, `page_id`, `question_id`, `answer_id`, `bucket`, `domain`, `question`, `answer`, `label`
    - Size:
        - test: `4454` examples, `271823` words



### In-domain:

- `hc3_en_qa`: English questions and answers pairs from HC3.
    - Features: `id`, `question`, `answer`, `label`, `source`
    - Size:
        - train: `68335` examples, `12306363` words
        - validation: `17114` examples, `3089634` words
        - test: `710` examples, `117001` words
- `hc3_en_sentence`: English answers split into sentences from HC3.
    - Features: `id`, `text`, `label`, `source`
    - Size:
        - train: `455320` examples, `9983784` words
        - validation: `113830` examples, `2510290` words
        - test: `4366` examples, `99965` words
- `hc3_en_full`: English questions and answers pairs concatenated from HC3.
    - Features: `id`, `text`, `label`, `source`
    - Size:
        - train: `68335` examples, `9982863` words
        - validation: `17114` examples, `2510058` words
        - test: `710` examples, `99926` words
- `hc3_fr_qa`: Translated French questions and answers pairs from HC3.
    - Features: `id`, `question`, `answer`, `label`, `source`
    - Size:
        - train: `68283` examples, `12660717` words
        - validation: `17107` examples, `3179128` words
        - test: `710`  examples, `127193` words
- `hc3_fr_sentence`: Translated French answers split into sentences from HC3.
    - Features: `id`, `text`, `label`, `source`
    - Size:
        - train: `464885` examples, `10189606` words
        - validation: `116524` examples, `2563258` words
        - test:  `4366` examples, `108374` words
- `hc3_fr_full`: Translated French questions and answers pairs concatenated from HC3.
    - Features: `id`, `text`, `label`, `source`
    - Size:
        - train: `68283` examples, `10188669` words
        - validation: `17107` examples, `2563037` words
        - test: `710` examples, `108352` words



## How to load

```python
from datasets import load_dataset

dataset = load_dataset("almanach/hc3_multi", "hc3_fr_qa")
```

## Dataset Copyright

If the source datasets used in this corpus has a specific license which is stricter than CC-BY-SA, our products follow the same.
If not, they follow CC-BY-SA license.

| English Split       | Source | Source License | Note |
|----------|-------------|--------|-------------|
| reddit_eli5 | [ELI5](https://github.com/facebookresearch/ELI5)   | BSD License    |     |
| open_qa  | [WikiQA](https://www.microsoft.com/en-us/download/details.aspx?id=52419)  | [PWC Custom](https://paperswithcode.com/datasets/license)   |      |
| wiki_csai   | Wikipedia | CC-BY-SA |   | [Wiki FAQ](https://en.wikipedia.org/wiki/Wikipedia:FAQ/Copyright) |
| medicine    | [Medical Dialog](https://github.com/UCSD-AI4H/Medical-Dialogue-System) | Unknown|  [Asking](https://github.com/UCSD-AI4H/Medical-Dialogue-System/issues/10)|
| finance     | [FiQA](https://paperswithcode.com/dataset/fiqa-1) | Unknown |  Asking by 📧  |
| FAQ | [MQA]( https://huggingface.co/datasets/clips/mqa) | CC0 1.0| |
| ChatGPT/BingGPT | | Unknown | This is ChatGPT/BingGPT generated data. |
| Human | | CC-BY-SA | |

## Citation

```bibtex
@proceedings{towards-a-robust-2023-antoun,
    title = "Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?",
    editor = "Antoun, Wissam  and
      Mouilleron, Virginie  and
      Sagot, Benoit  and
      Seddah, Djam{\'e}",
    month = "6",
    year = "2023",
    address = "Paris, France",
    publisher = "ATALA",
    url = "https://gitlab.inria.fr/wantoun/robust-chatgpt-detection/-/raw/main/towards_chatgpt_detection.pdf",
}
```


```bibtex
@article{guo-etal-2023-hc3,
    title = "How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection",
    author = "Guo, Biyang  and
      Zhang, Xin  and
      Wang, Ziyuan  and
      Jiang, Minqi  and
      Nie, Jinran  and
      Ding, Yuxuan  and
      Yue, Jianwei  and
      Wu, Yupeng",
    journal={arXiv preprint arxiv:2301.07597}
    year = "2023",
    url ="https://arxiv.org/abs/2301.07597"
}
```
task_categories:text-classification task_categories:question-answering task_categories:sentence-similarity task_categories:zero-shot-classification size_categories:10K<n<100K language:en language:fr license:cc-by-sa-4.0 ChatGPT Bing LM Detection Detection OOD arxiv:2301.07597 region:us
{"language": ["en", "fr"], "license": "cc-by-sa-4.0", "size_categories": ["10K<n<100K"], "task_categories": ["text-classification", "question-answering", "sentence-similarity", "zero-shot-classification"], "tags": ["ChatGPT", "Bing", "LM Detection", "Detection", "OOD"]}
TAGS
#task_categories-text-classification #task_categories-question-answering #task_categories-sentence-similarity #task_categories-zero-shot-classification #size_categories-10K<n<100K #language-English #language-French #license-cc-by-sa-4.0 #ChatGPT #Bing #LM Detection #Detection #OOD #arxiv-2301.07597 #region-us 

Dataset card for the dataset used in :


Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?
--------------------------------------------------------------------------------------------


Paper: URL


Source Code: URL


Dataset Summary
---------------


#### overview:


This dataset is made of two parts:


* First, an extension of the Human ChatGPT Comparison Corpus (HC3) dataset with French data automatically translated from the English source.
* Second, out-of-domain and adversarial French data set have been gathereed (Human adversarial, BingGPT, Native French ChatGPT responses).


#### Details:


* We first format the data into three subsets: 'sentence', 'question' and 'full' following the original paper.
* We then extend the data by translating the English questions and answers to French.
* We provide native French ChatGPT responses to a sample of the translated questions.
* We added a subset with QA pairs from BingGPT
* We included an adversarial subset with human-written answers in the style of conversational LLMs like Bing/ChatGPT.


Available Subsets
-----------------


### Out-of-domain:


* 'hc3\_fr\_qa\_chatgpt': Translated French questions and native French ChatGPT answers pairs from HC3. This is the 'ChatGPT-Native' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'chatgpt\_answer', 'label', 'source'
	+ Size:
		- test: '113' examples, '25592' words
* 'qa\_fr\_binggpt': French questions and BingGPT answers pairs. This is the 'BingGPT' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'label', 'deleted\_clues', 'deleted\_sources', 'remarks'
	+ Size:
		- test: '106' examples, '26291' words
* 'qa\_fr\_binglikehuman': French questions and human written BingGPT-like answers pairs. This is the 'Adversarial' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- test: '61' examples, '17328' words
* 'faq\_fr\_gouv': French FAQ questions and answers pairs from domain ending with '.gouv' from the MQA dataset (subset 'fr-faq-page'). URL This is the 'FAQ-Gouv' subset from the paper.
	+ Features: 'id', 'page\_id', 'question\_id', 'answer\_id', 'bucket', 'domain', 'question', 'answer', 'label'
	+ Size:
		- test: '235' examples, '22336' words
* 'faq\_fr\_random': French FAQ questions and answers pairs from random domain from the MQA dataset (subset 'fr-faq-page'). URL This is the 'FAQ-Rand' subset from the paper.
	+ Features: 'id', 'page\_id', 'question\_id', 'answer\_id', 'bucket', 'domain', 'question', 'answer', 'label'
	+ Size:
		- test: '4454' examples, '271823' words


### In-domain:


* 'hc3\_en\_qa': English questions and answers pairs from HC3.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- train: '68335' examples, '12306363' words
		- validation: '17114' examples, '3089634' words
		- test: '710' examples, '117001' words
* 'hc3\_en\_sentence': English answers split into sentences from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '455320' examples, '9983784' words
		- validation: '113830' examples, '2510290' words
		- test: '4366' examples, '99965' words
* 'hc3\_en\_full': English questions and answers pairs concatenated from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '68335' examples, '9982863' words
		- validation: '17114' examples, '2510058' words
		- test: '710' examples, '99926' words
* 'hc3\_fr\_qa': Translated French questions and answers pairs from HC3.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- train: '68283' examples, '12660717' words
		- validation: '17107' examples, '3179128' words
		- test: '710' examples, '127193' words
* 'hc3\_fr\_sentence': Translated French answers split into sentences from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '464885' examples, '10189606' words
		- validation: '116524' examples, '2563258' words
		- test: '4366' examples, '108374' words
* 'hc3\_fr\_full': Translated French questions and answers pairs concatenated from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '68283' examples, '10188669' words
		- validation: '17107' examples, '2563037' words
		- test: '710' examples, '108352' words


How to load
-----------


Dataset Copyright
-----------------


If the source datasets used in this corpus has a specific license which is stricter than CC-BY-SA, our products follow the same.
If not, they follow CC-BY-SA license.




#### overview:


This dataset is made of two parts:


* First, an extension of the Human ChatGPT Comparison Corpus (HC3) dataset with French data automatically translated from the English source.
* Second, out-of-domain and adversarial French data set have been gathereed (Human adversarial, BingGPT, Native French ChatGPT responses). #### Details:


* We first format the data into three subsets: 'sentence', 'question' and 'full' following the original paper.
* We then extend the data by translating the English questions and answers to French.
* We provide native French ChatGPT responses to a sample of the translated questions.
* We added a subset with QA pairs from BingGPT
* We included an adversarial subset with human-written answers in the style of conversational LLMs like Bing/ChatGPT.


Available Subsets
----------------- ### Out-of-domain:


* 'hc3\_fr\_qa\_chatgpt': Translated French questions and native French ChatGPT answers pairs from HC3. This is the 'ChatGPT-Native' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'chatgpt\_answer', 'label', 'source'
	+ Size:
		- test: '113' examples, '25592' words
* 'qa\_fr\_binggpt': French questions and BingGPT answers pairs. This is the 'BingGPT' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'label', 'deleted\_clues', 'deleted\_sources', 'remarks'
	+ Size:
		- test: '106' examples, '26291' words
* 'qa\_fr\_binglikehuman': French questions and human written BingGPT-like answers pairs. This is the 'Adversarial' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- test: '61' examples, '17328' words
* 'faq\_fr\_gouv': French FAQ questions and answers pairs from domain ending with '.gouv' from the MQA dataset (subset 'fr-faq-page'). URL This is the 'FAQ-Gouv' subset from the paper.
	+ Features: 'id', 'page\_id', 'question\_id', 'answer\_id', 'bucket', 'domain', 'question', 'answer', 'label'
	+ Size:
		- test: '235' examples, '22336' words
* 'faq\_fr\_random': French FAQ questions and answers pairs from random domain from the MQA dataset (subset 'fr-faq-page'). URL This is the 'FAQ-Rand' subset from the paper.
	+ Features: 'id', 'page\_id', 'question\_id', 'answer\_id', 'bucket', 'domain', 'question', 'answer', 'label'
	+ Size:
		- test: '4454' examples, '271823' words ### In-domain:


* 'hc3\_en\_qa': English questions and answers pairs from HC3.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- train: '68335' examples, '12306363' words
		- validation: '17114' examples, '3089634' words
		- test: '710' examples, '117001' words
* 'hc3\_en\_sentence': English answers split into sentences from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '455320' examples, '9983784' words
		- validation: '113830' examples, '2510290' words
		- test: '4366' examples, '99965' words
* 'hc3\_en\_full': English questions and answers pairs concatenated from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '68335' examples, '9982863' words
		- validation: '17114' examples, '2510058' words
		- test: '710' examples, '99926' words
* 'hc3\_fr\_qa': Translated French questions and answers pairs from HC3.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- train: '68283' examples, '12660717' words
		- validation: '17107' examples, '3179128' words
		- test: '710' examples, '127193' words
* 'hc3\_fr\_sentence': Translated French answers split into sentences from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '464885' examples, '10189606' words
		- validation: '116524' examples, '2563258' words
		- test: '4366' examples, '108374' words
* 'hc3\_fr\_full': Translated French questions and answers pairs concatenated from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '68283' examples, '10188669' words
		- validation: '17107' examples, '2563037' words
		- test: '710' examples, '108352' words


How to load
-----------


Dataset Copyright
-----------------


If the source datasets used in this corpus has a specific license which is stricter than CC-BY-SA, our products follow the same.
If not, they follow CC-BY-SA license.
TAGS
#task_categories-text-classification #task_categories-question-answering #task_categories-sentence-similarity #task_categories-zero-shot-classification #size_categories-10K<n<100K #language-English #language-French #license-cc-by-sa-4.0 #ChatGPT #Bing #LM Detection #Detection #OOD #arxiv-2301.07597 #region-us 
 #### overview:


This dataset is made of two parts:


* First, an extension of the Human ChatGPT Comparison Corpus (HC3) dataset with French data automatically translated from the English source.
* Second, out-of-domain and adversarial French data set have been gathereed (Human adversarial, BingGPT, Native French ChatGPT responses). #### Details:


* We first format the data into three subsets: 'sentence', 'question' and 'full' following the original paper.
* We then extend the data by translating the English questions and answers to French.
* We provide native French ChatGPT responses to a sample of the translated questions.
* We added a subset with QA pairs from BingGPT
* We included an adversarial subset with human-written answers in the style of conversational LLMs like Bing/ChatGPT.


Available Subsets
----------------- ### Out-of-domain:


* 'hc3\_fr\_qa\_chatgpt': Translated French questions and native French ChatGPT answers pairs from HC3. This is the 'ChatGPT-Native' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'chatgpt\_answer', 'label', 'source'
	+ Size:
		- test: '113' examples, '25592' words
* 'qa\_fr\_binggpt': French questions and BingGPT answers pairs. This is the 'BingGPT' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'label', 'deleted\_clues', 'deleted\_sources', 'remarks'
	+ Size:
		- test: '106' examples, '26291' words
* 'qa\_fr\_binglikehuman': French questions and human written BingGPT-like answers pairs. This is the 'Adversarial' subset from the paper.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- test: '61' examples, '17328' words
* 'faq\_fr\_gouv': French FAQ questions and answers pairs from domain ending with '.gouv' from the MQA dataset (subset 'fr-faq-page'). URL This is the 'FAQ-Gouv' subset from the paper.
	+ Features: 'id', 'page\_id', 'question\_id', 'answer\_id', 'bucket', 'domain', 'question', 'answer', 'label'
	+ Size:
		- test: '235' examples, '22336' words
* 'faq\_fr\_random': French FAQ questions and answers pairs from random domain from the MQA dataset (subset 'fr-faq-page'). URL This is the 'FAQ-Rand' subset from the paper.
	+ Features: 'id', 'page\_id', 'question\_id', 'answer\_id', 'bucket', 'domain', 'question', 'answer', 'label'
	+ Size:
		- test: '4454' examples, '271823' words ### In-domain:


* 'hc3\_en\_qa': English questions and answers pairs from HC3.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- train: '68335' examples, '12306363' words
		- validation: '17114' examples, '3089634' words
		- test: '710' examples, '117001' words
* 'hc3\_en\_sentence': English answers split into sentences from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '455320' examples, '9983784' words
		- validation: '113830' examples, '2510290' words
		- test: '4366' examples, '99965' words
* 'hc3\_en\_full': English questions and answers pairs concatenated from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '68335' examples, '9982863' words
		- validation: '17114' examples, '2510058' words
		- test: '710' examples, '99926' words
* 'hc3\_fr\_qa': Translated French questions and answers pairs from HC3.
	+ Features: 'id', 'question', 'answer', 'label', 'source'
	+ Size:
		- train: '68283' examples, '12660717' words
		- validation: '17107' examples, '3179128' words
		- test: '710' examples, '127193' words
* 'hc3\_fr\_sentence': Translated French answers split into sentences from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '464885' examples, '10189606' words
		- validation: '116524' examples, '2563258' words
		- test: '4366' examples, '108374' words
* 'hc3\_fr\_full': Translated French questions and answers pairs concatenated from HC3.
	+ Features: 'id', 'text', 'label', 'source'
	+ Size:
		- train: '68283' examples, '10188669' words
		- validation: '17107' examples, '2563037' words
		- test: '710' examples, '108352' words


How to load
-----------


Dataset Copyright
-----------------


If the source datasets used in this corpus has a specific license which is stricter than CC-BY-SA, our products follow the same.
If not, they follow CC-BY-SA license.