
# GPT Wiki Intro

## Overview

Dataset for training models to classify human written vs GPT/ChatGPT generated text.
This dataset contains Wikipedia introductions and GPT (Curie) generated introductions for 150k topics.

Prompt used for generating text
```
200 word wikipedia style introduction on '{title}'
{starter_text}
```
where `title` is the title for the wikipedia page, and `starter_text` is the first seven words of the wikipedia introduction.
Here's an example of prompt used to generate the introduction paragraph for 'Secretory protein' - 
>'200 word wikipedia style introduction on Secretory protein
>
> A secretory protein is any protein, whether'

Configuration used for GPT model
```
model="text-curie-001",
prompt=prompt,
temperature=0.7,
max_tokens=300,
top_p=1,
frequency_penalty=0.4,
presence_penalty=0.1
```


Schema for the dataset
|Column               |Datatype|Description                                |
|---------------------|--------|-------------------------------------------|
|id                   |int64   |ID                                         |
|url                  |string  |Wikipedia URL                              |
|title                |string  |Title                                      |
|wiki_intro           |string  |Introduction paragraph from wikipedia      |
|generated_intro      |string  |Introduction generated by GPT (Curie) model|
|title_len            |int64   |Number of words in title                   |
|wiki_intro_len       |int64   |Number of words in wiki_intro              |
|generated_intro_len  |int64   |Number of words in generated_intro         |
|prompt               |string  |Prompt used to generate intro              |
|generated_text       |string  |Text continued after the prompt            |
|prompt_tokens        |int64   |Number of tokens in the prompt             |
|generated_text_tokens|int64   |Number of tokens in generated text         |


## Credits
* [wikipedia dataset](https://huggingface.co/datasets/wikipedia#licensing-information)


## Code
Code to create this dataset can be found on [GitHub](https://github.com/aadityaubhat/wiki_gpt)

## Citation
```
@misc {aaditya_bhat_2023,
	author       = { {Aaditya Bhat} },
	title        = { GPT-wiki-intro (Revision 0e458f5) },
	year         = 2023,
	url          = { https://huggingface.co/datasets/aadityaubhat/GPT-wiki-intro },
	doi          = { 10.57967/hf/0326 },
	publisher    = { Hugging Face }
}
```
task_categories:text-classification task_categories:zero-shot-classification task_categories:text-generation size_categories:100K<n<1M language:en license:cc doi:10.57967/hf/0326 region:us
{"language": ["en"], "license": "cc", "size_categories": ["100K<n<1M"], "task_categories": ["text-classification", "zero-shot-classification", "text-generation"], "pretty_name": "GPT Wiki Intro"}
TAGS
#task_categories-text-classification #task_categories-zero-shot-classification #task_categories-text-generation #size_categories-100K<n<1M #language-English #license-cc #doi-10.57967/hf/0326 #region-us 

GPT Wiki Intro
==============


Overview
--------


Dataset for training models to classify human written vs GPT/ChatGPT generated text.
This dataset contains Wikipedia introductions and GPT (Curie) generated introductions for 150k topics.


Prompt used for generating text


where 'title' is the title for the wikipedia page, and 'starter\_text' is the first seven words of the wikipedia introduction.
Here's an example of prompt used to generate the introduction paragraph for 'Secretory protein' -



> 
> '200 word wikipedia style introduction on Secretory protein
> 
> 
> A secretory protein is any protein, whether'
> 
> 
> 


Configuration used for GPT model


Schema for the dataset


Column: id, Datatype: int64, Description: ID
Column: url, Datatype: string, Description: Wikipedia URL
Column: title, Datatype: string, Description: Title
Column: wiki\_intro, Datatype: string, Description: Introduction paragraph from wikipedia
Column: generated\_intro, Datatype: string, Description: Introduction generated by GPT (Curie) model
Column: title\_len, Datatype: int64, Description: Number of words in title
Column: wiki\_intro\_len, Datatype: int64, Description: Number of words in wiki\_intro
Column: generated\_intro\_len, Datatype: int64, Description: Number of words in generated\_intro
Column: prompt, Datatype: string, Description: Prompt used to generate intro
Column: generated\_text, Datatype: string, Description: Text continued after the prompt
Column: prompt\_tokens, Datatype: int64, Description: Number of tokens in the prompt
Column: generated\_text\_tokens, Datatype: int64, Description: Number of tokens in generated text


Credits
-------


* wikipedia dataset


Code
----


Code to create this dataset can be found on GitHub



TAGS
#task_categories-text-classification #task_categories-zero-shot-classification #task_categories-text-generation #size_categories-100K<n<1M #language-English #license-cc #doi-10.57967/hf/0326 #region-us 

passage: TAGS
#task_categories-text-classification #task_categories-zero-shot-classification #task_categories-text-generation #size_categories-100K<n<1M #language-English #license-cc #doi-10.57967/hf/0326 #region-us 
