13,217 sets split to 512 LLaMa tokens. Each output was trimmed to the nearest `\n` at 512 or less tokens, since KoboldCPP's chat format always formats it with names like this;

`\nJerry: Hello. I'm Jerry\nBob: Hi. I'm Bob.`.

Since there's 980 episodes, and it needs to be chunked into 512 tokens, there ends up being 13,217 splits. That's ~13.49 splits for each episode. 

TV Shows Added:
- 114 - Futurama
- 568 - The Simpsons
- 24 - Seinfeld
- 274 - South Park

Plans:
- I need to augment the instructions so that is way more varied, and natural (like maybe some are capitalized, or some spelt slightly wrong, or some have weird wording, etc).
- Possibly make each sample overlap by a small amount like 10-35%, potentially helping it understand contexts better?

Original Datasets:
- [Futurama](https://www.kaggle.com/datasets/josephvm/futurama-seasons-16-transcripts?select=only_spoken_text.csv)
- [The Simpsons](https://www.kaggle.com/datasets/prashant111/the-simpsons-dataset?resource=download&select=simpsons_script_lines.csv)
- [Seinfeld](https://www.kaggle.com/datasets/thec03u5/seinfeld-chronicles?select=scripts.csv)
- [South Park](https://www.kaggle.com/datasets/thedevastator/south-park-scripts-dataset?select=All-seasons.csv)
task_categories:text-generation task_categories:conversational size_categories:10K<n<100K language:en region:us
{"language": ["en"], "size_categories": ["10K<n<100K"], "task_categories": ["text-generation", "conversational"]}
TAGS
#task_categories-text-generation #task_categories-conversational #size_categories-10K<n<100K #language-English #region-us 

13,217 sets split to 512 LLaMa tokens. Each output was trimmed to the nearest '\n' at 512 or less tokens, since KoboldCPP's chat format always formats it with names like this;

'\nJerry: Hello. I'm Jerry\nBob: Hi. I'm Bob.'.

Since there's 980 episodes, and it needs to be chunked into 512 tokens, there ends up being 13,217 splits. That's ~13.49 splits for each episode. 

TV Shows Added:
- 114 - Futurama
- 568 - The Simpsons
- 24 - Seinfeld
- 274 - South Park

Plans:
- I need to augment the instructions so that is way more varied, and natural (like maybe some are capitalized, or some spelt slightly wrong, or some have weird wording, etc).
- Possibly make each sample overlap by a small amount like 10-35%, potentially helping it understand contexts better?

Original Datasets:
- Futurama
- The Simpsons
- Seinfeld
- South Park
TAGS
#task_categories-text-generation #task_categories-conversational #size_categories-10K<n<100K #language-English #region-us 
