tsvs are from here https://github.com/google-research/FLAN/tree/4c5f0ba64239f7ff98fe90a5426fa175f4d1713c/flan/v2/cot_data
1 dupe removed, 31356 instances of alignment removed

dataset preview is fucked, it looks like this
```
{"question": "What does the government have control over?\\nOptions:\\n- trouble\\n- country\\n- army\\n- city\\n- control", "answer": "city", "chain_of_thought": "A city is a large town. A government controls large towns.\n"}
```

inspired by https://huggingface.co/datasets/ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered

All credit to anon8231489123 for the cleanup script that I adapted to wizardlm_clean.py, I then took this script and adapted it to clean_format_dedupe.py
region:us
{}
TAGS
#region-us 

tsvs are from here URL
1 dupe removed, 31356 instances of alignment removed

dataset preview is fucked, it looks like this


inspired by URL

All credit to anon8231489123 for the cleanup script that I adapted to wizardlm_clean.py, I then took this script and adapted it to clean_format_dedupe.py
TAGS
#region-us 

passage: TAGS
#region-us 
