This dataset is Dahoas/synthetic-instruct-gptj-pairwise, removing 2453 instances of blatant alignment and 44 duplicates.
30646 instructions remain.

https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise/blob/987e7db3ef7e68c93950bc61edc10ab9b7c239ef/data/train-00000-of-00001-1e5d57b93c448e7a.parquet

turned above code_alpaca_data.jsonl with parquet2json and ran clean.py on it, renamed to .json not .jsonl

inspired by https://huggingface.co/datasets/ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered

All credit to anon8231489123 for the cleanup script that I adapted to wizardlm_clean.py, I then took this script and adapted it to clean.py
region:us
{}
TAGS
#region-us 

This dataset is Dahoas/synthetic-instruct-gptj-pairwise, removing 2453 instances of blatant alignment and 44 duplicates.
30646 instructions remain.

URL

turned above code_alpaca_data.jsonl with parquet2json and ran URL on it, renamed to .json not .jsonl

inspired by URL

All credit to anon8231489123 for the cleanup script that I adapted to wizardlm_clean.py, I then took this script and adapted it to URL
TAGS
#region-us 

passage: TAGS
#region-us 
