This dataset is HuggingFaceH4/CodeAlpaca_20K unfiltered, removing 36 instances of blatant alignment.
19986 instructions remain.

https://huggingface.co/datasets/HuggingFaceH4/CodeAlpaca_20K/blob/29ba7b7fdf0c55e5435c848cf6bbf9782fef62a6/data/test-00000-of-00001.parquet

https://huggingface.co/datasets/HuggingFaceH4/CodeAlpaca_20K/blob/a123ae447f02484d83c3457438b4422cd8417ad5/data/train-00000-of-00001.parquet

i combined all of these files above into code_alpaca_data.jsonl with parquet2json and ran clean.py on it, renamed to .json not .jsonl

normal dedupe.py script didn't find any dupes here.

inspired by https://huggingface.co/datasets/ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered

All credit to anon8231489123 for the cleanup script that I adapted to wizardlm_clean.py, I then took this script and adapted it to clean.py

region:us
{}
TAGS
#region-us 

This dataset is HuggingFaceH4/CodeAlpaca_20K unfiltered, removing 36 instances of blatant alignment.
19986 instructions remain.

URL

URL

i combined all of these files above into code_alpaca_data.jsonl with parquet2json and ran URL on it, renamed to .json not .jsonl

normal URL script didn't find any dupes here.

inspired by URL

All credit to anon8231489123 for the cleanup script that I adapted to wizardlm_clean.py, I then took this script and adapted it to URL

TAGS
#region-us 
