text
You can clean and format datasets entirely in the browser with a few lines of SQL. 
You can clean and format datasets entirely in the browser with a few lines of SQL. 
new_line
new_line
text
In this post, I replicate the process 
In this post, I replicate the process 
mention
@mlabonne
mlabonne
text
 used to clean the new 
 used to clean the new 
resource
https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1
dataset
microsoft/orca-agentinstruct-1M-v1
https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1
text
 dataset. 
 dataset. 
new_line
new_line
text
The cleaning process consists of:
The cleaning process consists of:
new_line
text
- Joining the separate splits together / add split column
- Joining the separate splits together / add split column
new_line
text
- Converting string messages into list of structs
- Converting string messages into list of structs
new_line
text
- Removing empty system prompts
- Removing empty system prompts
new_line
new_line
link
https://huggingface.co/blog/cfahlgren1/the-beginners-guide-to-cleaning-a-dataset
https://huggingface.co/blog/cfahlgren1/the-beginners-guide-to-cleaning-a-dataset
new_line
new_line
text
Here's his new cleaned dataset: 
Here's his new cleaned dataset: 
resource
https://huggingface.co/datasets/mlabonne/orca-agentinstruct-1M-v1-cleaned
dataset
mlabonne/orca-agentinstruct-1M-v1-cleaned
https://huggingface.co/datasets/mlabonne/orca-agentinstruct-1M-v1-cleaned
You can clean and format datasets entirely in the browser with a few lines of SQL. 

In this post, I replicate the process @mlabonne used to clean the new https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1 dataset. 

The cleaning process consists of:
- Joining the separate splits together / add split column
- Converting string messages into list of structs
- Removing empty system prompts

https://huggingface.co/blog/cfahlgren1/the-beginners-guide-to-cleaning-a-dataset

Here's his new cleaned dataset: https://huggingface.co/datasets/mlabonne/orca-agentinstruct-1M-v1-cleaned
❤️
clem lunarflu mlabonne John6666 Felladrin m-ric kramp nbroad qgallouedec Nymbo victor yjernite mmhamdy RedSparkie eliebak knight7561 SaylorTwift
https://cdn-avatars.huggingface.co/v1/production/uploads/6454aff9273f649830234978/cvVV08YHJpJx9xWVZqgVW.jpeg
Victor Nogueira
Felladrin
user