license:other region:us
---
license: other
---
This is a new version and experinmental version of the LosslessMegacodeTraining series. Its like the version 3 but only using the most refine parts of the dataset. 

The content of this dataset is roughly 80% coding instruction data and 20% non-coding instruction data. Amounting to 650,000 evol instruction-formatted lines of data.

The outcome of having 20% non coding instruction data in the dataset is to preserve logic and reasoning skills within the model while training on coding. The lack of such skills has been observed to be a major issue with coding models such as Wizardcoder-15b and NewHope, but training models on this dataset alleviates that issue while also giving similar levels of coding knowledge.

This dataset is a combination of the following datasets:

- https://huggingface.co/datasets/rombodawg/Platypus_Evol

- https://huggingface.co/datasets/rombodawg/Rombodawgs_commitpackft_Evolinstruct_Converted

- https://huggingface.co/datasets/rombodawg/airoboros-2.1_general_purpose

- https://huggingface.co/datasets/shahules786/megacode-best