
### Dataset Summary

A bunch of datasets preprocessed and formatted with https://github.com/openai/openai-python/blob/main/chatml.md (with an addition of a context message to help RWKV (no lookback))
The dataset makes use of two more tokens. You will need to use the supplied 20b_tokeniser file with both training and inference.

### Languages

English mainly, might be a few bits of other languages.

### Things to do

1. Improve system prompt effect on output.
2. Get more reasoning data.
3. More data!

### Format

Example below

```
<|im_start|>system
You are a teacher.<|im_end|>
<|im_start|>user
Given this paragraph about Dartmouth College traditions, which homecoming-related traditions are illegal?<|im_end|>
<|im_start|>context
Dartmouth Night starts the college's traditional \"Homecoming\" weekend with an evening of speeches, a parade, and a bonfire. Traditionally, the freshman class builds the bonfire and then runs around it a set number of times in concordance with their class year; the class of 2009 performed 109 circuits, the class of 1999 performed 99, etc. The College officially discourages a number of student traditions of varying degrees of antiquity. During the circling of the bonfire, upperclassmen encourage the freshmen to \"touch the fire\", an action legally considered trespassing and prohibited by police officials present. At halftime of the Homecoming football game on the Saturday of the weekend, some upperclassmen encourage freshman to \"rush the field\", although no upperclassman has seen a significant rush since several injuries sustained during the 1986 rush prompted the school to ban the practice. Among the two or three students who sometimes run across the field, those who are arrested are charged with trespassing (the independent newspaper The Dartmouth Review claimed to set up a fund to automatically pay any fines associated with freshman who rush the field.) However, in 2012 this was proven false when two students rushed the field and were arrested for disorderly conduct. The Dartmouth Review ignored their emails until finally replying and denying that this fund had ever existed. These students then had to pay $300 fines out of pocket. For the 2011 Homecoming game, however, over 40 members of the Class of 2015 rushed the field at homecoming without any action taken by Safety and Security or the Hanover Police Department.<|im_end|>
<|im_start|>assistant
Touching the bonfire, and rushing the football field during halftime of the homecoming game<|im_end|>
```
region:us
{}
TAGS
#region-us 


### Dataset Summary

A bunch of datasets preprocessed and formatted with URL (with an addition of a context message to help RWKV (no lookback))
The dataset makes use of two more tokens. You will need to use the supplied 20b_tokeniser file with both training and inference.

### Languages

English mainly, might be a few bits of other languages.

### Things to do

1. Improve system prompt effect on output.
2. Get more reasoning data.
3. More data!

### Format

Example below


### Dataset Summary

A bunch of datasets preprocessed and formatted with URL (with an addition of a context message to help RWKV (no lookback))
The dataset makes use of two more tokens. You will need to use the supplied 20b_tokeniser file with both training and inference. ### Languages

English mainly, might be a few bits of other languages. ### Things to do

1. Improve system prompt effect on output.
2. Get more reasoning data.
3. More data! ### Format

Example below
TAGS
#region-us 
 ### Dataset Summary

A bunch of datasets preprocessed and formatted with URL (with an addition of a context message to help RWKV (no lookback))
The dataset makes use of two more tokens. You will need to use the supplied 20b_tokeniser file with both training and inference. ### Languages

English mainly, might be a few bits of other languages. ### Things to do

1. Improve system prompt effect on output.
2. Get more reasoning data.
3. More data! ### Format

Example below