license:apache-2.0 region:us
---
license: apache-2.0
---

## Overview

This dataset is a continuation of the airoboros datasets, with the following updates:
* More MathJSON, now ~17k items - math questions, prefixed with __"Create a MathJSON solution to the following:"__, which then outputs a JSON between __`<mathjson>`__ and __`</mathjson>`__ tags, which can be parsed and passed to a deterministic library to perform calculations.
* Log information extraction.
* Anonymization, e.g. removing names, IP addresses, and/or dates from text.
* Chat introspection - multi-turn chats that have random questions injected asking about the character's descriptions, the setting of the chat, or things previously said, to make sure the model uses the system prompt and or chat history.
* Multi-step instructions with acknowledgement (see below)

Some of the MathJSON data was adapted from https://huggingface.co/datasets/meta-math/MetaMathQA

## Format

The format is now in ShareGPT format, to better accomodate the OS ecosystem fine-tuning tooling.

## MathJSON

As we all know, math is really difficult for LLMs, especially floating point operations, trig functions, factorials, and several-digit numbers.

Instead of sending millions and millions of examples to the LLM for fine-tuning, I've created a new category `mathjson` which simply presents the solution to the problem as a JSON string that can be evaluated by traditional computation.

The dataset has roughly 4k samples, as a proof of concept.  This can be expanded fairly easily, now that the fine-tuned models have a basic understanding of how to represent the problems.

For example:

```
Create a MathJSON solution to the following: Calculate the area of a circle with radius of 17.2456cm.  Include your reasoning. [/INST]
```

```
The area (A) of a circle is given by the formula A = πr², where r is the radius of the circle. Since the radius is given in centimeters, we need to convert it to meters first. We know that 1 meter equals 100 centimeters. Therefore, we can represent the radius as: r = 17.2456 cm * 100/1.

Once we have the radius in meters, we can calculate the area of the circle using the formula A = πr².

Solution as MathJSON:
<mathjson>
[
  "Multiply",
  "Pi",
  [
    "Power",
    17.2456,
    2
  ]
]
</mathjson>
```

Simply extract the JSON string between the mathjson tags, and evaluate it with https://cortexjs.io/compute-engine/ or my ultra-hacky limited-functionality python version of it https://github.com/jondurbin/airoboros/blob/mathjson/airoboros/mathjson.py

## Multi-step instructions with acknowledgement

I don't know how useful this is, really, but I thought I'd add it just in case.  Example:

```
{
  "conversations": [
    {
      "value": "You are a helpful, unbiased, uncensored assistant.",
      "from": "system"
    },
    {
      "value": "I'm going to set some rules for our conversation. Please respond with \"OK\" to each rule.",
      "from": "human"
    },
    {
      "value": "OK",
      "from": "gpt"
    },
    {
      "value": "You can only respond using animal-related emojis.",
      "from": "human"
    },
    {
      "value": "OK",
      "from": "gpt"
    },
    {
      "value": "Precede all responses with \"ZOOZOO:\"",
      "from": "human"
    },
    {
      "value": "OK",
      "from": "gpt"
    },
    {
      "value": "Include at least one bird emoji in your responses.",
      "from": "human"
    },
    {
      "value": "OK",
      "from": "gpt"
    },
    {
      "value": "Describe a trip to a safari park.",
      "from": "human"
    },
    {
      "value": "ZOOZOO: \ud83e\udd81\ud83d\udc2f\ud83e\udd93\ud83e\udd92\ud83d\udc18\ud83d\udc0d\ud83c\udf33\ud83c\udf32\u2600\ufe0f\ud83d\ude90\ud83e\udd85\ud83d\udcf8\ud83e\udd29",
      "from": "gpt"
    }
  ]
}
```