license:apache-2.0 arxiv:2305.15017 arxiv:1905.13319 region:us
---
license: apache-2.0
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: validation
    path: data/validation-*
  - split: test
    path: data/test-*
dataset_info:
  features:
  - name: question
    dtype: string
  - name: options
    struct:
    - name: A
      dtype: string
    - name: B
      dtype: string
    - name: C
      dtype: string
    - name: D
      dtype: string
    - name: E
      dtype: string
  - name: chain
    dtype: string
  - name: result
    dtype: string
  - name: result_float
    dtype: float64
  - name: annotated_formula
    dtype: string
  - name: linear_formula
    dtype: string
  - name: rationale
    dtype: string
  - name: index
    dtype: int64
  splits:
  - name: train
    num_bytes: 19483816
    num_examples: 20868
  - name: validation
    num_bytes: 2894695
    num_examples: 3102
  - name: test
    num_bytes: 1879346
    num_examples: 2029
  download_size: 10008725
  dataset_size: 24257857
---

# Dataset Card for "Calc-math_qa"


## Summary

This dataset is an instance of math_qa dataset, converted to a simple HTML-like language that can be easily parsed (e.g. by BeautifulSoup). The data contains 3 types of tags:
- gadget: A tag whose content is intended to be evaluated by calling an external tool (sympy-based calculator in this case)
- output: An output of the external tool
- result: The final answer of the mathematical problem (correct option)


## Supported Tasks

The dataset is intended for training Chain-of-Thought reasoning **models able to use external tools** to enhance the factuality of their responses.
This dataset presents in-context scenarios where models can outsource the computations in the reasoning chain to a calculator.


## Construction Process

We took the original math_qa dataset, parsed the nested formulas, linearized them into a sequence (chain) of operations, and replace all advanced
function calls (such as `circle_area`) with explicit elementary operations. We evaluate all the steps in each example and filter out examples if their
evaluation does not match the answer selected as correct in the data with a 5% tolerance. The sequence of steps is then saved in HTML-like language
in the `chain` column. We keep the original columns in the dataset for convenience.

You can read more information about this process in our [technical report](https://arxiv.org/abs/2305.15017).


## Content and Data splits

Content and splits correspond to the original math_qa dataset.
See [mathqa HF dataset](https://huggingface.co/datasets/math_qa) and [official website](https://math-qa.github.io/) for more info.

Columns:

- `question` - th description of a mathematical problem in natural language
- `options` - dictionary with choices 'A' to 'E' as possible solutions
- `chain` - solution in the form of step-by-step calculations encoded in simple html-like language. computed from `annotated_formula` column
- `result` - the correct option
- `result_float` - the result converted to a float
- `annotated_formula` - human-annotated nested expression that (approximately) evaluates to the selected correct answer
- `linear_formula` - same as `annotated_formula`, but linearized by original math_qa authors
- `rationale` - human-annotated free-text reasoning that leads to the correct answer
- `index` - index of the example in the original math_qa dataset




## Licence

Apache 2.0, consistently with the original dataset.


## Cite

If you use this version of dataset in research, please cite the [original MathQA paper](https://arxiv.org/abs/1905.13319), and also [our technical report](https://arxiv.org/abs/2305.15017) as follows:

```bibtex
@article{kadlcik2023calcx,
         title={Calc-X: Enriching Arithmetical Chain-of-Thoughts Datasets by Interaction with Symbolic Systems}, 
         author={Marek Kadlčík and Michal Štefánik},
         year={2023},
         eprint={2305.15017},
         archivePrefix={arXiv},
         primaryClass={cs.LG}
}
```