task_categories:question-answering task_categories:multiple-choice task_categories:visual-question-answering size_categories:1K<n<10K language:en language:zh language:fa license:cc-by-4.0 math science figure chart plot diagram reasoning document abstract scene natural image arxiv:2310.02255 region:us
---
language:
- en
- zh
- fa
license: cc-by-4.0
size_categories:
- 1K<n<10K
task_categories:
- question-answering
- multiple-choice
- visual-question-answering
pretty_name: mathvista
dataset_info:
  features:
  - name: pid
    dtype: string
  - name: question
    dtype: string
  - name: image
    dtype: string
  - name: decoded_image
    dtype: image
  - name: choices
    sequence: string
  - name: unit
    dtype: string
  - name: precision
    dtype: float64
  - name: answer
    dtype: string
  - name: question_type
    dtype: string
  - name: answer_type
    dtype: string
  - name: metadata
    struct:
    - name: category
      dtype: string
    - name: context
      dtype: string
    - name: grade
      dtype: string
    - name: img_height
      dtype: int64
    - name: img_width
      dtype: int64
    - name: language
      dtype: string
    - name: skills
      sequence: string
    - name: source
      dtype: string
    - name: split
      dtype: string
    - name: task
      dtype: string
  - name: query
    dtype: string
  splits:
  - name: testmini
    num_bytes: 142635198.0
    num_examples: 1000
  - name: test
    num_bytes: 648291344.22
    num_examples: 5141
  download_size: 0
  dataset_size: 790926542.22
tags:
- math
- science
- figure
- chart
- plot
- diagram
- reasoning
- document
- abstract scene
- natural image
configs:
- config_name: default
  data_files:
  - split: testmini
    path: data/testmini-*
  - split: test
    path: data/test-*
---
# Dataset Card for MathVista

## Dataset Description

Dataset for the Paper "[MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://arxiv.org/abs/2310.02255)".

[[Project Page](https://mathvista.github.io/)] [[Paper](https://arxiv.org/abs/2310.02255)]

The Movie Sentiment Analysis Dataset is a collection of 10,000 movie reviews, each labeled with its corresponding sentiment (positive, negative, or neutral) and a numeric rating provided by the reviewer. This dataset is useful for training and evaluating sentiment analysis models and can be used for various natural language processing tasks.

## Dataset Usage

You can access and download this dataset from the [dataset homepage](https://example.com/moviereviewsdataset). Please make sure to review the dataset's license and cite the appropriate paper when using it in your research or projects.

## Data Format

The dataset is provided in json format and contains the following attributes:

```json
"[string] Problem ID": {
    "question": [string] The question text,
    "image": [string] A file path pointing to the associated image,
    "choices": [list] Choice options for multiple-choice problems. For free-form problems, this could be a 'none' value,
    "unit": [string] The unit associated with the answer, e.g., "m^2", "years". If no unit is relevant, it can be a 'none' value,
    "precision": [integer] The number of decimal places the answer should be rounded to,
    "answer": [string] The correct answer for the problem,
    "question_type": [string] The type of question: "multi_choice" or "free_form",
    "answer_type": [string] The format of the answer: "text", "integer", "float", or "list",
    "pid": [string] Problem ID, e.g., "1",
    "metadata": {
        "split": [string] Data split: "testmini" or "test",
        "language": [string] Question language: "English", "Chinese", or "Persian",
        "img_width": [integer] The width of the associated image in pixels,
        "img_height": [integer] The height of the associated image in pixels,
        "source": [string] The source dataset from which the problem was taken,
        "category": [string] The category of the problem: "math-targeted-vqa" or "general-vqa",
        "task": [string] The task of the problem, e.g., "geometry problem solving",
        "context": [string] The visual context type of the associated image,
        "grade": [string] The grade level of the problem, e.g., "high school",
        "skills": [list] A list of mathematical reasoning skills that the problem tests
}
```

## Data Source

The **MathVista** dataset is derived from three newly collected datasets: IQTest, FunctionQA, and Paper, as well as 28 other source datasets. Details can be found in the [source.json](https://huggingface.co/datasets/AI4Math/MathVista/blob/main/source.json) file. All these source datasets have been preprocessed and labeled for evaluation purposes.

## License

The new contributions to our dataset are distributed under the [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) license, including

- The creation of three dataset: IQTest, FunctionQA, and Paper;
- The filtering and cleaning of source datasets;
- The standard formalization of instances for evaluation purposes;
- The annotations of metadata.

The copyright of the images and the questions belongs to the original authors, and the source of every image and original question can be found in the `metadata` field and in the [source.json](https://huggingface.co/datasets/AI4Math/MathVista/blob/main/source.json) file. Alongside this license, the following conditions apply:

- **Purpose:** The dataset was primarily designed for use as a test set.
- **Commercial Use:** The dataset can be used commercially as a test set, but using it as a training set is prohibited. By accessing or using this dataset, you acknowledge and agree to abide by these terms in conjunction with the [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) license.

## Citation

If you use the **MathVista** dataset in your work, please kindly cite the paper using this BibTeX:

```
@article{lu2023mathvista,
  title={MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts},
  author={Lu, Pan and Bansal, Hritik and Xia, Tony and Liu, Jiacheng and Li, Chunyuan and Hajishirzi, Hannaneh and Cheng, Hao and Chang, Kai-Wei and Galley, Michel and Gao, Jianfeng},
  journal={arXiv preprint arXiv:2310.02255},
  year={2023}
}
```