region:us
---
configs:
- config_name: default
  data_files:
  - split: test
    path: data/test-*
dataset_info:
  features:
  - name: image
    dtype: image
  - name: conversations
    dtype: string
  splits:
  - name: test
    num_bytes: 16069599.0
    num_examples: 34
  download_size: 15979558
  dataset_size: 16069599.0
---
# GPT-4V Eval samples

This is a hand curated images from the web and questions asked by myself to GPT-4V to understand its ability and limits. 

I am mainly focus in localization, OCR ability and understanding of GPT-4V vision module. So the language part is skipped as we already seen in GPT-4. As long as GPT-4V can extract the required information in text, the rest of the LLM shouldn't have any issue answering the rest of the questions. 

The numbers of examples is still pretty tiny and will continue to increase further in the future until I am satisfy with the size. So please check back from time to time.

Note : the dataset viewer had a bug which cause the image displayed differ from the actual dataset (Due to frequent update). Please load the dataset and save it on your local path for best accuracy.

## How to use:

```
import json
from datasets import load_dataset
dataset = load_dataset('theblackcat102/gpt-4v-eval-samples')['test']
print(dataset[0]['image'])
print(json.loads(dataset[0]['conversations']))
```

## Contributions

Please checkout my github repo for more details : [theblackcat102/gpt-4v-samples](https://github.com/theblackcat102/gpt-4v-samples)

## Citation

```
@article{yang2023dawn,
  title={The Dawn of LMMs: Preliminary Explorations with GPT-4V (ision)},
  author={Yang, Zhengyuan and Li, Linjie and Lin, Kevin and Wang, Jianfeng and Lin, Chung-Ching and Liu, Zicheng and Wang, Lijuan},
  journal={arXiv preprint arXiv:2309.17421},
  year={2023}
}
```

