task_categories:multiple-choice language:en language:zh license:cc-by-sa-4.0 arxiv:2304.03439 region:us
The dataset is an amendment and re-annotation of LogiQA in 2020, a large-scale logical reasoning reading comprehension dataset adapted from the Chinese Civil Service Examination. We increase the data size, refine the texts with manual translation by professionals, and improve the quality by removing items with distinctive cultural features like Chinese idioms. Furthermore, we conduct a fine-grained annotation on the dataset and turn it into a two-way natural language inference (NLI) task, resulting in 35k premise-hypothesis pairs with gold labels, making it the first large-scale NLI dataset for complex logical reasoning
@ARTICLE{10174688,
  author={Liu, Hanmeng and Liu, Jian and Cui, Leyang and Teng, Zhiyang and Duan, Nan and Zhou, Ming and Zhang, Yue},
  journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
  title={LogiQA 2.0 — An Improved Dataset for Logical Reasoning in Natural Language Understanding},
  year={2023},
  volume={},
  number={},
  pages={1-16},
  doi={10.1109/TASLP.2023.3293046}}
---
license: cc-by-sa-4.0
task_categories:
- multiple-choice
language:
- en
- zh
pretty_name: LogiQA2.0
data_splits:
- train
- validation
- test 
---

# Dataset Card for Dataset Name

## Dataset Description

- **Homepage:** https://github.com/csitfun/LogiQA2.0, https://github.com/csitfun/LogiEval
- **Repository:** https://github.com/csitfun/LogiQA2.0, https://github.com/csitfun/LogiEval
- **Paper:** https://ieeexplore.ieee.org/abstract/document/10174688

### Dataset Summary

Logiqa2.0 dataset - logical reasoning in MRC and NLI tasks

LogiEval: a benchmark suite for testing logical reasoning abilities of instruct-prompt large language models

### Licensing Information

 Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

### Citation Information

@ARTICLE{10174688,
  author={Liu, Hanmeng and Liu, Jian and Cui, Leyang and Teng, Zhiyang and Duan, Nan and Zhou, Ming and Zhang, Yue},
  journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing}, 
  title={LogiQA 2.0 — An Improved Dataset for Logical Reasoning in Natural Language Understanding}, 
  year={2023},
  volume={},
  number={},
  pages={1-16},
  doi={10.1109/TASLP.2023.3293046}}

@misc{liu2023evaluating,
      title={Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4}, 
      author={Hanmeng Liu and Ruoxi Ning and Zhiyang Teng and Jian Liu and Qiji Zhou and Yue Zhang},
      year={2023},
      eprint={2304.03439},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}