- Hideyuki Shibuki, Kotaro Sakamoto, Madoka Ishioroshi, Yoshinobu Kano, T. Mitamura, Tatsunori Mori, N. Kando. 2017. Overview of the NTCIR-13 QA Lab-3 Task. Abstract: The NTCIR-13 QA Lab-3 task aims at the real-world complex Question Answering (QA) technologies using Japanese university entrance exams and their English translation on the subject of “World history”. QA Lab-3 has three end-toend tasks for multiple-choice, term and essay questions. The essay task has three subtasks of extraction, summarization and evaluation-method. There were 85 submissions from 13 teams in total. We describe the used data, formal run results, and comparison between human marks and automatic evaluation scores for essay questions. Categories and Subject Descriptions H.3.4 [INFORMATION STORAGE AND RETRIEVAL]: Systems and Software Performance evaluation (efficiency and effectiveness), Question-answering (fact retrieval) systems.
