Dataset Script method: _generate_examples

🤗Datasets
1

Hi,

In the dataset script, under the method _generate_examples, why is “” assigned to “answer” under split==“test”?

"answer": "" if split == "test" else data["answer"]
2

In the question-answering task like the example, we want to predict answer given a question. So for the test dataset, normally we don’t have the answers. In fact it’s our job to predict them.

1 Like
3

I see, that makes sense. Thank you so much!