[Feature request] Add Google Natural Question dataset
Indeed, I think this one is almost ready cc @lhoestq  I'm doing the latest adjustments to make the processing of the dataset run on Dataflow Is there an update to this? It will be very beneficial for the QA community! Still work in progress :)
The idea is to have the dataset already processed somewhere so that the user only have to download the processed files. I'm also doing it for wikipedia. Super appreciate your hard work !!
I'll cross my fingers and hope easily loadable wikipedia dataset will come soon.  Quick update on NQ: due to some limitations I met using apache beam + parquet I was not able to use the dataset in a nested parquet structure in python to convert it to our Apache Arrow format yet.
However we had planned to change this conversion step anyways so we'll make just sure that it enables to process and convert the NQ dataset to arrow. NQ was added in #427 🎉
Would be great to have https://github.com/google-research-datasets/natural-questions as an alternative to SQuAD.