Testing and dummy data required for dataset loading script?

🤗Datasets
1

I’ve been following along many of the examples in the documentation here for creating a dataset loading script.

Supposing I have a dataset in csv format, have written a dataset loading script, and don’t have any intention of uploading the dataset to the HF Hub, is it necessary to run the following commands:

datasets-cli test path/to/my_dataset_folder --save_infos --all_configs
datasets-cli dummy_data path/to/my_dataset_folder --auto_generate

in order to be able to load the dataset using:

from datasets import load_dataset
load_dataset("path/to/my_dataset_folder")

Thanks in advance for your help!!

2

Hi! We require running these CLI commands only for “canonical” datasets created by submitting a GitHub PR (to verify they are working), so feel free to ignore them.

3

Will do @mariosasko, thanks for your response!