How to overwrite dataset with dataset.push_to_hub() or alternative?
I have a dataset that I want to update once in a while. When I calldataset.push_to_hub(repo_id = f"{COMPANY_NAME}/{dataset_name}", private=True, token=os.environ['HUGGINGFACE_TOKEN'], split=split), iteither silently does not update the dataset, even though I calleddatasets.disable_caching()or raisesValueError: Split train already presentinSplitInfo.Is there a simple way to force update an already present dataset split? Ideally, withpush_to_hub(), but any simple Python code will do, as long as I can update private datasets inCOMPANY_NAME`.
2023-08-08T09:22:00Z
I worked around this by not giving thesplitargument, but of course this is very restrictive, only allowing one default split (‘train’).To avoid caching issues during downloading, I usedownload_mode=DownloadMode.FORCE_REDOWNLOADforload_dataset.
2023-08-29T18:36:47Z
i have the same question. I want to be able to overwrite my split
2023-09-20T11:34:33Z
I would like to remove my split actually…what I did was to overwrite my train, and i ignore my test data