Load_dataset("file_location") disk quota out of space

🤗Datasets
1

I have downloaded a large dataset to a directory and I wanna load it up for the first time.
load_dataset("file_location")

However, I am getting this error out of the space error. I think I have enough space on my current directory but not enough space in my home directory. When I call load_dataset("file_location" for the first time, does it load it up using the space from my home directory? How do I set it

Traceback (most recent call last):
  File "/users/miniconda3/envs/handbook/lib/python3.10/site-packages/datasets/builder.py", line 1949, in _prepare_split_single
    num_examples, num_bytes = writer.finalize()
  File "/users/miniconda3/envs/handbook/lib/python3.10/site-packages/datasets/arrow_writer.py", line 594, in finalize
    self.stream.close()
  File "/users/miniconda3/envs/handbook/lib/python3.10/site-packages/fsspec/implementations/local.py", line 387, in close
    return self.f.close()
OSError: [Errno 122] Disk quota exceeded
2

We check available disk space using this logic (with directory being equal to Path(load_dataset_builder("file_location").cache_dir).parent).

I think I have enough space on my current directory but not enough space in my home directory.

Then pass this “current directory” as a cache_dir to load_dataset. Otherwise, generating the dataset defaults to ~/.cache/huggingface/datasets.