any possibility to download part of large datasets only?
Not yet but it’s on the short/mid-term roadmap (requested by many indeed). oh, great, really awesome feature to have, thank you very much for the great, fabulous work We'll work on dataset streaming soon. This should allow you to only load the examples you need ;) thanks a lot Quentin, this would be really really a great feature to have

On Wed, Apr 7, 2021 at 12:14 PM Quentin Lhoest ***@***.***>
wrote:

> We'll work on dataset streaming soon. This should allow you to only load
> the examples you need ;)
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/2161#issuecomment-814791922>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AS37NMROD62QAKIJMAKWISTTHQWBVANCNFSM42IUI5JQ>
> .
>
 Is streaming completed? On the 1.8.0 docs it is mentioned (https://huggingface.co/docs/datasets/dataset_streaming.html), but when following the example I get the following error:

```
>>> dataset2 = load_dataset("amazon_us_reviews", "Pet_Products_v1_00", split='train', streaming=True)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-21-1eedab26cff1> in <module>()
----> 1 en_dataset = load_dataset('oscar', "unshuffled_deduplicated_en", split='train', streaming=True)

3 frames
/usr/local/lib/python3.7/dist-packages/datasets/builder.py in _create_builder_config(self, name, custom_features, **config_kwargs)
    339                 if value is not None:
    340                     if not hasattr(builder_config, key):
--> 341                         raise ValueError(f"BuilderConfig {builder_config} doesn't have a '{key}' key.")
    342                     setattr(builder_config, key, value)
    343 

ValueError: BuilderConfig OscarConfig(name='unshuffled_deduplicated_en', version=1.0.0, data_dir=None, data_files=None, description='Unshuffled and deduplicated, English OSCAR dataset') doesn't have a 'streaming' key.
```

UPDATE: Managed to get streaming working by building from source and installing the additional `datasets[streaming]` package:

```
!pip install git+https://github.com/huggingface/datasets.git
!pip install datasets[streaming]
``` Hi ! Streaming is available on `master` only right now. We'll make a new release 1.9.0 on Monday :)
Hi
Some of the datasets I need like cc100 are very large, and then I wonder if I can download first X samples of the shuffled/unshuffled data without going through first downloading the whole data then sampling? thanks