Notes on Deep Learning — Data Loader
Deep learning is a success because of big data.

When it comes to machine learning or deep learning 80% or more is spent wrangling with data.

The effort spent by a machine learning engineer or a data scientist to just get the data into format prepare and clean for abnormality is immense. The part of wrangling isn’t even trivial and most important part, so it can’t even be skipped. The wrangling, in fact, is given special attention as it affects the performance the most. This chapter is often revisited a couple of times to be revised.

The deep learning loves noise and learns better with noise but it cannot be yet denied that one has to prepare the data. The loading of data is one of the inevitable tasks. PyTorch helps us with load, preprocess our non-trivial datasets.

It also makes code looks nice^^ so why not do it better :)

Until this point in series, we skillfully iterated over our nice hand-written while loop. This was simple efficient but we could do a lot more than a simple iteration over our data. In particular, we could

Create batches

Shuffle the data

Load data in parallel enhancing multiprocessing

Access easy to use functions to iterate so we don’t have to worry

Use state-of-art standards and common standards across all the programs we develop

These all feature are accessible at ease if we use torch.utils.data.DataLoader

A dataset

A dataset is your data. In Pytorch it is an abstract representation of a class with functions:

len so that len(dataset) returns the size of the dataset.

so that len(dataset) returns the size of the dataset. getitem to support the indexing such that dataset[i] can be used to get ith sample

If we need to write our custom data loader we need to override the above methods
['Venali Sonone']
['Machine Learning', 'Technology', 'Deep Learning', 'Data Science']