Title: Unsupervised Learning of Slow Features for Data Efficient Regression. Abstract: Research in computational neuroscience suggests that the human brain's unparalleled data efficiency is a result of highly efficient mechanisms to extract and organize slowly changing high level features from continuous sensory inputs.
In this paper, we apply this \textit{slowness principle} to a state of the art representation learning method with the goal of performing data efficient learning of down-stream regression tasks.
To this end, we propose the \textit{slow variational autoencoder} (S-VAE), an extension to the $\beta$-VAE which applies a temporal similarity constraint to the latent representations.
We empirically compare our method to the $\beta$-VAE and the Temporal Difference VAE (TD-VAE), a state-of-the-art method for next frame prediction in latent space with temporal abstraction.
We evaluate the three methods against their data-efficiency on down-stream tasks using a synthetic 2D ball tracking dataset and a dataset generated using the DeepMind Lab environment.
In both tasks, the proposed method outperformed the baselines both with dense and sparse labeled data.
Furthermore, the S-VAE achieved similar performance compared to the baselines with 1/5 to 1/11 of data.