Transfer Learning for Neuroimaging via Re-use of Deep Neural Network Features

A major problem in the application of machine learning to neuroimaging is the technological variability in the MRI scanners and the different subject populations across studies. Transfer learning (TL) promises to alleviate this problem. TL refers to a family of methods which acquire knowledge from related tasks to improve generalization in the tasks of interest. In this work, we show that features of deep neural networks are transferable across data sets if the network was pre-trained on the prediction task of interest. Based on this, we propose a transfer learning pipeline which relies on the re-use of deep neural network features across data sets. We empirically study this pipeline on age and sex prediction using the UK Biobank for pre-training and three different small data sets for fine-tuning and evaluation. We find that our method outperforms classical regression methods and training a network from scratch. In particular, we improve state-of-the-art results on age and sex prediction. Our transfer learning method may therefore provide a simple and efficient pipeline to achieve high performance on small data sets.