Self-supervised learning and multi-scale ensemble for EBV prediction based on whole slide image of gastric cancer

Gastric cancer is the fifth most common cancer and the fourth highest cause of cancer death in the world. One molecular subtype of gastric cancer called Epstein-Barr virus (EBV) positive tumor often responds remarkably well to immune checkpoint inhibitors and has a favorable prognosis. Considering EBV testing is often time-consuming and costly, it is of great significance to develop an automatic classification method for EBV subtype prediction based on only the cost efficient pathological images. In this study, a self-supervised learning method was proposed to train a more generalizable feature extractor (often consisting of multiple convolutional layers) using only unlabeled pathological images, based on which the classifier head (often consisting of two- or three-layer fully connected layers) can then be more efficiently trained using a large number of labelled pathological image patches. In particular, a novel formation of positive pairs for self-supervised learning was proposed by considering that neighboring patches in each pathological image often share similar visual features and therefore should have similar feature representation from output of the feature extractor. In addition, by imitating the diagnosis process of pathologists who often observe the pathological image at multiple magnifications, a multi-scale ensemble model was proposed, with each individual classifier for prediction of image patches with a unique magnification scale. Experiments on two external pathological image datasets show that the proposed self-supervised learning can help gain a more effective EBV classifier and the multi-scale ensemble model can further improve the prediction stability.