A robust and high-performance neural network for classifying landmarks in upper gastrointestinal endoscopy images

Gastrointestinal Endoscopy in real-time needs to be done quickly and accurately. During an endoscopic examination, the technical doctor must spend a lot of time classifying anatomical landmarks before proceeding with further work such as lesion detection or abnormal region segmentation. The endoscopist also can ensure the completion of the examination with the help of the anatomical landmarks. In this study, we develop a robust and high-performance neural network classifying ten anatomical landmarks in the Upper Gastrointestinal tract. To this end, the proposed method consists of a Siamese Neural Network (SNN) to learn an efficient (dis)similarity metric of a pair of landmarks. We then propose to use the weight that is learned by SNN to a CNN-based structure for the feature extraction. The proposed CNN-based architecture is a lightweight model of Resnet-18 that is designed to achieve the best performance as well as a low cost of computational time. Finally, we utilize an SVM classifier to automatically classify ten anatomical landmarks from the Upper Gastrointestinal endoscopic images. The dataset used in this study is collected in four lighting modes such as (WLI) White-light Imaging, FICE (Flexible spectral imaging color enhancement), BLI (Blue Light Imaging), and LCI (Linked color imaging). The dataset is labeled by endoscopists with more than five years of experience. To enrich the dataset, we also research and evaluate augmentation techniques as effectively as possible. The augmented data includes a combination of the geometrical transforms and the data generated using CycleGAN deep learning networks. The proposed method works well with four lighting modes without depending on a certain lighting mode. It achieved 96.3% sensitivity, 99.6% specificity, and 99.3% accuracy. As a result of the computational time, we achieved approximately 60 FPS (Frame Per Second) using 20W average power with the deep learning framework Pytorch on the Jetson Xavier AGX.