colorectal
51864659
Uncertainty and Interpretability in Convolutional Neural Networks for Semantic Segmentation of Colorectal Polyps
Colorectal polyps are known to be potential precursors to colorectal cancer, which is one of the leading causes of cancer related deaths on a global scale. Early detection and prevention of colorectal cancer is primarily enabled through manual screenings, where the intestines of a patient is visually examined. Such a procedure can be challenging and exhausting for the person performing the screening. This has resulted in numerous studies on designing automatic systems aimed at supporting physicians during the examination. Recently, such automatic systems have seen a significant improvement as a result of an increasing amount of publicly available colorectal imagery and advances in deep learning research for object image recognition. Specifically, decision support systems based on Convolutional Neural Networks (CNNs) have demonstrated state of the art performance on both detection and segmentation of colorectal polyps. However, CNN based models need to not only be precise in order to be helpful in a medical context. In addition, interpretability and uncertainty in predictions must be well understood. In this paper, we develop and evaluate recent advances in uncertainty estimation and model interpretability in the context of semantic segmentation of polyps from colonoscopy images. Furthermore, we propose a novel method for estimating the uncertainty associated with important features in the input and demonstrate how interpretability and uncertainty can be modeled in DSSs for semantic segmentation of colorectal polyps. Results indicate that deep models are utilizing the shape and edge information of polyps to make their prediction. Moreover, inaccurate predictions show a higher degree of uncertainty compared to precise predictions.
Medical Image Anal.
Kristoffer Wickstrom Michael C Kampffmeyer Robert Jenssen
198179466
Context Aware Convolutional Neural Network for Grading of Colorectal Cancer Histology Images
Digital histology images are amenable to the application of convolutional neural networks (CNNs) for analysis due to the sheer size of pixel data present in them. CNNs are generally used for representation learning from small image patches (e.g. <inline formula> <tex math notation="LaTeX"$224\times 224$ /tex math>/inline formula> extracted from digital histology images due to computational and memory constraints. However, this approach does not incorporate high resolution contextual information in histology images. We propose a novel way to incorporate a larger context by a context aware neural network based on images with a dimension of <inline formula> <tex math notation="LaTeX"$1792\times 1792$ /tex math>/inline formula> pixels. The proposed framework first encodes the local representation of a histology image into high dimensional features then aggregates the features by considering their spatial organization to make a final prediction. We evaluated the proposed method on two colorectal cancer datasets for the task of cancer grading. Our method outperformed the traditional patch based approaches, problem specific methods, and existing context based methods. We also presented a comprehensive analysis of different variants of the proposed method.
IEEE Transactions on Medical Imaging
Muhammad Shaban Ruqayya Awan Muhammad Moazam Fraz Ayesha Azam Yee-Wah Tsang David R J Snead Nasir M Rajpoot
49473566
3D RoI aware U Net for Accurate and Efficient Colorectal Tumor Segmentation
Segmentation of colorectal cancerous regions from 3 D magnetic resonance (MR) images is a crucial procedure for radiotherapy. Automatic delineation from 3 D whole volumes is in urgent demand yet very challenging. Drawbacks of existing deep learning based methods for this task are two fold: 1) extensive graphics processing unit (GPU) memory footprint of 3 D tensor limits the trainable volume size, shrinks effective receptive field, and therefore, degrades speed and segmentation performance and 2) in region segmentation methods supported by region of interest (RoI) detection are either blind to global contexts, detail richness compromising, or too expensive for 3 D tasks. To tackle these drawbacks, we propose a novel encoder decoder based framework for 3 D whole volume segmentation, referred to as 3 D RoI aware U Net (3 D RU Net) 3 D RU Net fully utilizes the global contexts covering large effective receptive fields. Specifically, the proposed model consists of a global image encoder for global understanding based RoI localization, and a local region decoder that operates on pyramid shaped in region global features, which is GPU memory efficient and thereby enables training and prediction with large 3 D whole volumes. To facilitate the global to local learning procedure and enhance contour detail richness, we designed a dice based multitask hybrid loss function. The efficiency of the proposed framework enables an extensive model ensemble for further performance gain at acceptable extra computational costs. Over a dataset of 64 T2 weighted MR images, the experimental results of four fold cross validation show that our method achieved 75.5% dice similarity coefficient (DSC) in 0.61 s per volume on a GPU, which significantly outperforms competing methods in terms of accuracy and efficiency. The code is publicly available.
IEEE transactions on cybernetics
Yijie Huang Qi Dou Zi-Xian Wang Li-Zhi Liu Ying Jin Chao-feng Li Lisheng Wang Hao Chen Rui-hua Xu
216131212
Cellular community detection for tissue phenotyping in colorectal cancer histology images
Classification of various types of tissue in cancer histology images based on the cellular compositions is an important step towards the development of computational pathology tools for systematic digital profiling of the spatial tumor microenvironment. Most existing methods for tissue phenotyping are limited to the classification of tumor and stroma and require large amount of annotated histology images which are often not available. In the current work, we pose the problem of identifying distinct tissue phenotypes as finding communities in cellular graphs or networks. First, we train a deep neural network for cell detection and classification into five distinct cellular components. Considering the detected nuclei as nodes, potential cell cell connections are assigned using Delaunay triangulation resulting in a cell level graph. Based on this cell graph, a feature vector capturing potential cell cell connection of different types of cells is computed. These feature vectors are used to construct a patch level graph based on chi square distance. We map patch level nodes to the geometric space by representing each node as a vector of geodesic distances from other nodes in the network and iteratively drifting the patch nodes in the direction of positive density gradients towards maximum density regions. The proposed algorithm is evaluated on a publicly available dataset and another new large scale dataset consisting of 280K patches of seven tissue phenotypes. The estimated communities have significant biological meanings as verified by the expert pathologists. A comparison with current state of the art methods reveals significant performance improvement in tissue phenotyping.
Medical Image Anal.
Sajid Javed Arif Mahmood Muhammad Moazam Fraz Navid Alemi Koohbanani Ksenija Benes Yee-Wah Tsang Katherine Jane Hewitt David B A Epstein David R J Snead Nasir M Rajpoot
53792619
Lineage tracking reveals dynamic relationships of T cells in colorectal cancer
T cells are key elements of cancer immunotherapy1 but certain fundamental properties, such as the development and migration of T cells within tumours, remain unknown. The enormous T cell receptor (TCR) repertoire, which is required for the recognition of foreign and self antigens2, could serve as lineage tags to track these T cells in tumours3. Here we obtained transcriptomes of 11,138 single T cells from 12 patients with colorectal cancer, and developed single T cell analysis by RNA sequencing and TCR tracking (STARTRAC) indices to quantitatively analyse the dynamic relationships among 20 identified T cell subsets with distinct functions and clonalities. Although both CD8+ effector and 'exhausted' T cells exhibited high clonal expansion, they were independently connected with tumour resident CD8+ effector memory cells, implicating a TCR based fate decision. Of the CD4+ T cells, most tumour infiltrating T regulatory (Treg) cells showed clonal exclusivity, whereas certain Treg cell clones were developmentally linked to several T helper (TH) cell clones. Notably, we identified two IFNG+ TH1 like cell clusters in tumours that were associated with distinct IFNg regulating transcription factors the GZMK+ effector memory T cells, which were associated with EOMES and RUNX3, and CXCL13+BHLHE40+ TH1 like cell clusters, which were associated with BHLHE40. Only CXCL13+BHLHE40+ TH1 like cells were preferentially enriched in patients with microsatellite instable tumours, and this might explain their favourable responses to immune checkpoint blockade. Furthermore, IGFLR1 was highly expressed in both CXCL13+BHLHE40+ TH1 like cells and CD8+ exhausted T cells and possessed co stimulatory functions. Our integrated STARTRAC analyses provide a powerful approach to dissect the T cell properties in colorectal cancer comprehensively, and could provide insights into the dynamic relationships of T cells in other cancers.An integrated RNA sequencing approach demonstrates that CXCL13+ TH1 like cells are preferentially enriched in microsatellite instable tumours from patients with colorectal cancer, and IGFLR1 is identified as a co stimulatory molecule.
Nature
Lei Zhang Xin Yu Liangtao Zheng Yuanyuan Zhang Yansen Li Qiao Fang Ranran Gao Boxi Kang Qiming Zhang Julie Y Huang Hiroyasu Konno Xinyi Guo Ying-jiang Ye Songyuan Gao Shan Wang Xueda Hu Xianwen Ren Zhanlong Shen Wenjun Ouyang Zemin Zhang
202537935
CGC Net: Cell Graph Convolutional Network for Grading of Colorectal Cancer Histology Images
Colorectal cancer (CRC) grading is typically carried out by assessing the degree of gland formation within histology images. To do this, it is important to consider the overall tissue micro environment by assessing the cell level information along with the morphology of the gland. However, current automated methods for CRC grading typically utilise small image patches and therefore fail to incorporate the entire tissue micro architecture for grading purposes. To overcome the challenges of CRC grading, we present a novel cell graph convolutional neural network (CGC Net) that converts each large histology image into a graph, where each node is represented by a nucleus within the original image and cellular interactions are denoted as edges between these nodes according to node similarity. The CGC Net utilises nuclear appearance features in addition to the spatial location of nodes to further boost the performance of the algorithm. To enable nodes to fuse multi scale information, we introduce Adaptive GraphSage, which is a graph convolution technique that combines multi level features in a data driven way. Furthermore, to deal with redundancy in the graph, we propose a sampling technique that removes nodes in areas of dense nuclear activity. We show that modeling the image as a graph enables us to effectively consider a much larger image (around 16x larger) than traditional patch based approaches and model the complex structure of the tissue micro environment. We construct cell graphs with an average of over 3,000 nodes on a large CRC histology image dataset and report state of the art results as compared to recent patch based as well as contextual patch based techniques, demonstrating the effectiveness of our method.
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
Yanning Zhou Simon Graham Navid Alemi Koohbanani Muhammad Shaban Pheng-Ann Heng Nasir M Rajpoot
211677666
Label free colorectal cancer screening using deep learning and spatial light interference microscopy (SLIM)
Current pathology workflow involves staining of thin tissue slices, which otherwise would be transparent, followed by manual investigation under the microscope by a trained pathologist. While the hematoxylin and eosin (H&E) stain is well established and a cost effective method for visualizing histology slides, its color variability across preparations and subjectivity across clinicians remain unaddressed challenges. To mitigate these challenges, recently we have demonstrated that spatial light interference microscopy (SLIM) can provide a path to intrinsic, objective markers, that are independent of preparation and human bias. Additionally, the sensitivity of SLIM to collagen fibers yields information relevant to patient outcome, which is not available in H&E. Here, we show that deep learning and SLIM can form a powerful combination for screening applications: training on 1,660 SLIM images of colon glands and validating on 144 glands, we obtained a benign vs. cancer classification accuracy of 99% We envision that the SLIM whole slide scanner presented here paired with artificial intelligence algorithms may prove valuable as a pre screening method, economizing the clinician's time and effort.
APL photonics
Jingfang Zhang Yuchen R He Nahil Atef Sobh Gabriel Alexandru Popescu
165163501
Colorectal tumor on a chip system: A 3D tool for precision onco nanomedicine
This colorectal tumor on chip model is an original and reliable platform for high throughput drug screening and genetic expression. Awareness that traditional two dimensional (2D) in vitro and nonrepresentative animal models may not completely emulate the 3D hierarchical complexity of tissues and organs is on the rise. Therefore, posterior translation into successful clinical application is compromised. To address this dearth, on chip biomimetic microenvironments powered by microfluidic technologies are being developed to better capture the complexity of in vivo pathophysiology. Here, we describe a "tumor on a chip" model for assessment of precision nanomedicine delivery on which we validate the efficacy of drug loaded nanoparticles in a gradient fashion. The model validation was performed by viability studies integrated with live imaging to confirm the dose response effect of cells exposed to the CMCht/PAMAM nanoparticle gradient. This platform also enables the analysis at the gene expression level, where a down regulation of all the studied genes (MMP 1, Caspase 3, and Ki 67) was observed. This tumor on chip model represents an important development in the use of precision nanomedicine toward personalized treatment.
Science Advances
Mariana Carvalho David Barata Liliana Moreira Teixeira Stefan Giselbrecht Rui Luis Reis J Miguel Oliveira Roman K Truckenmuller Pamela Habibovic
211047530
Predicting colorectal cancer risk from adenoma detection via a two type branching process model
Despite advances in the modeling and understanding of colorectal cancer development, the dynamics of the progression from benign adenomatous polyp to colorectal carcinoma are still not fully resolved. To take advantage of adenoma size and prevalence data in the National Endoscopic Database of the Clinical Outcomes Research Initiative (CORI) as well as colorectal cancer incidence and size data from the Surveillance Epidemiology and End Results (SEER) database, we construct a two type branching process model with compartments representing adenoma and carcinoma cells. To perform parameter inference we present a new large size approximation to the size distribution of the cancer compartment and validate our approach on simulated data. By fitting the model to the CORI and SEER data, we learn biologically relevant parameters, including the transition rate from adenoma to cancer. The inferred parameters allow us to predict the individualized risk of the presence of cancer cells for each screened patient. We provide a web application which allows the user to calculate these individual probabilities at https:/ccrc eth.shinyapps.io/CCRC/ For example, we find a 1 in 100 chance of cancer given the presence of an adenoma between 10 and 20mm size in an average risk patient at age 50. We show that our two type branching process model recapitulates the early growth dynamics of colon adenomas and cancers and can recover epidemiological trends such as adenoma prevalence and cancer incidence while remaining mathematically and computationally tractable.
PLoS Comput. Biol.
Brian Matthew Lang Jack Kuipers Benjamin Misselwitz Niko Beerenwinkel
221698095
Deep learning to find colorectal polyps in colonoscopy: A systematic literature review
Colorectal cancer has a great incidence rate worldwide, but its early detection significantly increases the survival rate. Colonoscopy is the gold standard procedure for diagnosis and removal of colorectal lesions with potential to evolve into cancer and computer aided detection systems can help gastroenterologists to increase the adenoma detection rate, one of the main indicators for colonoscopy quality and predictor for colorectal cancer prevention. The recent success of deep learning approaches in computer vision has also reached this field and has boosted the number of proposed methods for polyp detection, localization and segmentation. Through a systematic search, 35 works have been retrieved. The current systematic review provides an analysis of these methods, stating advantages and disadvantages for the different categories used; comments seven publicly available datasets of colonoscopy images; analyses the metrics used for reporting and identifies future challenges and recommendations. Convolutional neural networks are the most used architecture together with an important presence of data augmentation strategies, mainly based on image transformations and the use of patches. End to end methods are preferred over hybrid methods, with a rising tendency. As for detection and localization tasks, the most used metric for reporting is the recall, while Intersection over Union is highly used in segmentation. One of the major concerns is the difficulty for a fair comparison and reproducibility of methods. Even despite the organization of challenges, there is still a need for a common validation framework based on a large, annotated and publicly available database, which also includes the most convenient metrics to report results. Finally, it is also important to highlight that efforts should be focused in the future on proving the clinical value of the deep learning based methods, by increasing the adenoma detection rate.
Artif. Intell. Medicine
Luisa F Sanchez-Peralta Luis Bote-Curiel Artzai Picon Francisco Miguel Sanchez-Margallo Jose B Pagador