tumor segmentation
215404153
Analyzing the Quality and Challenges of Uncertainty Estimations for Brain Tumor Segmentation
Automatic segmentation of brain tumors has the potential to enable volumetric measures and high throughput analysis in the clinical setting. Reaching this potential seems almost achieved, considering the steady increase in segmentation accuracy. However, despite segmentation accuracy, the current methods still do not meet the robustness levels required for patient centered clinical use. In this regard, uncertainty estimates are a promising direction to improve the robustness of automated segmentation systems. Different uncertainty estimation methods have been proposed, but little is known about their usefulness and limitations for brain tumor segmentation. In this study, we present an analysis of the most commonly used uncertainty estimation methods in regards to benefits and challenges for brain tumor segmentation. We evaluated their quality in terms of calibration, segmentation error localization, and segmentation failure detection. Our results show that the uncertainty methods are typically well calibrated when evaluated at the dataset level. Evaluated at the subject level, we found notable miscalibrations and limited segmentation error localization (e.g. for correcting segmentations) which hinder the direct use of the voxel wise uncertainties. Nevertheless, voxel wise uncertainty showed value to detect failed segmentations when uncertainty estimates are aggregated at the subject level. Therefore, we suggest a careful usage of voxel wise uncertainty measures and highlight the importance of developing solutions that address the subject level requirements on calibration and segmentation error localization.
Frontiers in Neuroscience
Alain Jungo Fabian Balsiger Mauricio Reyes
58004589
The Liver Tumor Segmentation Benchmark (LiTS)
In this work, we report the set up and results of the Liver Tumor Segmentation Benchmark (LITS) organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2016 and International Conference On Medical Image Computing Computer Assisted Intervention (MICCAI) 2017. Twenty four valid state of the art liver and liver tumor segmentation algorithms were applied to a set of 131 computed tomography (CT) volumes with different types of tumor contrast levels (hyper /hypo intense) abnormalities in tissues (metastasectomie) size and varying amount of lesions. The submitted algorithms have been tested on 70 undisclosed volumes. The dataset is created in collaboration with seven hospitals and research institutions and manually reviewed by independent three radiologists. We found that not a single algorithm performed best for liver and tumors. The best liver segmentation algorithm achieved a Dice score of 0.96(MICCAI) whereas for tumor segmentation the best algorithm evaluated at 0.67(ISBI) and 0.70(MICCAI) The LITS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.
ArXiv
Patrick Bilic Patrick Ferdinand Christ Eugene Vorontsov Grzegorz Chlebus Hao Chen Qi Dou Chi-Wing Fu Xiao Han Pheng-Ann Heng Jurgen Hesser Samuel Kadoury Tomasz K Konopczynski M Le Chunming Li X Li Jana Lipkova John S Lowengrub Hans Meine Jan Hendrik Moltz Christopher Joseph Pal Marie Piraud Xiaojuan Qi Jin Qi Markus Rempfler Karsten Roth Andrea Schenk Anjany Kumar Sekuboyina Ping Zhou Christian Hulsemeyer Marcel Beetz Florian Ettlinger Felix Grun G Kaissis Fabian K Lohofer Rickmer F Braren Julian Walter Holch Felix Hofmann Wieland H Sommer Volker Heinemann Colin Jacobs Gabriel Efrain Humpire Mamani Bram van Ginneken Gabriel Chartrand An Tang Michal Drozdzal Avi Ben-Cohen Eyal Klang Michal Marianne Amitai Eli Konen Hayit Greenspan Johan Moreau Alexandre Hostettler Luc Soler Refael Vivanti Adi Szeskin Naama Lev-Cohain Jacob Sosna Leo Joskowicz Bjoern H Menze
174799359
One Pass Multi Task Networks With Cross Task Guided Attention for Brain Tumor Segmentation
Class imbalance has emerged as one of the major challenges for medical image segmentation. The model cascade (MC) strategy, a popular scheme, significantly alleviates the class imbalance issue via running a set of individual deep models for coarse to fine segmentation. Despite its outstanding performance, however, this method leads to undesired system complexity and also ignores the correlation among the models. To handle these flaws in the MC approach, we propose in this paper a light weight deep model, i.e. the One pass Multi task Network (OM Net) to solve class imbalance better than MC does, while requiring only one pass computation for brain tumor segmentation. First, OM Net integrates the separate segmentation tasks into one deep model, which consists of shared parameters to learn joint features, as well as task specific parameters to learn discriminative features. Second, to more effectively optimize OM Net, we take advantage of the correlation among tasks to design both an online training data transfer strategy and a curriculum learning based training strategy. Third, we further propose sharing prediction results between tasks, which enables us to design a cross task guided attention (CGA) module. By following the guidance of the prediction results provided by the previous task, CGA can adaptively recalibrate channel wise feature responses based on the category specific statistics. Finally, a simple yet effective post processing method is introduced to refine the segmentation results of the proposed attention network. Extensive experiments are conducted to demonstrate the effectiveness of the proposed techniques. Most impressively, we achieve state of the art performance on the BraTS 2015 testing set and BraTS 2017 online validation set. Using these proposed approaches, we also won joint third place in the BraTS 2018 challenge among 64 participating teams. The code is publicly available at https:/github.com/chenhong zhou/OM Net.
IEEE Transactions on Image Processing
Chenhong Zhou Changxing Ding Xinchao Wang Zhentai Lu Dacheng Tao
149763213
Brain tumor segmentation with deep convolutional symmetric neural network
Abstract Gliomas are the most frequent primary brain tumors, which have a high mortality. Surgery is the most commonly used treatment. Magnetic resonance imaging (MRI) is especially useful to assess gliomas and evaluate the success of the treatment in clinical practice. So accurately segmenting the brain tumor from MRI images is the key to clinical diagnostics and treatment planning. However, a large quantity of data produced by MRI prevents manual segmentation in a reasonable time. So automatic approaches are required for quick and effective segmentation. But the spatial and structural variability among brain tumors bring a challenge to automatically segment MRI images. Deep Convolutional Neural Network (DCNN) can achieve complex function mapping. This contributes to response to the challenge. Therefore, DCNN is the commonly used method in brain tumor segmentation. Although a large number of DCNN based methods have been proposed, these methods mainly aim to improve the quality of the image features extracted by DCNN. Actually, these methods usually ignore the prior knowledge in medical images, and the simple prior knowledge in brain tumor segmentation is that most of the tumor regions in images are left right asymmetry. Based on this, in this paper, a novel deep convolutional neural network which combines symmetry have been proposed to automatically segment brain tumors. Our neural networks, called Deep Convolutional Symmetric Neural Network (DCSNN) extends DCNN based segmentation networks by adding symmetric masks in several layers. Our proposal was validated in the BRATS 2015 database, and we also give some baselines. The results are evaluated by dice similarity coefficient metric (DSC) And our proposed method achieved a competitive result with average DSC of 0.852. Furthermore, our method only takes about 10.8 s to segment a patient case. Although our method is not the best performance in BRATS 2015 challenge, as far as we know, our method outperforms other recent DCNN based methods.
Neurocomputing
Hao Chen Zhiguang Qin Yi Ding Tian Lan Zhen Qin
226217811
Breast Tumor Segmentation in 3D Automatic Breast Ultrasound Using Mask Scoring R CNN.
PURPOSE Automatic breast ultrasound (ABUS) imaging has become an essential tool in breast cancer diagnosis since it provides complementary information to other imaging modalities. Lesion segmentation on ABUS is a prerequisite step of breast cancer computer aided diagnosis (CAD) This work aims to develop a deep learning based method for breast tumor segmentation using three dimensional ABUS automatically. METHODS For breast tumor segmentation in ABUS, we developed a Mask scoring region based convolutional neural network (R CNN) that consists of five subnetworks, i.e. a backbone, a regional proposal network, a region CNN head, a mask head, and a mask score head. A network block building direct correlation between mask quality and region class was integrated into a Mask scoring R CNN based framework for the segmentation of new ABUS images with ambiguous regions of interest (ROIs) For segmentation accuracy evaluation, we retrospectively investigated 70 patients with breast tumor confirmed with needle biopsy and manually delineated on ABUS, of which 40 were used for five fold cross validation and 30 were used for hold out test. The comparison between the automatic breast tumor segmentations and the manual contours was quantified by I) six metrics including Dice similarity coefficient (DSC) Jaccard index, 95% Hausdorff distance (HD95) mean surface distance (MSD) residual mean square distance (RMSD) and center of mass distance (CMD) II) Pearson correlation analysis and Bland Altman analysis. RESULTS The mean (median) DSC was 85% 10.4% (89.4% and 82.1% 14.5% (85.6% for cross validation and hold out test, respectively. The corresponding HD95, MSD, RMSD and CMD of the two tests was 1.646 1.191 and 1.665 1.129 mm, 0.489 0.406 and 0.475 0.371 mm, 0.755 0.755 and 0.751 0.508 mm, and 0.672 0.612 and 0.665 0.729 mm. The mean volumetric difference (mean and 1.96 standard deviation) was 0.47 cc 0.77, 1.71) for the cross validation and 0.23 cc 0.23 0.69] for hold out test respectively. CONCLUSION We developed a novel Mask scoring R CNN approach for the automated segmentation of the breast tumor in ABUS images and demonstrated its accuracy for breast tumor segmentation. Our learning based method can potentially assist the clinical CAD of breast cancer using 3D ABUS imaging.
Medical physics
Yang Lei Xiuxiu He Jincao Yao Tonghe Wang Lijing Wang Wei Li Walter J Curran Tian Liu Dong-Guo Xu Xiaofeng Yang
219095543
Brain tumor segmentation and grading of lower grade glioma using deep learning in MRI images
Gliomas are the most common malignant brain tumors with different grades that highly determine the rate of survival in patients. Tumor segmentation and grading using magnetic resonance imaging (MRI) are common and essential for diagnosis and treatment planning. To achieve this clinical need, a deep learning approach that combines convolutional neural networks (CNN) based on the U net for tumor segmentation and transfer learning based on a pre trained convolution base of Vgg16 and a fully connected classifier for tumor grading was developed. The segmentation and grading models use the same pipeline of T1 precontrast, fluid attenuated inversion recovery (FLAIR) and T1 postcontrast MRI images of 110 patients of lower grade glioma (LGG) for training and evaluations. The mean dice similarity coefficient (DSC) and tumor detection accuracy achieved by the segmentation model are 0.84 and 0.92, respectively. The grading model classifies LGG into grade II and grade III with accuracy, sensitivity, and specificity of 0.89, 0.87, and 0.92, respectively at the MRI images' level and 0.95, 0.97, and 0.98 at the patients' level. This work demonstrates the potential of using deep learning in MRI images to provide a non invasive tool for simultaneous and automated tumor segmentation, detection, and grading of LGG for clinical applications.
Comput. Biol. Medicine
Mohamed A Naser M Jamal Deen
149788569
Deep learning for variational multimodality tumor segmentation in PET/CT
Positron emission tomography/computed tomography (PET/CT) imaging can simultaneously acquire functional metabolic information and anatomical information of the human body. How to rationally fuse the complementary information in PET/CT for accurate tumor segmentation is challenging. In this study, a novel deep learning based variational method was proposed to automatically fuse multimodality information for tumor segmentation in PET/CT. A 3D fully convolutional network (FCN) was first designed and trained to produce a probability map from the CT image. The learnt probability map describes the probability of each CT voxel belonging to the tumor or the background, and roughly distinguishes the tumor from its surrounding soft tissues. A fuzzy variational model was then proposed to incorporate the probability map and the PET intensity image for an accurate multimodality tumor segmentation, where the probability map acted as a membership degree prior. A split Bregman algorithm was used to minimize the variational model. The proposed method was validated on a non small cell lung cancer dataset with 84 PET/CT images. Experimental results demonstrated that: 1) Only a few training samples were needed for training the designed network to produce the probability map; 2) The proposed method can be applied to small datasets, normally seen in clinic research; 3) The proposed method successfully fused the complementary information in PET/CT, and outperformed two existing deep learning based multimodality segmentation methods and other multimodality segmentation methods using traditional fusion strategies (without deep learning) 4) The proposed method had a good performance for tumor segmentation, even for those with Fluorodeoxyglucose (FDG) uptake inhomogeneity and blurred tumor edges (two major challenges in PET single modality segmentation) and complex surrounding soft tissues (one major challenge in CT single modality segmentation) and achieved an average dice similarity indexes (DSI) of 0.86 0.05, sensitivity (SE) of 0.86 0.07, positive predictive value (PPV) of 0.87 0.10, volume error (VE) of 0.16 0.12, and classification error (CE) of 0.30 0.12.
Neurocomputing
Laquan Li Xiangming Zhao Wei Lu Shan Tan
204740265
Automated Brain Tumor Segmentation Using Multimodal Brain Scans: A Survey Based on Models Submitted to the BraTS 2012 2018 Challenges
Reliable brain tumor segmentation is essential for accurate diagnosis and treatment planning. Since manual segmentation of brain tumors is a highly time consuming, expensive and subjective task, practical automated methods for this purpose are greatly appreciated. But since brain tumors are highly heterogeneous in terms of location, shape, and size, developing automatic segmentation methods has remained a challenging task over decades. This paper aims to review the evolution of automated models for brain tumor segmentation using multimodal MR images. In order to be able to make a just comparison between different methods, the proposed models are studied for the most famous benchmark for brain tumor segmentation, namely the BraTS challenge [1] The BraTS 2012 2018 challenges and the state of the art automated models employed each year are analysed. The changing trend of these automated methods since 2012 are studied and the main parameters that affect the performance of different models are analysed.
IEEE Reviews in Biomedical Engineering
Mina Ghaffari Arcot Sowmya Ruth Oliver
204833519
Modified U Net (mU Net) With Incorporation of Object Dependent High Level Features for Improved Liver and Liver Tumor Segmentation in CT Images
Segmentation of livers and liver tumors is one of the most important steps in radiation therapy of hepatocellular carcinoma. The segmentation task is often done manually, making it tedious, labor intensive, and subject to intra /inter operator variations. While various algorithms for delineating organ at risks (OARs) and tumor targets have been proposed, automatic segmentation of livers and liver tumors remains intractable due to their low tissue contrast with respect to the surrounding organs and their deformable shape in CT images. The U Net has gained increasing popularity recently for image analysis tasks and has shown promising results. Conventional U Net architectures, however, suffer from three major drawbacks. First, skip connections allow for the duplicated transfer of low resolution information in feature maps to improve efficiency in learning, but this often leads to blurring of extracted image features. Secondly, high level features extracted by the network often do not contain enough high resolution edge information of the input, leading to greater uncertainty where high resolution edge dominantly affects the network's decisions such as liver and liver tumor segmentation. Thirdly, it is generally difficult to optimize the number of pooling operations in order to extract high level global features, since the number of pooling operations used depends on the object size. To cope with these problems, we added a residual path with deconvolution and activation operations to the skip connection of the U Net to avoid duplication of low resolution information of features. In the case of small object inputs, features in the skip connection are not incorporated with features in the residual path. Furthermore, the proposed architecture has additional convolution layers in the skip connection in order to extract high level global features of small object inputs as well as high level features of high resolution edge information of large object inputs. Efficacy of the modified U Net (mU Net) was demonstrated using the public dataset of Liver tumor segmentation (LiTS) challenge 2017. For liver tumor segmentation, Dice similarity coefficient (DSC) of 89.72 volume of error (VOE) of 21.93 and relative volume difference (RVD) of 0.49 were obtained. For liver segmentation, DSC of 98.51 VOE of 3.07 and RVD of 0.26 were calculated. For the public 3D Image Reconstruction for Comparison of Algorithm Database (3Dircadb) DSCs were 96.01 for the liver and 68.14 for liver tumor segmentations, respectively. The proposed mU Net outperformed existing state of art networks.
IEEE Transactions on Medical Imaging
Hyunseok Seo Charles Huang Maxime Bassenne Ruoxiu Xiao Lei Xing
212665307
Cascading handcrafted features and Convolutional Neural Network for IoT enabled brain tumor segmentation
Abstract The Internet of Things (IoT) has revolutionized the medical world by facilitating data acquisition using various IoT devices. These devices generate the data in multiple forms including text, images, and videos. Given this, the extraction of accurate and useful information from the massive serge IoT generated data is a highly challenging task. Recently, the brain tumor segmentation from IoT generated images has emerged as a promising issue that requires sophisticated and efficient techniques. The accurate brain tumor segmentation is challenging due to large variations in tumor appearance. Existing methods either use handcrafted features based techniques or Convolutional Neural Network (CNN) In this paper, a novel cascading approach for fully automatic brain tumor segmentation has been proposed, which intelligently combines handcrafted features and CNN. First, three handcrafted features are computed namely mean intensity, Local Binary Pattern (LBP) and Histogram of Oriented Gradients (HOG) and then Support Vector Machine (SVM) is employed to perform pixel classification that results in Confidence Surface Modality (CSM) This CSM along with the given Magnetic Resonance Imaging (MRI) is fed to a novel three pathways CNN architecture. In the experiments on BRATS 2015 dataset, the proposed method achieves promising results with Dice similarity scores of 0.81, 0.76 and 0.73 on complete, core and enhancing tumor, respectively.
Comput. Commun.
Hikmat Ullah Khan Pir Masoom Shah Munam Ali Shah Saif ul Islam Joel J P C Rodrigues