U nET
60440399
MultiResUNet Rethinking the U Net Architecture for Multimodal Biomedical Image Segmentation
In recent years Deep Learning has brought about a breakthrough in Medical Image Segmentation. In this regard, U Net has been the most popular architecture in the medical imaging community. Despite outstanding overall performance in segmenting multimodal medical images, through extensive experimentations on some challenging datasets, we demonstrate that the classical U Net architecture seems to be lacking in certain aspects. Therefore, we propose some modifications to improve upon the already state of the art U Net model. Following these modifications, we develop a novel architecture, MultiResUNet, as the potential successor to the U Net architecture. We have tested and compared MultiResUNet with the classical U Net on a vast repertoire of multimodal medical images. Although only slight improvements in the cases of ideal images are noticed, remarkable gains in performance have been attained for the challenging ones. We have evaluated our model on five different datasets, each with their own unique challenges, and have obtained a relative improvement in performance of 10.15% 5.07% 2.63% 1.41% and 0.62% respectively. We have also discussed and highlighted some qualitatively superior aspects of MultiResUNet over classical U Net that are not really reflected in the quantitative measures.
Neural Networks
N Ibtehaz Mohammad Sohel Rahman
211572480
A U Net Based Discriminator for Generative Adversarial Networks
Among the major remaining challenges for generative adversarial networks (GANs) is the capacity to synthesize globally and locally coherent images with object shapes and textures indistinguishable from real images. To target this issue we propose an alternative U Net based discriminator architecture, borrowing the insights from the segmentation literature. The proposed U Net based architecture allows to provide detailed per pixel feedback to the generator while maintaining the global coherence of synthesized images, by providing the global image feedback as well. Empowered by the per pixel response of the discriminator, we further propose a per pixel consistency regularization technique based on the CutMix data augmentation, encouraging the U Net discriminator to focus more on semantic and structural changes between real and fake images. This improves the U Net discriminator training, further enhancing the quality of generated samples. The novel discriminator improves over the state of the art in terms of the standard distribution and image quality metrics, enabling the generator to synthesize images with varying structure, appearance and levels of detail, maintaining global and local realism. Compared to the BigGAN baseline, we achieve an average improvement of 2.7 FID points across FFHQ, CelebA, and the proposed COCO Animals dataset.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Edgar Schonfeld Bernt Schiele Anna Khoreva
215745370
Residual Attention U Net for Automated Multi Class Segmentation of COVID 19 Chest CT Images
The novel coronavirus disease 2019 (COVID 19) has been spreading rapidly around the world and caused significant impact on the public health and economy. However, there is still lack of studies on effectively quantifying the lung infection caused by COVID 19. As a basic but challenging task of the diagnostic framework, segmentation plays a crucial role in accurate quantification of COVID 19 infection measured by computed tomography (CT) images. To this end, we proposed a novel deep learning algorithm for automated segmentation of multiple COVID 19 infection regions. Specifically, we use the Aggregated Residual Transformations to learn a robust and expressive feature representation and apply the soft attention mechanism to improve the capability of the model to distinguish a variety of symptoms of the COVID 19. With a public CT image dataset, we validate the efficacy of the proposed algorithm in comparison with other competing methods. Experimental results demonstrate the outstanding performance of our algorithm for automated segmentation of COVID 19 Chest CT images. Our study provides a promising deep leaning based segmentation tool to lay a foundation to quantitative diagnosis of COVID 19 lung infection in CT images.
ArXiv
Xiaocong Chen Lina Yao Yanyan Zhang
204833519
Modified U Net (mU Net) With Incorporation of Object Dependent High Level Features for Improved Liver and Liver Tumor Segmentation in CT Images
Segmentation of livers and liver tumors is one of the most important steps in radiation therapy of hepatocellular carcinoma. The segmentation task is often done manually, making it tedious, labor intensive, and subject to intra /inter operator variations. While various algorithms for delineating organ at risks (OARs) and tumor targets have been proposed, automatic segmentation of livers and liver tumors remains intractable due to their low tissue contrast with respect to the surrounding organs and their deformable shape in CT images. The U Net has gained increasing popularity recently for image analysis tasks and has shown promising results. Conventional U Net architectures, however, suffer from three major drawbacks. First, skip connections allow for the duplicated transfer of low resolution information in feature maps to improve efficiency in learning, but this often leads to blurring of extracted image features. Secondly, high level features extracted by the network often do not contain enough high resolution edge information of the input, leading to greater uncertainty where high resolution edge dominantly affects the network's decisions such as liver and liver tumor segmentation. Thirdly, it is generally difficult to optimize the number of pooling operations in order to extract high level global features, since the number of pooling operations used depends on the object size. To cope with these problems, we added a residual path with deconvolution and activation operations to the skip connection of the U Net to avoid duplication of low resolution information of features. In the case of small object inputs, features in the skip connection are not incorporated with features in the residual path. Furthermore, the proposed architecture has additional convolution layers in the skip connection in order to extract high level global features of small object inputs as well as high level features of high resolution edge information of large object inputs. Efficacy of the modified U Net (mU Net) was demonstrated using the public dataset of Liver tumor segmentation (LiTS) challenge 2017. For liver tumor segmentation, Dice similarity coefficient (DSC) of 89.72 volume of error (VOE) of 21.93 and relative volume difference (RVD) of 0.49 were obtained. For liver segmentation, DSC of 98.51 VOE of 3.07 and RVD of 0.26 were calculated. For the public 3D Image Reconstruction for Comparison of Algorithm Database (3Dircadb) DSCs were 96.01 for the liver and 68.14 for liver tumor segmentations, respectively. The proposed mU Net outperformed existing state of art networks.
IEEE Transactions on Medical Imaging
Hyunseok Seo Charles Huang Maxime Bassenne Ruoxiu Xiao Lei Xing
214612502
Attention U Net Based Adversarial Architectures for Chest X ray Lung Segmentation information
Chest X ray is the most common test among medical imaging modalities. It is applied for detection and differentiation of, among others, lung cancer, tuberculosis, and pneumonia, the last with importance due to the COVID 19 disease. Integrating computer aided detection methods into the radiologist diagnostic pipeline, greatly reduces the doctors' workload, increasing reliability and quantitative analysis. Here we present a novel deep learning approach for lung segmentation, a basic, but arduous task in the diagnostic pipeline. Our method uses state of the art fully convolutional neural networks in conjunction with an adversarial critic model. It generalized well to CXR images of unseen datasets with different patient profiles, achieving a final DSC of 97.5% on the JSRT dataset.
ADGN@ECAI
Guszt'av Ga'al Bal'azs Maga Andras Lukacs
202568314
Automated Muscle Segmentation from Clinical CT Using Bayesian U Net for Personalized Musculoskeletal Modeling
We propose a method for automatic segmentation of individual muscles from a clinical CT. The method uses Bayesian convolutional neural networks with the U Net architecture, using Monte Carlo dropout that infers an uncertainty metric in addition to the segmentation label. We evaluated the performance of the proposed method using two data sets: 20 fully annotated CTs of the hip and thigh regions and 18 partially annotated CTs that are publicly available from The Cancer Imaging Archive (TCIA) database. The experiments showed a Dice coefficient (DC) of 0.891+ 0.016 (mean+ std) and an average symmetric surface distance (ASD) of 0.994+ 0.230 mm over 19 muscles in the set of 20 CTs. These results were statistically significant improvements compared to the state of the art hierarchical multi atlas method which resulted in 0.845 0.031 DC and 1.556 0.444 mm ASD. We evaluated validity of the uncertainty metric in the multi class organ segmentation problem and demonstrated a correlation between the pixels with high uncertainty and the segmentation failure. One application of the uncertainty metric in active learning is demonstrated, and the proposed query pixel selection method considerably reduced the manual annotation cost for expanding the training data set. The proposed method allows an accurate patient specific analysis of individual muscle shapes in a clinical routine. This would open up various applications including personalization of biomechanical simulation and quantitative evaluation of muscle atrophy.
IEEE Transactions on Medical Imaging
Yuta Hiasa Yoshito Otake Masaki Takao Takeshi Ogawa Nobuhiko Sugano Yoshinobu Sato
67856288
Segmentation of roots in soil with U Net
Background Plant root research can provide a way to attain stress tolerant crops that produce greater yield in a diverse array of conditions. Phenotyping roots in soil is often challenging due to the roots being difficult to access and the use of time consuming manual methods. Rhizotrons allow visual inspection of root growth through transparent surfaces. Agronomists currently manually label photographs of roots obtained from rhizotrons using a line intersect method to obtain root length density and rooting depth measurements which are essential for their experiments. We investigate the effectiveness of an automated image segmentation method based on the U Net Convolutional Neural Network (CNN) architecture to enable such measurements. We design a data set of 50 annotated chicory Cichorium intybus L. root images which we use to train, validate and test the system and compare against a baseline built using the Frangi vesselness filter. We obtain metrics using manual annotations and line intersect counts. Results Our results on the held out data show our proposed automated segmentation system to be a viable solution for detecting and quantifying roots. We evaluate our system using 867 images for which we have obtained line intersect counts, attaining a Spearman rank correlation of 0.9748 and an $r^2$ r 2 of 0.9217. We also achieve an $F_1$ F 1 of 0.7 when comparing the automated segmentation to the manual annotations, with our automated segmentation system producing segmentations with higher quality than the manual annotations for large portions of the image. Conclusion We have demonstrated the feasibility of a U Net based CNN system for segmenting images of roots in soil and for replacing the manual line intersect method. The success of our approach is also a demonstration of the feasibility of deep learning in practice for small research groups needing to create their own custom labelled dataset from scratch.
Plant Methods
Abraham George Smith Jens Petersen Raghavendra Selvan Camilla Ruo Rasmussen
212692498
DENSE INception U net for medical image segmentation
BACKGROUND AND OBJECTIVE Convolutional neural networks (CNNs) play an important role in the field of medical image segmentation. Among many kinds of CNNs, the U net architecture is one of the most famous fully convolutional network architectures for medical semantic segmentation tasks. Recent work shows that the U net network can be substantially deeper thus resulting in improved performance on segmentation tasks. Though adding more layers directly into network is a popular way to make a network deeper, it may lead to gradient vanishing or redundant computation during training. METHODS A novel CNN architecture is proposed that integrates the Inception Res module and densely connecting convolutional module into the U net architecture. The proposed network model consists of the following parts: firstly, the Inception Res block is designed to increase the width of the network by replacing the standard convolutional layers; secondly, the Dense Inception block is designed to extract features and make the network more deep without additional parameters; thirdly, the down sampling block is adopted to reduce the size of feature maps to accelerate learning and the up sampling block is used to resize the feature maps. RESULTS The proposed model is tested on images of blood vessel segmentations from retina images, the lung segmentation of CT Data from the benchmark Kaggle datasets and the MRI scan brain tumor segmentation datasets from MICCAI BraTS 2017. The experimental results show that the proposed method can provide better performance on these two tasks compared with the state of the art algorithms. The results reach an average Dice score of 0.9857 in the lung segmentation. For the blood vessel segmentation, the results reach an average Dice score of 0.9582. For the brain tumor segmentation, the results reach an average Dice score of 0.9867. CONCLUSIONS The experiments highlighted that combining the inception module with dense connections in the U Net architecture is a promising approach for semantic medical image segmentation.
Comput. Methods Programs Biomed.
Ziang Zhang Chengdong Wu S Coleman D Kerr
210166617
Siam U Net: encoder decoder siamese network for knee cartilage tracking in ultrasound images
The tracking of the knee femoral condyle cartilage during ultrasound guided minimally invasive procedures is important to avoid damaging this structure during such interventions. In this study, we propose a new deep learning method to track, accurately and efficiently, the femoral condyle cartilage in ultrasound sequences, which were acquired under several clinical conditions, mimicking realistic surgical setups. Our solution, that we name Siam U Net, requires minimal user initialization and combines a deep learning segmentation method with a siamese framework for tracking the cartilage in temporal and spatio temporal sequences of 2D ultrasound images. Through extensive performance validation given by the Dice Similarity Coefficient, we demonstrate that our algorithm is able to track the femoral condyle cartilage with an accuracy which is comparable to experienced surgeons. It is additionally shown that the proposed method outperforms state of the art segmentation models and trackers in the localization of the cartilage. We claim that the proposed solution has the potential for ultrasound guidance in minimally invasive knee procedures.
Medical Image Anal.
Matteo Dunnhofer Maria Antico Fumio Sasazawa Yukio Takeda S M Camps Niki Martinel Christian Micheloni G Carneiro Davide Fontanarosa
215754529
An automatic COVID 19 CT segmentation based on U Net with attention mechanism
The coronavirus disease (COVID 19) pandemic has led a devastating effect on the global public health. Computed Tomography (CT) is an effective tool in the screening of COVID 19. It is of great importance to rapidly and accurately segment COVID 19 from CT to help diagnostic and patient monitoring. In this paper, we propose a U Net based segmentation network using attention mechanism. As not all the features extracted from the encoders are useful for segmentation, we propose to incorporate an attention mechanism to a U Net architecture to capture rich contextual relationships for better feature representations. In addition, the focal tversky loss is introduced to deal with small lesion segmentation. The experiment results, evaluated on a small dataset where only 100 CT slices are available, demonstrate the proposed method can achieve an accurate and rapid segmentation on COVID 19 segmentation. The obtained Dice Score, Sensitivity and Specificity are 69.1% 81.1% and 97.2% respectively.
Tongxue Zhou Stephane Canu Su Ruan