view classification radiology deep learning
211677432
Deep Learning for Musculoskeletal Image Analysis
The diagnosis, prognosis, and treatment of patients with musculoskeletal (MSK) disorders require radiology imaging (using computed tomography, magnetic resonance imaging (MRI) and ultrasound) and their precise analysis by expert radiologists. Radiology scans can also help assessment of metabolic health, aging, and diabetes. This study presents how machine learning, specifically deep learning methods, can be used for rapid and accurate image analysis of MRI scans, an unmet clinical need in MSK radiology. As a challenging example, we focus on automatic analysis of knee images from MRI scans and study machine learning classification of various abnormalities including meniscus and anterior cruciate ligament tears. Using widely used convolutional neural network (CNN) based architectures, we comparatively evaluated the knee abnormality classification performances of different neural network architectures under limited imaging data regime and compared single and multi view imaging when classifying the abnormalities. Promising results indicated the potential use of multi view deep learning based classification of MSK abnormalities in routine clinical assessment.
2019 53rd Asilomar Conference on Signals, Systems, and Computers
Ismail Irmakci Syed Muhammad Anwar Drew A Torigian Ulas Bagci
198148007
Automatic Radiology Report Generation based on Multi view Image Fusion and Medical Concept Enrichment
Generating radiology reports is time consuming and requires extensive expertise in practice. Therefore, reliable automatic radiology report generation is highly desired to alleviate the workload. Although deep learning techniques have been successfully applied to image classification and image captioning tasks, radiology report generation remains challenging in regards to understanding and linking complicated medical visual contents with accurate natural language descriptions. In addition, the data scales of open access datasets that contain paired medical images and reports remain very limited. To cope with these practical challenges, we propose a generative encoder decoder model and focus on chest x ray images and reports with the following improvements. First, we pretrain the encoder with a large number of chest x ray images to accurately recognize 14 common radiographic observations, while taking advantage of the multi view images by enforcing the cross view consistency. Second, we synthesize multi view visual features based on a sentence level attention mechanism in a late fusion fashion. In addition, in order to enrich the decoder with descriptive semantics and enforce the correctness of the deterministic medical related contents such as mentions of organs or diagnoses, we extract medical concepts based on the radiology reports in the training data and fine tune the encoder to extract the most frequent medical concepts from the x ray images. Such concepts are fused with each decoding step by a word level attention model. The experimental results conducted on the Indiana University Chest X Ray dataset demonstrate that the proposed model achieves the state of the art performance compared with other baseline approaches.
MICCAI
Jianbo Yuan Haofu Liao Rui Luo Jiebo Luo
4709730
CARS 2018 Computer Assisted Radiology and Surgery Proceedings of the 32nd International Congress and Exhibition Berlin, Germany, June 20 23, 2018
Advanced atherosclerosis in the coronary arteries is one of the leading causes of deaths worldwide while being preventable and treatable. In order to image atherosclerotic lesions (plaque) intravascular optical coherence tomography (IVOCT) can be used. The technique provides high resolution images of arterial walls which allows for early plaque detection by experts. Due to the vast amount of IVOCT images acquired in clinical routines, automatic plaque detection has been addressed. For example, attenuation profiles in single A Scans of IVOCT images are examined to detect plaque. We address automatic plaque classification from entire IVOCT images, the cross sectional view of the artery, using deep feature learning. In this way, we take context between A Scans into account and we directly learn relevant features from the image source without the need for handcrafting features.
International Journal of Computer Assisted Radiology and Surgery
Nils Gessert Markus Heyder Sarah Latus Matthias Lutz A Schlaefer
208646727
[Artificial intelligence and radiomics in MRI based prostate diagnostics]
CLINICAL/METHODICAL ISSUE In view of the diagnostic complexity and the large number of examinations, modern radiology is challenged to identify clinically significant prostate cancer (PCa) with high sensitivity and specificity. Meanwhile overdiagnosis and overtreatment of clinically nonsignificant carcinomas need to be avoided. STANDARD RADIOLOGICAL METHODS Increasingly, international guidelines recommend multiparametric magnetic resonance imaging (mpMRI) as first line investigation in patients with suspected PCa. METHODICAL INNOVATIONS Image interpretation according to the PI RADS criteria is limited by interobserver variability. Thus, rapid developments in the field of automated image analysis tools, including radiomics and artificial intelligence (AI; machine learning, deep learning) give hope for further improvement in patient care. PERFORMANCE AI focuses on the automated detection and classification of PCa, but it also attempts to stratify tumor aggressiveness according to the Gleason score. Recent studies present good to very good results in radiomics or AI supported mpMRI diagnosis. Nevertheless, these systems are not widely used in clinical practice. ACHIEVEMENTS AND PRACTICAL RECOMMENDATIONS In order to apply these innovative technologies, a growing awareness for the need of structured data acquisition, development of robust systems and an increased acceptance of AI as diagnostic support are needed. If AI overcomes these obstacles, it may play a key role in the quantitative and reproducible image based diagnosis of ever increasing prostate MRI examination volumes.
Der Radiologe
Charlie Alexander Hamm Nick Lasse Beetz Lynn Jeanette Savic Tobias Penzkofer
3852220
Fast and accurate view classification of echocardiograms using deep learning
Echocardiography is essential to cardiology. However, the need for human interpretation has limited echocardiography's full potential for precision medicine. Deep learning is an emerging tool for analyzing images but has not yet been widely applied to echocardiograms, partly due to their complex multi view format. The essential first step toward comprehensive computer assisted echocardiographic interpretation is determining whether computers can learn to recognize these views. We trained a convolutional neural network to simultaneously classify 15 standard views (12 video, 3 still) based on labeled still images and videos from 267 transthoracic echocardiograms that captured a range of real world clinical variation. Our model classified among 12 video views with 97.8% overall test accuracy without overfitting. Even on single low resolution images, accuracy among 15 views was 91.7% vs. 70.2 84.0% for board certified echocardiographers. Data visualization experiments showed that the model recognizes similarities among related views and classifies using clinically relevant image features. Our results provide a foundation for artificial intelligence assisted echocardiographic interpretation.Computer aided diagnosis: Model accurately classifies heart scan dataA computer model can accurately classify what heart anatomy is depicted on an ultrasound scan. Rima Arnaout from the University of California, San Francisco, and colleagues used a machine learning technique to teach a computer to recognize different types of video and still images produced by echocardiogram tests. After training the computer with real scans gathered from patients with a range of heart conditions, the researchers showed that their model could correctly classify what heart anatomy was shown in videos with 98% accuracy. The accuracy for still images fell to 92% but that was still above the 80% average achieved by four professional echocardiographers tested on the same dataset. Now that the researchers have achieved successful view classification, the next challenge, they say, will be comprehensive computer assisted interpretation of echocardiogram results.
npj Digital Medicine
Ali Madani Ramy Arnaout Mohammad R Kaazempur Mofrad Rima Arnaout
88476152
Designing lightweight deep learning models for echocardiography view classification
Transthoracic echocardiography (echo) is the most common imaging modality for diagnosis of cardiac conditions. Echo is acquired from a multitude of views, each of which distinctly highlights specific regions of the heart anatomy. In this paper, we present an approach based on knowledge distillation to obtain a highly accurate lightweight deep learning model for classification of 12 standard echocardiography views. The knowledge of several deep learning architectures based on the three common state of the art architectures, VGG 16, DenseNet, and Resnet, are distilled to train a set of lightweight models. Networks were developed and evaluated using a dataset of 16,612 echo cines obtained from 3,151 unique patients across several ultrasound imaging machines. The best accuracy of 89.0% is achieved by an ensemble of the three very deep models while we show an ensemble of lightweight models has a comparable accuracy of 88.1% The lightweight models have approximately 1% of the very deep model parameters and are six times faster in run time. Such lightweight view classification models could be used to build fast mobile applications for real time point of care ultrasound diagnosis.
Medical Imaging
Hooman Vaseli Zhibin Liao Amir Hossein Abdi Hany Girgis Delaram Behnami Christina L Luong Fatemeh Taheri Dezaki Neeraj Dhungel Robert Rohling Ken Gin Purang Abolmaesumi Teresa S M Tsang
212409801
View Classification and Object Detection in Cardiac Ultrasound to Localize Valves via Deep Learning
Echocardiography provides an important tool for clinicians to observe the function of the heart in real time, at low cost, and without harmful radiation. Automated localization and classification of heart valves enables automatic extraction of quantities associated with heart mechanical function and related blood flow measurements. We propose a machine learning pipeline that uses deep neural networks for separate classification and localization steps. As the first step in the pipeline, we apply view classification to echocardiograms with ten unique anatomic views of the heart. In the second step, we apply deep learning based object detection to both localize and identify the valves. Image segmentation based object detection in echocardiography has been shown in many earlier studies but, to the best of our knowledge, this is the first study that predicts the bounding boxes around the valves along with classification from 2D ultrasound images with the help of deep neural networks. Our object detection experiments suggest that it is possible to localize and identify multiple valves precisely.
Derya Gol Gungor Bimba Rao Cynthia Wolverton Ismayil Guracar
216645446
Clinically Feasible and Accurate View Classification of Echocardiographic Images Using Deep Learning
A proper echocardiographic study requires several video clips recorded from different acquisition angles for observation of the complex cardiac anatomy. However, these video clips are not necessarily labeled in a database. Identification of the acquired view becomes the first step of analyzing an echocardiogram. Currently, there is no consensus whether the mislabeled samples can be used to create a feasible clinical prediction model of ejection fraction (EF) The aim of this study was to test two types of input methods for the classification of images, and to test the accuracy of the prediction model for EF in a learning database containing mislabeled images that were not checked by observers. We enrolled 340 patients with five standard views (long axis, short axis, 3 chamber view, 4 chamber view and 2 chamber view) and 10 images in a cycle, used for training a convolutional neural network to classify views (total 17,000 labeled images) All DICOM images were rigidly registered and rescaled into a reference image to fit the size of echocardiographic images. We employed 5 fold cross validation to examine model performance. We tested models trained by two types of data, averaged images and 10 selected images. Our best model (from 10 selected images) classified video views with 98.1% overall test accuracy in the independent cohort. In our view classification model, 1.9% of the images were mislabeled. To determine if this 98.1% accuracy was acceptable for creating the clinical prediction model using echocardiographic data, we tested the prediction model for EF using learning data with a 1.9% error rate. The accuracy of the prediction model for EF was warranted, even with training data containing 1.9% mislabeled images. The CNN algorithm can classify images into five standard views in a clinical setting. Our results suggest that this approach may provide a clinically feasible accuracy level of view classification for the analysis of echocardiographic data.
Biomolecules
Kenya Kusunose Akihiro Haga Mizuki Inoue Daiju Fukuda Hirotsugu Yamada Masataka Sata
211684485
A Multi View Deep Learning Model for Encrypted Website Service Classification
This paper presents a multi view deep learning model for encrypted traffic classification which uses different neural network structures to process different types of features. Compared to previous work, our multi view deep learning model can make full use of different types of features extracted from the flow. We evaluate our deep learning model in the real world datasets acquired from the website servers of a large internet corporation. We found that the proposed multi view deep learning model gets better performance than the state of the art. Moreover, this paper focuses on a new traffic classification scenario in which the traffic under the specific website is classified on the basis of different website services. Each website service represents a specific website function which can be composed of several webpages. Therefore, the granularity of website service classification is between website classification and webpage classification.
2019 IEEE Global Communications Conference (GLOBECOM)
Kaiqi Liang Gaopeng Gou Cuicui Kang Chang Liu Min Yang Y Guo
215740500
Multi layer Joint Classification Metric Deep Learning for Top View Image Person Re identification
Person re identification (ReID) is the problem of identifying pedestrian images among multiple non overlapping cameras. Most of the researches are based on the front view or side view of the human body. However, there are many challenges, such as occlusion, posture changes. Therefore, based on top view images, we propose a person reidentification method, called ML CM (multi layer joint classification metric deep learning) The proposed method uses classification models and integrates them into the Siamese network, which is the unification of classification and metric learning. It is more suitable for top view images. In this way, minimum intra class variance and maximum inter class variance are achieved. We have collected the top view images of 1008 people as TVPR (top view person re identification) dataset and conducted experiments, demonstrating that the deep features obtained by our approach can get a competitive result.
2019 IEEE 2nd International Conference on Electronics and Communication Engineering (ICECE)
Shanshan Jiao Jiabao Wang Zhisong Pan Guyu Hu Junhua Zou Mingyong Zeng