capsnet
210970500
Classification of LiDAR Data Combined Octave Convolution With Capsule Network
Light Detection and Ranging (LiDAR) data are widely used for high resolution land cover mapping, which can provide very valuable information about the height of the surveyed area for the discrimination of classes. In order to utilize the advantages of deep models for the classification of LiDAR derived features, a new classification algorithm combined Octave Convolution (OctConv) with Capsule Network (CapsNet) is proposed here to hierarchically extract robust and discriminant features of the input data, called as OctConv CapsNet. In the proposed approach, CapsNet captures the spatial information of the data, and OctConv processes separately for high and low frequency feature. OctConv is embedded in the primary capsule layer of CapsNet so that the proposed approach can make the most of both the spatial information and the high and low frequency information simultaneously. The proposed framework performs experiments on two LiDAR DSM datasets (i.e. Bayview Park and Recology datasets) The results show that, compared with the traditional deep convolution model, OctConv CapsNet can improve the classification accuracy of LiDAR DSM data, and when the number of training samples of the experiment is 800, the classification accuracies reached 96.12% and 96.79% on Bayview Park and Recology datasets, respectively.
IEEE Access
Haibin Wu Mengqing Cao Aili Wang Minhui Wang
211018692
Capsule Network Performance With Autonomous Navigation
Capsule Networks (CapsNets) have been proposed as an alternative to Convolutional Neural Networks (CNNs) This paper showcases how CapsNets are more capable than CNNs for autonomous agent exploration of realistic scenarios. In real world navigation, rewards external to agents may be rare. In turn, reinforcement learning algorithms can struggle to form meaningful policy functions. This paper's approach Capsules Exploration Module (Caps EM) pairs a CapsNets architecture with an Advantage Actor Critic algorithm. Other approaches for navigating sparse environments require intrinsic reward generators, such as the Intrinsic Curiosity Module (ICM) and Augmented Curiosity Modules (ACM) Caps EM uses a more compact architecture without need for intrinsic rewards. Tested using ViZDoom, the Caps EM uses 44% and 83% fewer trainable network parameters than the ICM and Depth Augmented Curiosity Module (D ACM) respectively, for 1141% and 437% average time improvement over the ICM and D ACM, respectively, for converging to a policy function across "My Way Home" scenarios.
ArXiv
Tom Molnar Eugenio Culurciello
211011099
Self attention based BiGRU and capsule network for named entity recognition
Named entity recognition(NER) is one of the tasks of natural language processing(NLP) In view of the problem that the traditional character representation ability is weak and the neural network method is unable to capture the important sequence information. An self attention based bidirectional gated recurrent unit(BiGRU) and capsule network(CapsNet) for NER is proposed. This model generates character vectors through bidirectional encoder representation of transformers(BERT) pre trained model. BiGRU is used to capture sequence context features, and self attention mechanism is proposed to give different focus on the information captured by hidden layer of BiGRU. Finally, we propose to use CapsNet for entity recognition. We evaluated the recognition performance of the model on two datasets. Experimental results show that the model has better performance without relying on external dictionary information.
ArXiv
Jianfeng Deng Lianglun Cheng Zhuowei Wang
208117836
Scaling Analysis of Specialized Tensor Processing Architectures for Deep Learning Models
Specialized tensor processing architectures (TPA) targeted for neural network processing has attracted a lot of attention in recent years. The computing complexity of the algorithmically different components of some deep neural networks (DNNs) was considered with regard to their further use on such TPAs. To demonstrate the crucial difference between TPU and GPU computing architectures, the real computing complexity of various algorithmically different DNNs was estimated by the proposed scaling analysis of time and speedup dependencies of training and inference times as functions of batch and image sizes. The main accent was made on the widely used and algorithmically different DNNs like VGG16, ResNet50, and CapsNet on the cloud based implementation of TPA (actually Google Cloud TPUv2) The results of performance study were demonstrated by the proposed scaling method for estimation of efficient usage of these DNNs on this infrastructure. The most important and intriguing results are the scale invariant behaviors of time and speedup dependencies which allow us to use the scaling method to predict the running training and inference times on the new specific TPAs without detailed information about their internals even. The scaling dependencies and scaling powers are different for algorithmically different DNNs (VGG16, ResNet50, CapsNet) and for architecturally different computing hardwares (GPU and TPU) These results give the precise estimation of the higher performance (throughput) of TPAs as Google TPUv2 in comparison to GPU for the large number of computations under conditions of low overhead calculations and high utilization of TPU units by means of the large image and batch sizes. In general, the usage of TPAs like Google TPUv2 is quantitatively proved to be promising tool for increasing performance of inference and training stages even, especially in the view of availability of the similar specific TPAs like TCU in Tesla V100 and Titan V provided by NVIDIA, and others.
Deep Learning: Concepts and Architectures
Yuri Gordienko Yuriy Kochura Vlad Taran Nikita Gordienko Alexandr Rokovyi Oleg Alienin Sergii Stirenko
210862123
1D CADCapsNet: One dimensional deep capsule networks for coronary artery disease detection using ECG signals.
PURPOSE Cardiovascular disease (CVD) is a leading cause of death globally. Electrocardiogram (ECG) which records the electrical activity of the heart, has been used for the diagnosis of CVD. The automated and robust detection of CVD from ECG signals plays a significant role for early and accurate clinical diagnosis. The purpose of this study is to provide automated detection of coronary artery disease (CAD) from ECG signals using capsule networks (CapsNet) METHODS Deep learning based approaches have become increasingly popular in computer aided diagnosis systems. Capsule networks are one of the new promising approaches in the field of deep learning. In this study, we used 1D version of CapsNet for the automated detection of coronary artery disease (CAD) on two second (95,300) and five second long (38,120) ECG segments. These segments are obtained from 40 normal and 7 CAD subjects. In the experimental studies, 5 fold cross validation technique is employed to evaluate performance of the model. RESULTS The proposed model, which is named as 1D CADCapsNet, yielded a promising 5 fold diagnosis accuracy of 99.44% and 98.62% for two and five second ECG signal groups, respectively. We have obtained the highest performance results using 2 s ECG segment than the state of art studies reported in the literature. CONCLUSIONS 1D CADCapsNet model automatically learns the pertinent representations from raw ECG data without using any hand crafted technique and can be used as a fast and accurate diagnostic tool to help cardiologists.
Physica medica PM an international journal devoted to the applications of physics to medicine and biology official journal of the Italian Association of Biomedical Physics
Ertan Butun Ozal Yildirim Muhammed Talo Ru San Tan U Rajendra Acharya
211215654
Detecting Pneumonia Using Convolutions and Dynamic Capsule Routing for Chest X ray Images
An entity's existence in an image can be depicted by the activity instantiation vector from a group of neurons (called capsule) Recently, multi layered capsules, called CapsNet, have proven to be state of the art for image classification tasks. This research utilizes the prowess of this algorithm to detect pneumonia from chest X ray (CXR) images. Here, an entity in the CXR image can help determine if the patient (whose CXR is used) is suffering from pneumonia or not. A simple model of capsules (also known as Simple CapsNet) has provided results comparable to best Deep Learning models that had been used earlier. Subsequently, a combination of convolutions and capsules is used to obtain two models that outperform all models previously proposed. These models Integration of convolutions with capsules (ICC) and Ensemble of convolutions with capsules (ECC) detect pneumonia with a test accuracy of 95.33% and 95.90% respectively. The latter model is studied in detail to obtain a variant called EnCC, where n 3, 4, 8, 16. Here, the E4CC model works optimally and gives test accuracy of 96.36% All these models had been trained, validated, and tested on 5857 images from Mendeley.
Sensors
Ansh Mittal Deepika Kumar Mamta Mittal Tanzila Saba Ibrahim Abunadi Amjad Rehman Sudipta Roy
211135806
Enhancement of surgical hand gesture recognition using a capsule network for a contactless interface in the operating room
BACKGROUND AND OBJECTIVE Hand gesture recognition systems in operating rooms (ORs) are crucial for browsing and controlling computer aided devices, which have been developed to decrease the risk of contamination during surgical procedures. METHODS We proposed the use of hand gesture recognition to enhance accuracies and recognition areas with the capsule network (CapsNet) of deep neural network and Leap Motiona Our method includes the i) extraction and preprocessing of infrared (IR) images (60 frames per second) from Leap Motion(tm) ii) training of various types of networks, and iii) gesture recognition evaluation in the OR. We trained the images of training dataset (N=903) and tested images (N=100) using five types of surgical hand gestures including hovering, grab, click, one peak, and two peaks by 10 subjects with various types of augmentation methods including rotate (0, 90, 180) scale, translation, illumination, and resize. RESULTS CapsNet achieved a classification accuracy of 86.46% (around 10% improvement) compared with 73.67% for the baseline convolutional neural network (CNN) and 76.4% for VGG16. CONCLUSIONS In conclusion, the accuracy of hand gesture recognition with CapsNet was better than that of conventional CNNs, which could be used to navigate and manipulate various types of computer aided devices and applications through contactless gesture interaction.
Comput. Methods Programs Biomed.
Areum Lee Yongwon Cho Seong Il Jin Namkug Kim
211058407
A Motor Imagery EEG Feature Extraction Method Based on Energy Principal Component Analysis and Deep Belief Networks
The motor imagery electroencephalography (MI EEG) reflects the subjective motor intention, which has received increasing attention in rehabilitation. How to extract the features of MI EEG accurately and quickly is the key to its successful application. Based on the analysis and comparison of the existing feature extraction algorithms, a feature extraction method based on principal component analysis (PCA) and deep belief networks (DBN) is proposed, namely PCA DBN. Firstly, the second order moment is used to analyze the time domain of MI EEG, select the effective time interval. Secondly, PCA is used to analyze the selected time domain interval and obtain the principal component feature points. Then, feature points are imported into DBN to realize the final feature extraction. Finally, use the softmax classifier to complete task classification. Perform algorithm validation on the BCI Competition II Data set III and BCI Competition IV Data sets 2b, classification accuracies are 96.25% and 91.71% kappa values are 0.925 and 0.8342. The paired sample t test with FDR correction is carried out on the verification results, and the comparison with some better classification algorithms shows that the algorithm has better performance. In the end, this method is used to extract the features of laboratory data, the optimal classification accuracy is 97.69% and kappa value is 0.9538, the validity of the method is further verified.
IEEE Access
Liwei Cheng Duanling Li Gongjing Yu Zhonghai Zhang Xiao-lin Li Shuyue Yu
211029605
Classification of Non Discriminant ERD/ERS Comprising Motor Imagery Electroencephalography Signals
Classification of Motor Imagery (MI) Electroencephalography (EEG) signals has always been an important aspect of Brain Computer Interface (BCI) systems. Event Related Desynchronization (ERD) Event Related Synchronization (ERS) plays a significant role in finding discriminant features of MI EEG signals. ERD/ERS is one type and Evoked Potential (EP) is another type of brain response. This study focuses upon the classification of MI EEG signals by Removing Evoked Potential (REP) from non discriminant MI EEG data in filter band selection, called REP. This optimization is done to enhance the classification performance. A comprehensive comparison of several pipelines is presented by using famous feature extraction methods, namely Common Spatial Pattern (CSP) XDawn. The effectiveness of REP is demonstrated on the PhysioNet dataset which is an online data resource. Comparison is done between the performance of pipelines including proposed one (Common Spatial Pattern (CSP) and Gaussian Process Classifier (GPC) as well as before and after applying REP. It is observed that the REP approach has improved the classification accuracy of all the subjects used as well as all the pipelines, including state of the art algorithms, up to 20%
Zaib unnisa Asi Mohamad T Sultan Umair Muneer Aneela Abbas Sadaf Ilyas
211027751
SBAG: A Hybrid Deep Learning Model for Large Scale Traffic Speed Prediction
Intelligent Transportation System (ITS) is the fundamental requirement to an intelligent transport system. The proposed hybrid model Stacked Bidirectional LSTM and Attention based GRU (SBAG) is used for predicting the large scale traffic speed. To capture bidirectional temporal dependencies and spatial features, BDLSTM and attention based GRU are exploited. It is the first time in traffic speed prediction that bidirectional LSTM and attention based GRU are exploited as a building block of network architecture to measure the backward dependencies of a network. We have also examined the behaviour of the attention layer in our proposed model. We compared the proposed model with state of the art models e.g. Fully Convolutional Network, Gated Recurrent Unit, Long short term Memory, Bidirectional Long short term Memory and achieved superior performance in large scale traffic speed prediction.
Adnan Riaz Muhammad Nabeel Mehak Khan Huma Jamil