bi-lstm
221178477
Predictions for COVID 19 with deep learning models of LSTM, GRU and Bi LSTM
Abstract COVID 19, responsible of infecting billions of people and economy across the globe, requires detailed study of the trend it follows to develop adequate short term prediction models for forecasting the number of future cases. In this perspective, it is possible to develop strategic planning in the public health system to avoid deaths as well as managing patients. In this paper, proposed forecast models comprising autoregressive integrated moving average (ARIMA) support vector regression (SVR) long shot term memory (LSTM) bidirectional long short term memory (Bi LSTM) are assessed for time series prediction of confirmed cases, deaths and recoveries in ten major countries affected due to COVID 19. The performance of models is measured by mean absolute error, root mean square error and r2_score indices. In the majority of cases, Bi LSTM model outperforms in terms of endorsed indices. Models ranking from good performance to the lowest in entire scenarios is Bi LSTM, LSTM, GRU, SVR and ARIMA. Bi LSTM generates lowest MAE and RMSE values of 0.0070 and 0.0077, respectively, for deaths in China. The best r2_score value is 0.9997 for recovered cases in China. On the basis of demonstrated robustness and enhanced prediction accuracy, Bi LSTM can be exploited for pandemic prediction for better planning and management.
Chaos, Solitons Fractals
Farah Shahid Aneela Zameer Muhammad Muneeb
208845734
Bi LSTM Network for Multimodal Continuous Human Activity Recognition and Fall Detection
This paper presents a framework based on multi layer bi LSTM network (bidirectional Long Short Term Memory) for multimodal sensor fusion to sense and classify daily activities' patterns and high risk events such as falls. The data collected in this work are continuous activity streams from FMCW radar and three wearable inertial sensors on the wrist, waist, and ankle. Each activity has a variable duration in the data stream so that the transitions between activities can happen at random times within the stream, without resorting to conventional fixed duration snapshots. The proposed bi LSTM implements soft feature fusion between wearable sensors and radar data, as well as two robust hard fusion methods using the confusion matrices of both sensors. A novel hybrid fusion scheme is then proposed to combine soft and hard fusion to push the classification performances to approximately 96% accuracy in identifying continuous activities and fall events. These fusion schemes implemented with the proposed bi LSTM network are compared with conventional sliding window approach, and all are validated with realistic "leaving one participant out" (L1PO) method (i.e. testing subjects unknown to the classifier) The developed hybrid fusion approach is capable of stabilizing the classification performance among different participants in terms of reducing accuracy variance of up to 18.1% and increasing minimum, worst case accuracy up to 16.2%
IEEE Sensors Journal
Haobo Li Aman Shrestha Hadi Heidari Julien Le Kernec Francesco Fioranelli
211123192
Constructing a PM2.5 concentration prediction model by combining auto encoder with Bi LSTM neural networks
Abstract Air pollution problems have a severe effect on the natural environment and public health. The application of machine learning to air pollutant data can result in a better understanding of environmental quality. Of these methods, the deep learning method has proven to be a very efficient and accurate method to forecast complex air quality data. This paper proposes a deep learning model based on an auto encoder and bidirectional long short term memory (Bi LSTM) to forecast PM2.5 concentrations to reveal the correlation between PM2.5 and multiple climate variables. The model comprises several aspects, including data preprocessing, auto encoder layer, and Bi LSTM layer. The performance of the proposed model was verified based on a real world air pollution dataset, and the results indicated this model can improve the prediction accuracy in an experimental scenario.
Environ. Model. Softw.
Bo Zhang Hanwen Zhang Gengming Zhao Jie Lian
220507228
Dynamic prediction of jet grouted column diameter in soft soil using Bi LSTM deep learning
The bidirectional long short term memory (Bi LSTM) network is an innovative computation paradigm that learns bidirectional long term dependencies between time steps and sequence data to predict future occurrences. This study proposes a framework to incorporate Bi LSTM and data sequencing to predict diameter of jet grouted columns in soft soil in real time. The models are tested using a case study of jet grouting treatment of soft soil. The results show that the proposed strategies can efficiently predict the variation in column diameter with the depth. A comparative performance analysis among the Bi LSTM, original long short term memory (LSTM) and support vector regression (SVR) approaches is also conducted. The Bi LSTM performs better than both the LSTM and SVR in root mean square error. This result substantiates the efficacy of modeling sequential step by step jet grouting process using the Bi LSTM. Based on the analyzed results, some recommendations for improving the current design of jet grout columns are proposed.
Shuilong Shen Pierre Guy Atangana Njock Annan Zhou Hai-Min Lyu
225242195
Bi LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism
There is a need to extract meaningful information from big data, classify it into different categories, and predict end user behavior or emotions. Large amounts of data are generated from various sources such as social media and websites. Text classification is a representative research topic in the field of natural language processing that categorizes unstructured text data into meaningful categorical classes. The long short term memory (LSTM) model and the convolutional neural network for sentence classification produce accurate results and have been recently used in various natural language processing (NLP) tasks. Convolutional neural network (CNN) models use convolutional layers and maximum pooling or max overtime pooling layers to extract higher level features, while LSTM models can capture long term dependencies between word sequences hence are better used for text classification. However, even with the hybrid approach that leverages the powers of these two deep learning models, the number of features to remember for classification remains huge, hence hindering the training process. In this study, we propose an attention based Bi LSTM+CNN hybrid model that capitalize on the advantages of LSTM and CNN with an additional attention mechanism. We trained the model using the Internet Movie Database (IMDB) movie review data to evaluate the performance of the proposed model, and the test results showed that the proposed hybrid attention Bi LSTM+CNN model produces more accurate classification results, as well as higher recall and F1 scores, than individual multi layer perceptron (MLP) CNN or LSTM models as well as the hybrid models.
Beakcheol Jang Myeonghwi Kim Gaspard Harerimana Sang-ug Kang Jong Wook Kim
204107924
An attention mechanism and multi granularity based Bi LSTM model for Chinese Q&A system
Natural language processing (NLP) is one of the key techniques in intelligent question answering (Q&A) systems. Although recurrent neural networks and long short term memory (LSTM) networks exhibit obvious advantages on well known English Q&A datasets, they still suffer from several defects including indeterminateness, polysemy and the lack of changing morphology in Chinese, which results in complex NLP on large and diverse Chinese Q&A datasets. In this paper, we first analyze limitations of applying LSTM and bidirectional LSTM (Bi LSTM) models to noisy Chinese Q&A datasets. Then, we focus on integrating attention mechanisms and multi granularity word segmentation into Bi LSTM and propose an attention mechanism and multi granularity based Bi LSTM model (AM Bi LSTM) which combines the improved attention mechanism with a novel processing of multi granularity word segmentation to handle the complex NLP in Chinese Q&A datasets. Furthermore, similarity of questions and answers is formulated to implement the quantitative computation which helps to achieve better performance in Chinese Q&A systems. Finally, we verify the proposed model on a noisy Chinese Q&A dataset. The experimental results demonstrate that the novel AM Bi LSTM model achieves significant improvement on evaluation metrics of accuracy, mean average precision and so on. Moreover, the experimental results indicate that the novel AM Bi LSTM model outperforms baseline methods and other LSTM based models.
Soft Comput.
Xiaomei Yu Wenxiao Feng Hong Wang Qian Chu Qi Chen
221679367
Continuous Human Activity Classification From FMCW Radar With Bi LSTM Networks
Recognition of human movements with radar for ambient activity monitoring is a developed area of research that yet presents outstanding challenges to address. In real environments, activities and movements are performed with seamless motion, with continuous transitions between activities of different duration and a large range of dynamic motions, compared with discrete activities of fixed time lengths which are typically analysed in the literature. This paper proposes a novel approach based on recurrent LSTM and Bi LSTM network architectures for continuous activity monitoring and classification. This approach uses radar data in the form of a continuous temporal sequence of micro Doppler or range time information, differently from from other conventional approaches based on convolutional networks that interpret the radar data as images. Experimental radar data involving 15 participants and different sequences of 6 actions are used to validate the proposed approach. It is demonstrated that using the Doppler domain data together with the Bi LSTM network and an optimal learning rate can achieve over 90% mean accuracy, whereas range domain data only achieved approximately 76% The details of the network architectures, insights in their behaviour as a function of key hyper parameters such as the learning rate, and a discussion on their performance across are provided in the paper.
IEEE Sensors Journal
Aman Shrestha Haobo Li Julien Le Kernec Francesco Fioranelli
216441189
Averaged Bi LSTM networks for RUL prognostics with non life cycle labeled dataset
Abstract LSTM network is an effective RNN model to predict the system RUL for its superior performance in sequential data processing. Usually, networks trained by life cycle labeled dataset would possess similar RUL predicting accuracies, because the network training algorithm could ensure the network optimality for the whole training dataset. However, for networks trained by non life cycle labeled samples, the network uncertainty caused by different training conditions could lead to degradation prediction uncertainty for some local points. Further, the RUL predicting results that are computed by these uncertain local points may shows relatively large differences. Therefore, in order to obtain an accurate RUL prediction with networks trained by non life cycle labeled samples, our paper proposes a novel network model averaging method to deal with the network uncertainty. What is more, to learn the temporal correlation information of training samples sufficiently, we adopt the Bi LSTM network to illustrate the application of the proposed network model averaging method. Finally, degradation values of Graphite/LiCoO2 battery are used to verify the effectiveness of the proposed method. The results show that the proposed method could improve the RUL prediction accuracy and reduce the prediction error.
Neurocomputing
Yong Yu Chang-Hua Hu Xiaosheng Si Jianfei Zheng Jian-xun Zhang
214683125
Movement Estimation Using Soft Sensors Based on Bi LSTM and Two Layer LSTM for Human Motion Capture
The importance of estimating human movement has increased in the field of human motion capture. HTC VIVE is a popular device that provides a convenient way of capturing human motions using several sensors. Recently, the motion of only users' hands has been captured, thereby greatly reducing the range of motion captured. This paper proposes a framework to estimate single arm orientations using soft sensors mainly by combining a Bi long short term memory (Bi LSTM) and two layer LSTM. Positions of the two hands are measured using an HTC VIVE set, and the orientations of a single arm, including its corresponding upper arm and forearm, are estimated using the proposed framework based on the estimated positions of the two hands. Given that the proposed framework is meant for a single arm, if orientations of two arms are required to be estimated, the estimations are performed twice. To obtain the ground truth of the orientations of single arm movements, two Myo gesture control sensory armbands are employed on the single arm: one for the upper arm and the other for the forearm. The proposed framework analyzed the contextual features of consecutive sensory arm movements, which provides an efficient way to improve the accuracy of arm movement estimation. In comparison with the ground truth, the proposed method estimated the arm movements using a dynamic time warping distance, which was the average of 73.90% less than that of a conventional Bayesian framework. The distinct feature of our proposed framework is that the number of sensors attached to end users is reduced. Additionally, with the use of our framework, the arm orientations can be estimated with any soft sensor, and good accuracy of the estimations can be ensured. Another contribution is the suggestion of the combination of the Bi LSTM and two layer LSTM.
Sensors
Haitao Guo Yunsick Sung
208131608
Attentive Stacked Denoising Autoencoder With Bi LSTM for Personalized Context Aware Citation Recommendation
The rapid growth of scientific publications brings the problem of finding appropriate citations for authors. Context aware citation recommendation is an essential technology to overcome this obstacle when given a fragment of manuscript. In this article, we propose a novel neural network model for context aware citation recommendation by combining stacked denoising autoencoders (SDAE) and Bi LSTM. To obtain effective embedding for cited paper, we extend SDAE into attentive SDAE (ASDAE) by utilizing the attentive information from citation context, which essentially enhance the learning ability of original SDAE. For citation context, we devise an attentive Bi LSTM to obtain effective embedding. Specifically, the attentive Bi LSTM is able to extract suitable citation context and recommend citations simultaneously when given a long text, which is a issue that few papers addressed before. We also integrate personalized author information to improve the performance of recommendation. Our model is essentially a seemly integration of different types of neural network with latent variables. We derive the generative process of our model, and develop a learning algorithm based on maximum a posteriori (MAP) estimation. Experimental results on the RefSeer, ANN and DBLP datasets show that our model outperforms baseline methods.
IEEE/ACM Transactions on Audio, Speech, and Language Processing
Tao Dai Li Zhu Yaxiong Wang Kathleen M Carley