Aspect category detection via topic- attention network
57572922
Aspect Category Detection via Topic Attention Network
The e commerce has started a new trend in natural language processing through sentiment analysis of user generated reviews. Different consumers have different concerns about various aspects of a specific product or service. Aspect category detection, as a subtask of aspect based sentiment analysis, tackles the problem of categorizing a given review sentence into a set of pre defined aspect categories. In recent years, deep learning approaches have brought revolutionary advances in multiple branches of natural language processing including sentiment analysis. In this paper, we propose a deep neural network method based on attention mechanism to identify different aspect categories of a given review sentence. Our model utilizes several attentions with different topic contexts, enabling it to attend to different parts of a review sentence based on different topics. Experimental results on two datasets in the restaurant domain released by SemEval workshop demonstrates that our approach outperforms existing methods on both datasets. Visualization of the topic attention weights shows the effectiveness of our model in identifying words related to different topics.
ArXiv
Sajad Movahedi Erfan Ghadery Heshaam Faili Azadeh Shakery
234477133
Utilizing BERT for Detecting Aspect Categories on TABSA via Adjusting Self attention among Words
Aspect based sentiment analysis (ABSA) has become a popular research topic in recent years due to its strong function of breaking down text into aspects and identifying sentiment polarity towards a specific target, generating a significant amount of discussion among researchers. Motivated by recent work of application of sentence pair classification task into ABSA, this article discusses how to further fine tune the pre trained Bidirectional Encoder Representations from Transformers (BERT) model and obtain the results on SentiHood dataset. In a contrast to the previous work, this article considers that the sentiment analysis has relations to every single word in each sentence and shows the process of modifying the forward network in BERT to create self attention between words. The proposed model demonstrates a certain degree of improvement in some aspects, in particular to aspect category detection.
2020 International Conference on Intelligent Computing and Human Computer Interaction (ICHCI)
Hong Hong Jiawen Song
221507538
Multivariate Time series Anomaly Detection via Graph Attention Network
Anomaly detection on multivariate time series is of great importance in both data mining research and industrial applications. Recent approaches have achieved significant progress in this topic, but there is remaining limitations. One major limitation is that they do not capture the relationships between different time series explicitly, resulting in inevitable false alarms. In this paper, we propose a novel self supervised framework for multivariate time series anomaly detection to address this issue. Our framework considers each univariate time series as an individual feature and includes two graph attention layers in parallel to learn the complex dependencies of multivariate time series in both temporal and feature dimensions. In addition, our approach jointly optimizes a forecasting based model and a reconstruction based model, obtaining better time series representations through a combination of single timestamp prediction and reconstruction of the entire time series. We demonstrate the efficacy of our model through extensive experiments. The proposed method outperforms other state of the art models on three real world datasets. Further analysis shows that our method has good interpretability and is useful for anomaly diagnosis.
2020 IEEE International Conference on Data Mining (ICDM)
Hang Zhao Juanyong Duan Congrui Huang Defu Cao Yunhai Tong Bixiong Xu Jing Bai Jie Tong Qi Zhang
235254178
Multi Label Few Shot Learning for Aspect Category Detection
Aspect category detection (ACD) in sentiment analysis aims to identify the aspect categories mentioned in a sentence. In this paper, we formulate ACD in the few shot learning scenario. However, existing few shot learning approaches mainly focus on single label predictions. These methods can not work well for the ACD task since a sentence may contain multiple aspect categories. Therefore, we propose a multi label few shot learning method based on the prototypical network. To alleviate the noise, we design two effective attention mechanisms. The support set attention aims to extract better prototypes by removing irrelevant aspects. The query set attention computes multiple prototype specific representations for each query instance, which are then used to compute accurate distances with the corresponding prototypes. To achieve multilabel inference, we further learn a dynamic threshold per instance by a policy network. Extensive experimental results on three datasets demonstrate that the proposed method significantly outperforms strong baselines.
ACL/IJCNLP
Mengting Hu Shiwan Zhao Honglei Guo Chao Xue H Gao Tiegang Gao Renhong Cheng Zhong Su
222133379
Sentence Constituent Aware Aspect Category Sentiment Analysis with Graph Attention Networks
Aspect category sentiment analysis (ACSA) aims to predict the sentiment polarities of the aspect categories discussed in sentences. Since a sentence usually discusses one or more aspect categories and expresses different sentiments toward them, various attention based methods have been developed to allocate the appropriate sentiment words for the given aspect category and obtain promising results. However, most of these methods directly use the given aspect category to find the aspect category related sentiment words, which may cause mismatching between the sentiment words and the aspect categories when an unrelated sentiment word is semantically meaningful for the given aspect category. To mitigate this problem, we propose a Sentence Constituent Aware Network (SCAN) for aspect category sentiment analysis. SCAN contains two graph attention modules and an interactive loss function. The graph attention modules generate representations of the nodes in sentence constituency parse trees for the aspect category detection (ACD) task and the ACSA task, respectively. ACD aims to detect aspect categories discussed in sentences and is a auxiliary task. For a given aspect category, the interactive loss function helps the ACD task to find the nodes which can predict the aspect category but can't predict other aspect categories. The sentiment words in the nodes then are used to predict the sentiment polarity of the aspect category by the ACSA task. The experimental results on five public datasets demonstrate the effectiveness of SCAN.
NLPCC
Yuncong Li Cunxiang Yin Sheng-hua Zhong
104292826
LICD: A Language Independent Approach for Aspect Category Detection
Aspect based sentiment analysis (ABSA) deals with processing and summarizing customer reviews and has been a topic of interest in recent years. Given a set of predefined categories, Aspect Category Detection (ACD) as a subtask of ABSA, aims to assign a subset of these categories to a given review sentence. Thanks to the existence of websites such as Yelp and TripAdvisor, there exist a huge amount of reviews in several languages, and therefore the need for language independent methods in this task seems necessary. In this paper, we propose Language Independent Category Detector (LICD) a supervised method based on text matching without the need for any language specific tools and hand crafted features for identifying aspect categories. For a given sentence, our proposed method performs ACD based on two hypotheses: First, a category should be assigned to a sentence if there is a high semantic similarity between the sentence and a set of representative words of that category. Second, a category should be assigned to a sentence if sentences with high semantic and structural similarity to that sentence belong to that category. To apply the former hypothesis, we used soft cosine measure, and for the latter, word mover's distance measure is utilized. Using these two measures, for a given sentence we calculate a set of similarity scores as features for a one vs all logistic regression classifier per category. Experimental results on the multilingual SemEval 2016 datasets in the restaurant domain demonstrate that our approach outperforms baseline methods in English, Russian, and Dutch languages, and obtains competitive results with the strong deep neural network based baselines in French, Turkish, and Spanish languages.
ECIR
Erfan Ghadery Sajad Movahedi Masoud Jalili Sabet Heshaam Faili Azadeh Shakery
208331852
Negative Focus Detection via Contextual Attention Mechanism
Negation is a universal but complicated linguistic phenomenon, which has received considerable attention from the NLP community over the last decade, since a negated statement often carries both an explicit negative focus and implicit positive meanings. For the sake of understanding a negated statement, it is critical to precisely detect the negative focus in context. However, how to capture contextual information for negative focus detection is still an open challenge. To well address this, we come up with an attention based neural network to model contextual information. In particular, we introduce a framework which consists of a Bidirectional Long Short Term Memory (BiLSTM) neural network and a Conditional Random Fields (CRF) layer to effectively encode the order information and the long range context dependency in a sentence. Moreover, we design two types of attention mechanisms, word level contextual attention and topic level contextual attention, to take advantage of contextual information across sentences from both the word perspective and the topic perspective, respectively. Experimental results on the SEM'12 shared task corpus show that our approach achieves the best performance on negative focus detection, yielding an absolute improvement of 2.11% over the state of the art. This demonstrates the great effectiveness of the two types of contextual attention mechanisms.
EMNLP
Longxiang Shen Bowei Zou Yu Hong Guodong Zhou Qiaoming Zhu AiTi Aw
202642086
Adverse drug reaction detection via a multihop self attention mechanism
BackgroundThe adverse reactions that are caused by drugs are potentially life threatening problems. Comprehensive knowledge of adverse drug reactions (ADRs) can reduce their detrimental impacts on patients. Detecting ADRs through clinical trials takes a large number of experiments and a long period of time. With the growing amount of unstructured textual data, such as biomedical literature and electronic records, detecting ADRs in the available unstructured data has important implications for ADR research. Most of the neural network based methods typically focus on the simple semantic information of sentence sequences; however, the relationship of the two entities depends on more complex semantic information.MethodsIn this paper, we propose multihop self attention mechanism (MSAM) model that aims to learn the multi aspect semantic information for the ADR detection task. first, the contextual information of the sentence is captured by using the bidirectional long short term memory (Bi LSTM) model. Then, via applying the multiple steps of an attention mechanism, multiple semantic representations of a sentence are generated. Each attention step obtains a different attention distribution focusing on the different segments of the sentence. Meanwhile, our model locates and enhances various keywords from the multiple representations of a sentence.ResultsOur model was evaluated by using two ADR corpora. It is shown that the method has a stable generalization ability. Via extensive experiments, our model achieved F measure of 0.853, 0.799 and 0.851 for ADR detection for TwiMed PubMed, TwiMed Twitter, and ADE, respectively. The experimental results showed that our model significantly outperforms other compared models for ADR detection.ConclusionsIn this paper, we propose a modification of multihop self attention mechanism (MSAM) model for an ADR detection task. The proposed method significantly improved the learning of the complex semantic information of sentences.
BMC Bioinformatics
Tongxuan Zhang Hongfei Lin Yuqi Ren Liang Yang Bo Xu Zhihao Yang Jiajun Wang Yijia Zhang
221319903
Cross regional oil palm tree counting and detection via multi level attention domain adaptation network
Abstract Providing an accurate evaluation of palm tree plantation in a large region can bring meaningful impacts in both economic and ecological aspects. However, the enormous spatial scale and the variety of geological features across regions has made it a grand challenge with limited solutions based on manual human monitoring efforts. Although deep learning based algorithms have demonstrated potential in forming an automated approach in recent years, the labelling efforts needed for covering different features in different regions largely constrain its effectiveness in large scale problems. In this paper, we propose a novel domain adaptive oil palm tree detection method, i.e. a Multi level Attention Domain Adaptation Network (MADAN) to reap cross regional oil palm tree counting and detection. MADAN consists of 4 procedures: First, we adopted a batch instance normalization network (BIN) based feature extractor for improving the generalization ability of the model, integrating batch normalization and instance normalization. Second, we embedded a multi level attention mechanism (MLA) into our architecture for enhancing the transferability, including a feature level attention and an entropy level attention. Then we designed a minimum entropy regularization (MER) to increase the confidence of the classifier predictions through assigning the entropy level attention value to the entropy penalty. Finally, we employed a sliding window based prediction and an IOU based post processing approach to attain the final detection results. We conducted comprehensive ablation experiments using three different satellite images of large scale oil palm plantation area with six transfer tasks. MADAN improves the detection accuracy by 14.98% in terms of average F1 score compared with the Baseline method (without DA) and performs 3.55 14.49% better than existing domain adaptation methods. Experimental results demonstrate the great potential of our MADAN for large scale and cross regional oil palm tree counting and detection, guaranteeing a high detection accuracy as well as saving the manual annotation efforts.
ArXiv
Juepeng Zheng Haohuan Fu Weijia Li Wenzhao Wu Yi Zhao Runmin Dong Le Yu
192494822
Employee profiling via aspect based sentiment and network for insider threats detection
Abstract Historically, the harm caused by insiders has proven to be one of the greatest concerns for any organization. As such, it has received considerable attention from both the industrial and research communities. Existing works mainly focused on modeling the employees' normal biometric behavior (e.g. human to device interaction pattern) to detect anomalous behavior which corresponds to the insider activity. However, it is unattainable to stop the insider at the final moment when the malicious act is being carried out. In this paper, we propose a novel framework which performs employee profiling based on aspect based sentiments and social network information and examine its applicability for early detection of potential insider threats. On the contrary to the traditional sentiment analysis, aspect based sentiment analysis provides more fine grained information on the employee. Our framework employs a combination of deep learning techniques such as Gated Recurrent Unit (GRU) and skipgram to build temporal sentiment profiles for the employees. It then performs anomaly detection on the profiles and ranks the employees based on their respective anomaly score. Due to the absence of relevant benchmark dataset, we augmented the publicly available real world Enron email corpus with an insider threat scenario to evaluate our framework. The evaluation results demonstrate that the augmentation is indeed reflected in the augmented employee's anomaly ranking (i.e. from normal to abnormal) and her close associates are indeed placed closely to her when the profiles are visualized in the 2D space. The profiles obtained from our framework can also be used to complement any existing expert and intelligent systems with additional capabilities in handling textual information such as, integration with profiles obtained from biometric behavior to form a more comprehensive threat detection system.
Expert Syst. Appl.
Charlie Soh S Yu Annamalai Narayanan Santhiya Duraisamy Lihui Chen