Relation extraction
49544037
Graph Convolution over Pruned Dependency Trees Improves Relation Extraction
Dependency trees help relation extraction models capture long range relations between words. However, existing dependency based models either neglect crucial information (e.g. negation) by pruning the dependency trees too aggressively, or are computationally inefficient because it is difficult to parallelize over different tree structures. We propose an extension of graph convolutional networks that is tailored for relation extraction, which pools information over arbitrary dependency structures efficiently in parallel. To incorporate relevant information while maximally removing irrelevant content, we further apply a novel pruning strategy to the input trees by keeping words immediately around the shortest path between the two entities among which a relation might hold. The resulting model achieves state of the art performance on the large scale TACRED dataset, outperforming existing sequence and dependency based neural models. We also show through detailed analysis that this model has complementary strengths to sequence models, and combining them further improves the state of the art.
EMNLP
Yuhao Zhang Peng Qi Christopher D Manning
131773936
Simple BERT Models for Relation Extraction and Semantic Role Labeling
We present simple BERT based models for relation extraction and semantic role labeling. In recent years, state of the art performance has been achieved using neural models by incorporating lexical and syntactic features such as part of speech tags and dependency trees. In this paper, extensive experiments on datasets for these two tasks show that without using any external features, a simple BERT based model can achieve state of the art performance. To our knowledge, we are the first to successfully apply BERT in this manner. Our models provide strong baselines for future research.
ArXiv
Peng Shi Jimmy J Lin
190001673
Attention Guided Graph Convolutional Networks for Relation Extraction
Dependency trees convey rich structural information that is proven useful for extracting relations among entities in text. However, how to effectively make use of relevant information while ignoring irrelevant information from the dependency trees remains a challenging research question. Existing approaches employing rule based hard pruning strategies for selecting relevant partial dependency structures may not always yield optimal results. In this work, we propose Attention Guided Graph Convolutional Networks (AGGCNs) a novel model which directly takes full dependency trees as inputs. Our model can be understood as a soft pruning approach that automatically learns how to selectively attend to the relevant sub structures useful for the relation extraction task. Extensive results on various tasks including cross sentence n ary relation extraction and large scale sentence level relation extraction show that our model is able to better leverage the structural information of the full dependency trees, giving significantly better results than previous approaches.
ACL
Zhijiang Guo Yan Zhang Wei Lu
53250562
A Hierarchical Framework for Relation Extraction with Reinforcement Learning
Most existing methods determine relation types only after all the entities have been recognized, thus the interaction between relation types and entity mentions is not fully modeled. This paper presents a novel paradigm to deal with relation extraction by regarding the related entities as the arguments of a relation. We apply a hierarchical reinforcement learning (HRL) framework in this paradigm to enhance the interaction between entity mentions and relation types. The whole extraction process is decomposed into a hierarchy of two level RL policies for relation detection and entity extraction respectively, so that it is more feasible and natural to deal with overlapping relations. Our model was evaluated on public datasets collected via distant supervision, and results show that it gains better performance than existing methods and is more powerful for extracting overlapping relations1.
AAAI
Ryuichi Takanobu Tianyang Zhang Jiexi Liu Minlie Huang
196211486
GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction
In this paper, we present GraphRel, an end to end relation extraction model which uses graph convolutional networks (GCNs) to jointly learn named entities and relations. In contrast to previous baselines, we consider the interaction between named entities and relations via a 2nd phase relation weighted GCN to better extract relations. Linear and dependency structures are both used to extract both sequential and regional features of the text, and a complete word graph is further utilized to extract implicit features among all word pairs of the text. With the graph based approach, the prediction for overlapping relations is substantially improved over previous sequential approaches. We evaluate GraphRel on two public datasets: NYT and WebNLG. Results show that GraphRel maintains high precision while increasing recall substantially. Also, GraphRel outperforms previous work by 3.2% and 5.8% (F1 score) achieving a new state of the art for relation extraction.
ACL
Tsu-Jui Fu Peng-Hsuan Li Wei-Yun Ma
153312535
Entity Relation Extraction as Multi Turn Question Answering
In this paper, we propose a new paradigm for the task of entity relation extraction. We cast the task as a multi turn question answering problem, i.e. the extraction of entities and elations is transformed to the task of identifying answer spans from the context. This multi turn QA formalization comes with several key advantages: firstly, the question query encodes important information for the entity/relation class we want to identify; secondly, QA provides a natural way of jointly modeling entity and relation; and thirdly, it allows us to exploit the well developed machine reading comprehension (MRC) models. Experiments on the ACE and the CoNLL04 corpora demonstrate that the proposed paradigm significantly outperforms previous best models. We are able to obtain the state of the art results on all of the ACE04, ACE05 and CoNLL04 datasets, increasing the SOTA results on the three datasets to 49.6 +1.2) 60.3 +0.7) and 69.2 +1.4) respectively. Additionally, we construct and will release a newly developed dataset RESUME, which requires multi step reasoning to construct entity dependencies, as opposed to the single step dependency extraction in the triplet exaction in previous datasets. The proposed multi turn QA model also achieves the best performance on the RESUME dataset.
ACL
Xiaoya Li Fan Yin Zijun Sun Xiayu Li Arianna Yuan Duo Chai Mingxin Zhou Jiwei Li
189898081
DocRED: A Large Scale Document Level Relation Extraction Dataset
Multiple entities in a document generally exhibit complex inter sentence relations, and cannot be well handled by existing relation extraction (RE) methods that typically focus on extracting intra sentence relations for single entity pairs. In order to accelerate the research on document level RE, we introduce DocRED, a new dataset constructed from Wikipedia and Wikidata with three features: (1) DocRED annotates both named entities and relations, and is the largest human annotated dataset for document level RE from plain text; (2) DocRED requires reading multiple sentences in a document to extract entities and infer their relations by synthesizing all information of the document; (3) along with the human annotated data, we also offer large scale distantly supervised data, which enables DocRED to be adopted for both supervised and weakly supervised scenarios. In order to verify the challenges of document level RE, we implement recent state of the art methods for RE and conduct a thorough evaluation of these methods on DocRED. Empirical results show that DocRED is challenging for existing RE methods, which indicates that document level RE remains an open problem and requires further efforts. Based on the detailed analysis on the experiments, we discuss multiple promising directions for future research.
ACL
Yuan Yao Deming Ye Peng Li Xu Han Yankai Lin Zhenghao Liu Zhiyuan Liu Lixin Huang Jie Zhou Maosong Sun
67856607
Long tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks
We propose a distance supervised relation extraction approach for long tailed, imbalanced data which is prevalent in real world settings. Here, the challenge is to learn accurate "few shot" models for classes existing at the tail of the class distribution, for which little data is available. Inspired by the rich semantic correlations between classes at the long tail and those at the head, we take advantage of the knowledge from data rich classes at the head of the distribution to boost the performance of the data poor classes at the tail. First, we propose to leverage implicit relational knowledge among class labels from knowledge graph embeddings and learn explicit relational knowledge using graph convolution networks. Second, we integrate that relational knowledge into relation extraction model by coarse to fine knowledge aware attention mechanism. We demonstrate our results for a large scale benchmark dataset which show that our approach significantly outperforms other baselines, especially for long tail relations.
NAACL
Ningyu Zhang Shumin Deng Zhanlin Sun Guanying Wang X Chen Wei Zhang Huajun Chen
184487889
Inter sentence Relation Extraction with Document level Graph Convolutional Neural Network
Inter sentence relation extraction deals with a number of complex semantic relationships in documents, which require local, non local, syntactic and semantic dependencies. Existing methods do not fully exploit such dependencies. We present a novel inter sentence relation extraction model that builds a labelled edge graph convolutional neural network model on a document level graph. The graph is constructed using various inter and intra sentence dependencies to capture local and non local dependency information. In order to predict the relation of an entity pair, we utilise multi instance learning with bi affine pairwise scoring. Experimental results show that our model achieves comparable performance to the state of the art neural models on two biochemistry datasets. Our analysis shows that all the types in the graph are effective for inter sentence relation extraction.
ACL
Sunil Kumar Sahu Fenia Christopoulou Makoto Miwa Sophia Ananiadou
53116045
Knowledge oriented convolutional neural network for causal relation extraction from natural language texts
Abstract Causal relation extraction is a challenging yet very important task for Natural Language Processing (NLP) There are many existing approaches developed to tackle this task, either rule based (non statistical) or machine learning based (statistical) method. For rule based method, extensive manual work is required to construct handcrafted patterns, however, the precision and recall are low due to the complexity of causal relation expressions in natural language. For machine learning based method, current approaches either rely on sophisticated feature engineering which is error prone, or rely on large amount of labeled data which is impractical for causal relation extraction problem. To address the above issues, we propose a Knowledge oriented Convolutional Neural Network (K CNN) for causal relation extraction in this paper. K CNN consists of a knowledge oriented channel that incorporates human prior knowledge to capture the linguistic clues of causal relationship, and a data oriented channel that learns other important features of causal relation from the data. The convolutional filters in knowledge oriented channel are automatically generated from lexical knowledge bases such as WordNet and FrameNet. We propose filter selection and clustering techniques to reduce dimensionality and improve the performance of K CNN. Furthermore, additional semantic features that are useful for identifying causal relations are created. Three datasets have been used to evaluate the ability of K CNN to effectively extract causal relation from texts, and the model outperforms current state of art models for relation extraction.
Expert Syst. Appl.
Pengfei Li Kezhi Mao