Question Answering over Knowledge Base with Variational Auto-Encoder

Knowledge Base is a semantic data repos-itory made available over the web. Knowledge bases are multi-relational graphs consisting of entities and relations representing facts about the world. These facts follow a controlled vocabulary that drives the knowledge base, often called ontology. Entities are the graph nodes, while relations are the edges connecting the nodes. While knowledge base attention prolifer-ates, extracting information from these resources is challenging. One of the promising approaches is Question Answering systems. The Question Answering over Knowledge Base(KB-QA) system aims to provide a natural language interface to pose a query to the KB. QA systems use embeddings to map the posed Question to the facts stored in the knowledge bases. QA systems can be effectively improved using large transformer models. These transformer models are computationally expensive, which prevents their use in real-world applications. To solve this, we started our study with the use-case of Knowledge Base Question Answering systems. In KB-QA systems, representing the entities and relationships play a pivotal role in extracting the answer to the posed query. Traditional KB embedding techniques represent entities/relations as vectors, mapping them in different semantic spaces and ignoring the subtle cor-relation. A pre-training language model for encoding the KB facts will preserve the semantic and contextual correlation. While learning linguistic knowledge, these models also store relational knowledge in the training data. These Language Models are often trained on a massive corpus addressing the Out-Of- Vocabulary problem. In this paper, we incorporate a co-embedding model for knowledge base embedding, which learns low-dimensional representations of entities and relations in the same semantic space. We propose a Variational Auto-Encoder, an efficient transformer architecture that represents knowledge base representations as Gaussian distributions to address the issue of neglecting uncertainty. In addition, our method has high quality and interpretability advantages compared with previous methods. Our experimental results show the effectiveness and the superiority of the VAE approach for question-answering systems on knowledge bases over other well-known pre-trained embedding methods.