RNNLOGIC: LEARNING LOGIC RULES FOR REASON-ING ON KNOWLEDGE GRAPHS

Abstract

This paper studies learning logic rules for reasoning on knowledge graphs. Logic rules provide interpretable explanations when used for prediction as well as being able to generalize to other tasks, and hence are critical to learn. Existing methods either suffer from the problem of searching in a large search space (e.g., neural logic programming) or ineffective optimization due to sparse rewards (e.g., techniques based on reinforcement learning). To address these limitations, this paper proposes a probabilistic model called RNNLogic. RNNLogic treats logic rules as a latent variable, and simultaneously trains a rule generator as well as a reasoning predictor with logic rules. We develop an EM-based algorithm for optimization. In each iteration, the reasoning predictor is first updated to explore some generated logic rules for reasoning. Then in the E-step, we select a set of high-quality rules from all generated rules with both the rule generator and reasoning predictor via posterior inference; and in the M-step, the rule generator is updated with the rules selected in the E-step. Experiments on four datasets prove the effectiveness of RNNLogic.

1. INTRODUCTION

Knowledge graphs are collections of real-world facts, which are useful in various applications. Each fact is typically specified as a triplet (h, r, t) or equivalently r(h, t), meaning entity h has relation r with entity t. For example, Bill Gates is the Co-founder of Microsoft. As it is impossible to collect all facts, knowledge graphs are incomplete. Therefore, a fundamental problem on knowledge graphs is to predict missing facts by reasoning with existing ones, a.k.a. knowledge graph reasoning. This paper studies learning logic rules for reasoning on knowledge graphs. For example, one may extract a rule ∀X, Y, Z hobby(X, Y ) ← friend(X, Z) ∧ hobby(Z, Y ), meaning that if Z is a friend of X and Z has hobby Y , then Y is also likely the hobby of X. Then the rule can be applied to infer new hobbies of people. Such logic rules are able to improve interpretability and precision of reasoning (Qu & Tang, 2019; Zhang et al., 2020) . Moreover, logic rules can also be reused and generalized to other domains and data (Teru & Hamilton, 2020) . However, due to the large search space, inferring high-quality logic rules for reasoning on knowledge graphs is a challenging task. Indeed, a variety of methods have been proposed for learning logic rules from knowledge graphs. Most traditional methods such as path ranking (Lao & Cohen, 2010) and Markov logic networks (Richardson & Domingos, 2006) enumerate relational paths on graphs as candidate logic rules, and then learn a weight for each rule as an assessment of rule qualities. There are also some recent methods based on neural logic programming (Yang et al., 2017) and neural theorem provers (Rocktäschel & Riedel, 2017) , which are able to learn logic rules and their weights simultaneously in a differentiable way. Though empirically effective for prediction, the search space of these methods is exponentially large, making it hard to identify high-quality logic rules. Besides, some recent efforts (Xiong et al., 2017) formulate the problem as a sequential decision making process, and use reinforcement learning to search for logic rules, which significantly reduces search complexity. However, due to the large action space and sparse reward in training, the performance of these methods is not yet satisfying. In this paper, we propose a principled probabilistic approach called RNNLogic which overcomes the above limitations. Our approach consists of a rule generator as well as a reasoning predictor with logic rules, which are simultaneously trained to enhance each other. The rule generator provides logic rules which are used by the reasoning predictor for reasoning, while the reasoning predictor provides effective reward to train the rule generator, which helps significantly reduce the search space. Specifically, for each query-answer pair, e.g., q = (h, r, ?) and a = t, we model the probability of the answer conditioned on the query and existing knowledge graph G, i.e., p(a|G, q), where a set of logic rules zfoot_0 is treated as a latent variable. The rule generator defines a prior distribution over logic rules for each query, i.e., p(z|q), which is parameterized by a recurrent neural network. The reasoning predictor computes the likelihood of the answer conditioned on the logic rules and the existing knowledge graph G, i.e., p(a|G, q, z). At each training iteration, we first sample a few logic rules from the rule generator, and further update the reasoning predictor to try out these rules for prediction. Then an EM algorithm (Neal & Hinton, 1998 ) is used to optimize the rule generator. In the E-step, a set of high-quality logic rules are selected from all the generated rules according to their posterior probabilities. In the M-step, the rule generator is updated to imitate the high-quality rules selected in the E-step. Extensive experimental results show that RNNLogic outperforms state-of-the-art methods for knowledge graph reasoningfoot_1 . Besides, RNNLogic is able to generate high-quality logic rules.

2. RELATED WORK

Our work is related to existing efforts on learning logic rules for knowledge graph reasoning. Most traditional methods enumerate relational paths between query entities and answer entities as candidate logic rules, and further learn a scalar weight for each rule to assess the quality. Representative methods include Markov logic networks (Kok & Domingos, 2005; Richardson & Domingos, 2006; Khot et al., 2011) , relational dependency networks (Neville & Jensen, 2007; Natarajan et al., 2010) , rule mining algorithms (Galárraga et al., 2013; Meilicke et al., 2019) , path ranking (Lao & Cohen, 2010; Lao et al., 2011) and probabilistic personalized page rank (ProPPR) algorithms (Wang et al., 2013; 2014a; b) . Some recent methods extend the idea by simultaneously learning logic rules and the weights in a differentiable way, and most of them are based on neural logic programming (Rocktäschel & Riedel, 2017; Yang et al., 2017; Cohen et al., 2018; Sadeghian et al., 2019; Yang & Song, 2020) or neural theorem provers (Rocktäschel & Riedel, 2017; Minervini et al., 2020) . These methods and our approach are similar in spirit, as they are all able to learn the weights of logic rules efficiently. However, these existing methods try to simultaneously learn logic rules and their weights, which is nontrivial in terms of optimization. The main innovation of our approach is to separate rule generation and rule weight learning by introducing a rule generator and a reasoning predictor respectively, which can mutually enhance each other. The rule generator generates a few high-quality logic rules, and the reasoning predictor only focuses on learning the weights of such high-quality rules, which significantly reduces the search space and leads to better reasoning results. Meanwhile, the reasoning predictor can in turn help identify some useful logic rules to improve the rule generator. The other kind of rule learning method is based on reinforcement learning. The general idea is to train pathfinding agents, which search for reasoning paths in knowledge graphs to answer questions, and then extract logic rules from reasoning paths (Xiong et al., 2017; Chen et al., 2018; Das et al., 2018; Lin et al., 2018; Shen et al., 2018) . However, training effective pathfinding agents is highly challenging, as the reward signal (i.e., whether a path ends at the correct answer) can be extremely sparse. Although some studies (Lin et al., 2018) try to get better reward by using embedding-based methods for reward shaping, the performance is still worse than most embedding-based methods. In our approach, the rule generator has a similar role to those pathfinding agents. The major difference is that we simultaneously train the rule generator and a reasoning predictor with logic rules, which mutually enhance each other. The reasoning predictor provides effective reward for training the rule generator, and the rule generator offers high-quality rules to improve the reasoning predictor. Our work is also related to knowledge graph embedding, which solves knowledge graph reasoning by learning entity and relation embeddings in latent spaces (Bordes et al., 2013; Wang et al., 2014c; Yang et al., 2015; Nickel et al., 2016; Trouillon et al., 2016; Cai & Wang, 2018; Dettmers et al., 2018; Balazevic et al., 2019; Sun et al., 2019) . With proper architectures, these methods are able to learn



More precisely, z is a multiset. In this paper, we use "set" to refer to "multiset" for conciseness. The codes of RNNLogic are available: https://github.com/DeepGraphLearning/RNNLogic

