HIERARCHICAL RELATIONAL LEARNING FOR FEW-SHOT KNOWLEDGE GRAPH COMPLETION

Abstract

Knowledge graphs (KGs) are powerful in terms of their inference abilities, but are also notorious for their incompleteness and long-tail distribution of relations. To address these challenges and expand the coverage of KGs, few-shot KG completion aims to make predictions for triplets involving novel relations when only a few training triplets are provided as reference. Previous methods have focused on designing local neighbor aggregators to learn entity-level information and/or imposing a potentially invalid sequential dependency assumption at the triplet level to learn meta relation information. However, pairwise triplet-level interactions and context-level relational information have been largely overlooked for learning meta representations of few-shot relations. In this paper, we propose a hierarchical relational learning method (HiRe) for few-shot KG completion. By jointly capturing three levels of relational information (entity-level, triplet-level and contextlevel), HiRe can effectively learn and refine meta representations of few-shot relations, and thus generalize well to new unseen relations. Extensive experiments on benchmark datasets validate the superiority of HiRe over state-of-the-art methods. The code can be found in https://github.com/alexhw15/HiRe.git.

1. INTRODUCTION

Knowledge graphs (KGs) comprise a collection of factual triplets, (h, r, t), where each triplet expresses the relationship r between a head entity h and a tail entity t. Large-scale KGs (Vrandečić & Krötzsch, 2014; Mitchell et al., 2018; Suchanek et al., 2007; Bollacker et al., 2008) can provide powerful inference capabilities for many intelligent applications, including question answering (Yao & Van Durme, 2014) , web search (Eder, 2012) and recommendation systems (Wang et al., 2019) . As KGs are often built semi-automatically from unstructured data, real-world KGs are far from complete and suffer from the notorious long-tail problem -a considerable proportion of relations are associated with only very few triplets. As a result, the performance of current KG completion methods significantly degrades when predicting relations with a limited number (few-shot) of training triplets. To tackle this challenge, few-shot KG completion methods have been proposed including GMatching (Xiong et al., 2018 ), MetaR(Chen et al., 2019 ), FSRL(Zhang et al., 2020) , FAAN (Sheng et al., 2020) and GANA (Niu et al., 2021) . These methods focus on predicting the missing tail entity t for query triplets by learning from only K reference triplets about a target relation r. Given a target relation r and K reference triplets, K-shot KG completion aims to correctly predict the tail entity t for each query triplet (h, r, ?) using the generalizable knowledge learned from reference triplets. Thus, the crucial aspect of few-shot KG completion is to learn the meta representation of each few-shot relation from a limited amount of reference triplets that can generalize to novel relations. To facilitate the learning of meta relation representations, we identify three levels of relational information (see Figure 1 ). ( 1) At the context level, each reference triplet is closely related to its wider contexts, providing crucial evidence for enriching entity and relation embeddings. ( 2) At the triplet level, capturing the commonality among limited reference triplets is essential for learning meta relation representations. (3) At the entity level, the learned meta relation representations should well generalize to unseen query triplets. Current few-shot KG methods have, however, focused on designing local neighbor aggregators to learn entity-level information, and/or imposing a sequential assumption at the triplet level to learn meta relation information (See Table 1 ). The potential of leveraging pairwise triplet-level interactions and context-level relational information has been largely unexplored. … (a) (b) (c) ℎ 1 𝑡𝑡 1 context information triplet ℎ 2 𝑡𝑡 2 ℎ 𝑘𝑘 𝑡𝑡 k Triplet-level relational information ℎ 2 𝑡𝑡 2 Entity-level relational information … (a) (b) (c) ℎ 𝑡𝑡 context Context-level relational information triplet ℎ 1 𝑡𝑡 1 ℎ 2 𝑡𝑡 2 ℎ 𝑘𝑘 𝑡𝑡 k Triplet-level relational information ℎ 1 𝑡𝑡 1 ℎ 2 𝑡𝑡 2 Entity-level relational information In this paper, we propose a Hierarchical Relational learning framework (HiRe) for few-shot KG completion. HiRe jointly models three levels of relational information (entity-level, triplet-level, and context-level) within each few-shot task as mutually reinforcing sources of information to generalize to few-shot relations. Here, "hierarchical" references relational learning performed at three different levels of granularity. Specifically, we make the following contributions: GMatching ✓ ✗ ✗ ✗ MetaR ✓ ✗ ✗ ✗ FSRL ✓ ✓ ✗ ✗ FAAN ✓ ✓ ✗ ✗ GANA ✓ ✓ ✗ ✗ HiRe (ours) ✓ ✗ ✓ ✓ • We propose a contrastive learning based context-level relational learning method to learn expressive entity/relation embeddings by modeling correlations between the target triplet and its true/false contexts. We argue that a triplet itself has a close relationship with its true context. Thus, we take a contrastive approach -a given triplet should be pulled close to its true context, but pushed apart from its false contexts -to learn better entity embeddings. • We propose a transformer based meta relation learner (MRL) to learn generalizable meta relation representations. Our proposed MRL is capable of capturing pairwise interactions among reference triplets, while preserving the permutation-invariance property and being insensitive to the size of the reference set. • We devise a meta representation based embedding learner named MTransD that constrains the learned meta relation representations to hold between unseen query triplets, enabling better generalization to novel relations. Lastly, we adopt a model agnostic meta learning (MAML) based training strategy (Finn et al., 2017) to optimize HiRe on each meta task within a unified framework. By performing relational learning at three granular levels, HiRe offers significant advantages for extracting expressive meta relation representations and improving model generalizability for few-shot KG completion. Extensive experiments on two benchmark datasets validate the superiority of HiRe over state-of-the-art methods.

2.1. RELATIONAL LEARNING IN KNOWLEDGE GRAPHS

KG completion methods utilize relational information available in KGs to learn a unified lowdimensional embedding space for the input triplets. TransE (Bordes et al., 2013) is the first to use relation r as a translation for learning an embedding space, i.e., h + r ≈ t for triplet (h, r, t). A scoring function is then used to measure the quality of the translation and to learn a unified embedding space. TransH (Wang et al., 2014) and TransR (Lin et al., 2015) further model relation-specific information for learning an embedding space. ComplEx (Trouillon et al., 2016 ), RotatE (Sun et al., 2019b ), and ComplEx-N3 (Lacroix et al., 2018) improve the modeling of relation patterns in a vector/complex space. ConvE (Dettmers et al., 2018) and ConvKB (Nguyen et al., 2018) employ convolution operators to enhance entity/relation embedding learning. However, these methods require a large number of triplets for each relation to learn a unified embedding space. Their performance significantly degrades at few-shot settings, where only very few triplets are available for each relation.

2.2. FEW-SHOT KG COMPLETION

Existing few-shot KG completion methods can be grouped into two main categories: (1) Metric learning based methods: GMatching (Xiong et al., 2018) is the first work to formulate few-shot (one-shot) KG completion. GMatching consists of two parts: a neighbor encoder that aggregates one-hop neighbors of any given entity, and a matching processor that compares similarity between query and reference entity pairs. FSRL (Zhang et al., 2020) relaxes the setting to more shots and explores how to integrate the information learned from multiple reference triplets. FAAN (Sheng et al., 2020) proposes a dynamic attention mechanism for designing one-hop neighbor aggregators.



Figure 1: Three levels of relational information: (a) Context-level, (b) Triplet-level, (c) Entity-level.

Summary of few-shot KG completion methods based on different levels of relational information used.

