TWO BIRDS, ONE STONE: AN EQUIVALENT TRANS-FORMATION FOR HYPER-RELATIONAL KNOWLEDGE GRAPH MODELING Anonymous authors Paper under double-blind review

Abstract

By representing knowledge in a primary triple associated with additional attributevalue qualifiers, hyper-relational knowledge graph (HKG) that generalizes triplebased knowledge graph (KG) has been attracting research attention recently. Compared with KG, HKG is enriched with the semantic difference between the primary triple and additional qualifiers as well as the structural connection between entities in hyper-relational graph structure. However, to model HKG, existing studies mainly focus on either semantic information or structural information therein, fail to capture both simultaneously. To tackle this issue, in this paper, we propose an equivalent transformation for HKG modeling, referred to as TransEQ. Specifically, the equivalent transformation transforms a HKG to a KG, which considers both semantic and structural characteristics. Then a generalized encoder-decoder framework is developed to bridge the modeling research between KG and HKG. In the encoder part, KG-based graph neural networks are leveraged for structural modeling; while in the decoder part, various HKG-based scoring functions are exploited for semantic modeling. Especially, we design the sharing embedding mechanism in the encoder-decoder framework with semantic relatedness captured. We further theoretically prove that TransEQ preserves complete information in the equivalent transformation, and also achieves full expressivity. Finally, extensive experiments on three benchmarks demonstrate the superior performance of TransEQ in terms of both effectiveness and efficiency. On the largest benchmark WikiPeople, TransEQ significantly improves the state-of-the-art models by 15% on MRR.

1. INTRODUCTION

In the past decade, knowledge graph (KG) has been widely studied in artificial intelligence area (Ji et al., 2021) . By representing facts into a triple of (s, r, o) with subject entity s, object entity o and relation r, KG stores real-world knowledge in a graph structure. However, recent studies find that KG with simple triples provides incomplete information (Galkin et al., 2020; Rosso et al., 2020) . For example, both (Alan Turing, educated at, Cambridge) and (Alan Turing, educated at, Princeton) are true facts in KG, which might be ambiguous when the degree matters. Hence, the hyper-relational KG (HKG) (Galkin et al., 2020; Rosso et al., 2020; Yu & Yang, 2021 ), a.k.a., knowledge hypergraph (Fatemi et al., 2020; 2021) and n-ary knowledge base (Guan et al., 2019; Liu et al., 2021) , is proposed for more generalized knowledge representation. Formally, in HKG, a primary triple is augmented with additional attribute-value qualifiers for rich semantics, called the hyper-relational fact (Guan et al., 2020) . Note that the triple without qualifiers is a special case of hyper-relational facts. Taking Figure 1 as an example, both (Alan Turing, educated at, Cambridge, (degree, Bachelor)) and (Alan Turing, educated at, Princeton, (degree, PhD)) are hyper-relational facts, where (degree, Bachelor) and (degree, PhD) are qualifiers with the degree attribute considered. Such hyper-relational facts are ubiquitous that over 1/3 of the entities in Freebase (Bollacker et al., 2008) involve in them (Wen et al., 2016) . To learn from HKG and further benefit the downstream tasks, HKG modeling learns low-dimensional vector representations (embeddings) of entities and relations (Wang et al., 2021) , which designs a scoring function (SF) based on the embeddings to measure the hyper-relational fact plausibility such that valid ones obtain higher scores than invalid ones. Especially, existing studies mainly consider two aspects of semantic information and structural information in HKG for modeling. The semantic information emphasizes the interaction between entities and relations in a hyper-relational fact. Especially, there is a distinction, a.k.a., semantic difference (Galkin et al., 2020) between the primary triple and attribute-value qualifiers, e.g., the primary triple (Alan Turing, educated at, Cambridge) serves as the fundamental part and preserves the essential knowledge of Alan Turing's education experience at Cambridge, while the attribute-value qualifier (degree, Bachelor) serves as the auxiliary part and enriches the primary triple. To model the semantic information, early studies treat the primary relation and qualifier relations as an n-ary (n≥2) composed relation (Abboud et al., 2020) or multiple semantically equal attributes (Guan et al., 2019; Liu et al., 2021) , largely ignoring the semantic difference. Various SFs are further developed in recent studies (Galkin et al., 2020; Rosso et al., 2020; Yu & Yang, 2021) with semantic difference considered. On the other hand, the structural information focuses on the topological connection between entities in the hyper-relational graph structure, like an entity's neighboring entities under various hyperrelational links, e.g., in Figure 1 Bachelor and Michelle Obama are neighbors of Alan Turing via degree and alumni, respectively. Only few studies (Galkin et al., 2020; Yadati, 2020) extend hypergraph neural network (HGNN) based modules to capture the structural information in HKG, however, empirical results in (Yu & Yang, 2021) demonstrate that removing such modules will not bring performance degradation, i.e., the direct extensions are quite immature for effective structural information capture. Hence, to the best of our knowledge, none of existing studies achieve HKG modeling with both semantic information and structural information completely captured, and it is still an open problem to be addressed. Targeting on this open problem, we look back to KG modeling with an interesting observation that, recent studies (Vashishth et al., 2019; Yu et al., 2021) leverage an encoder-decoder framework for KG modeling, i.e., a powerful graph neural network (GNN) based encoder and an expressive SF-based decoder on triples are leveraged for structural information and semantic information, respectively. Inspired by this, in this paper, we propose an EQuivalent Transformation for HKG modeling, termed as TransEQ. Specifically, TransEQ designs an equivalent transformation on the hyper-relational graph structure, transforming a HKG to a KG with semantic difference considered, based on which a generalized encoder-decoder framework is further developed to capture information. For structural information, TransEQ introduces a GNN-based encoder on transformed KG with transformation characteristics combined. As for semantic information, to measure the plausibility of a hyper-relational fact, TransEQ exploits various SFs in existing HKG modeling studies as the decoder. The sharing embedding mechanism is further designed to capture the semantic relatedness between hyper-relational facts. In this way, with the equivalent transformation, the encoder-decoder framework in TransEQ captures not only structural information but also semantic information, which is the very innovation of this work, just like killing two birds with one stone. Besides, the flexible choice of SF in decoder ensures the full expressivity of TransEQ, representing all types of relations. We further theoretically prove that the proposed transformation is equivalent between a HKG and a KG without information loss. Extensive experiments show that TransEQ achieves the state-of-the-art results, obtaining a 15% relative increase of MRR on the largest benchmark WikiPeople.



RELATED WORKAs described before, related studies mainly exploit two aspects of semantic information and structural information for HKG modeling, considering HKG-based SF design and hyper-relational graph structure, respectively.



Figure 1: An example of a HKG including primary triples and attributevalue qualifiers. The entities/relation in the triple are called as primary entities/relation, and attributes/values in qualifiers are called as qualifier entities/relations.

