LEARNING LATENT TOPOLOGY FOR GRAPH MATCHING

Abstract

Graph matching (GM) has been traditionally modeled as a deterministic optimization problem characterized by an affinity matrix under pre-defined graph topology. Though there have been several attempts on learning more effective node-level affinity/representation for matching, they still heavily rely on the initial graph structure/topology which is typically obtained through heuristic ways (e.g. Delaunay or k-nearest) and will not be adjusted during the learning process to adapt to problem-specific patterns. We argue that such a mechanism for learning on the fixed topology may restrict the potential of a GM solver for specific tasks, and propose to learn latent graph topology in replacement of the fixed topology as input. To this end, we devise two types of latent graph generation procedures in a deterministic and generative fashion, respectively. Particularly, the generative procedure emphasizes the across-graph consistency and thus can be viewed as a matching-guided generative model. Our methods show superior performance over previous state-of-the-arts on public benchmarks.

1. INTRODUCTION

Being a long standing NP-hard problem (Loiola et al., 2007) , graph matching (GM) has received persistent attention from the machine learning and optimization communities for many years. Concretely, for two graphs with n nodes for each, graph matching seeks to solvefoot_0 : max z z Mz s.t. Z ∈ {0, 1} n×n , Hz = 1 (1) where the affinity matrix M ∈ R n 2 ×n 2 + encodes node (diagonal elements) and edge (off-diagonal) affinities/similarities and z is the column-wise vectorization form of the permutation matrix Z. H is a selection matrix ensuring each row and column of Z summing to 1. 1 is a column vector filled with 1. Eq. ( 1) is the so-called quadratic assignment problem (QAP) (Cho et al., 2010) . Maximizing Eq. (1) amounts to maximizing the sum of the similarity induced by matching vector Z. While Eq. (1) does not encode the topology of graphs, Zhou & Torre (2016) further propose to factorize M to explicitly incorporate topology matrix, where a connectivity matrix A ∈ {0, 1} n×n is used to indicate the topology of a single graph (A ij = 1 if there exists an edge between nodes i and j; A ij = 0 otherwise). To ease the computation, Eq. ( 1) is typically relaxed by letting z ∈ [0, 1] n 2 and keeping other parts of Eq. ( 1) intact. Traditional solvers to such relaxed problem generally fall into the categories of iterative update (Cho et al., 2010; Jiang et al., 2017) or numerical continuation (Zhou & Torre, 2016; Yu et al., 2018) , where the solvers are developed under two key assumptions: 1) Affinity M is pre-computed with some non-negative metrics, e.g. Gaussian kernel, L 2 -distance or Manhattan distance; 2) Graph topology is pre-defined as input either in dense (Schellewald & Schnörr, 2005) or sparse (Zhou & Torre, 2016) fashion. There have been several successful attempts towards adjusting the first assumption by leveraging the power of deep networks to learn more effective graph representation for GM (Wang et al., 2019a; Yu et al., 2020; Fey et al., 2020) . However, to our best knowledge, there is little previous work questioning and addressing the problem regarding the second assumption in the context of learning-based graph matching 2 . For example, existing



Without loss of generality, we discuss graph matching under the setting of equal number of nodes without outliers. The unequal case can be readily handled by introducing extra constraints or dummy nodes. Bipartite matching and graph isomorphism are subsets of this quadratic formulation(Loiola et al., 2007).2 There are some loosely related works(Du et al., 2019; 2020) on network alignment and link prediction without learning, which will be discussed in detail in the related works.

