GRAPH DOMAIN ADAPTATION VIA THEORY-GROUNDED SPECTRAL REGULARIZATION

Abstract

Transfer learning on graphs drawn from varied distributions (domains) is in great demand across many applications. Emerging methods attempt to learn domaininvariant representations using graph neural networks (GNNs), yet the empirical performances vary and the theoretical foundation is limited. This paper aims at designing theory-grounded algorithms for graph domain adaptation (GDA). (i) As the first attempt, we derive a model-based GDA bound closely related to two GNN spectral properties: spectral smoothness (SS) and maximum frequency response (MFR). This is achieved by cross-pollinating between the OT-based (optimal transport) DA and graph filter theories. (ii) Inspired by the theoretical results, we propose algorithms regularizing spectral properties of SS and MFR to improve GNN transferability. We further extend the GDA theory into the more challenging scenario of conditional shift, where spectral regularization still applies. (iii) More importantly, our analyses of the theory reveal which regularization would improve performance of what transfer learning scenario, (iv) with numerical agreement with extensive real-world experiments: SS and MFR regularizations bring more benefits to the scenarios of node transfer and link transfer, respectively. In a nutshell, our study paves the way toward explicitly constructing and training GNNs that can capture more transferable representations across graph domains.

1. INTRODUCTION

Many applications call for "transferring" graph representations learned from one distribution (domain) to another, which we refer to as graph domain adaptation (GDA). Examples include temporally-evolved social networks (Wang et al., 2021) , molecules of different scaffolds (Hu et al., 2019) , and protein-protein interaction networks in various species (Cho et al., 2016) . In general, this setting of transfer learning is challenging due to the data-distribution shift between the training (source) and test (target) domains (i.e. P S (G, Y ) ̸ = P T (G, Y )). In particular, such a challenge escalates for graph-structured data that are abstractions of diverse nature (You et al., 2021; 2022) . Despite the tremendous needs arising from real-world applications, current methods for GDA (as reviewed in Section 2) mostly fall short in delivering competitive target performance with theoretical guarantee. Inevitably those approaches assuming distribution invariance (or adopting heuristic principles) are restricted in theory (Garg et al., 2020; Verma & Zhang, 2019) . The emerging approaches (Zhang et al., 2019; Wu et al., 2020) straightforwardly apply adversarial training between source and target representations, intentionally founded on the DA theory to bound the target risk (Redko et al., 2020) . However, the generic DA bound in theory is agnostic to graph data and models, which could be more precisely tailored for graphs. We therefore set out to explore the following question: How to design algorithms to boost transfer performance across different graph domains, with the grounded theoretical foundation? Our step-by-step answers are as follows. (i) Derivation of model-based GDA bound. Building upon the rigorous assurance established in the DA theory (Section 3), we start by directly rewriting the OT-based (optimal transport) DA bound (Redko et al., 2017; Shen et al., 2018) for graphs (Corollary 1), which is closely coupled with the Lipschitz constant of graph encoders. The nontrivial challenge here is how to formulate GNN Lipschitz w.r.t the distance metric of non-Euclidean data. Leveraging the graph filter theory (Gama et al., 2020; Arghal et al., 2021) , we first state that GNNs can be constructed stably w.r.t. the 1

availability

://github.com/Shen-Lab/

