CHASING ALL-ROUND GRAPH REPRESENTATION RO-BUSTNESS: MODEL, TRAINING, AND OPTIMIZATION

Abstract

Graph Neural Networks (GNNs) have achieved state-of-the-art results on a variety of graph learning tasks, however, it has been demonstrated that they are vulnerable to adversarial attacks, raising serious security concerns. A lot of studies have been developed to train GNNs in a noisy environment and increase their robustness against adversarial attacks. However, existing methods have not uncovered a principled difficulty: the convoluted mixture distribution between clean and attacked data samples, which leads to sub-optimal model design and limits their frameworks' robustness. In this work, we first begin by identifying the root cause of mixture distribution, then, for tackling it, we propose a novel method GAME -Graph Adversarial Mixture of Experts to enlarge model capacity and enrich the representation diversity of adversarial samples, from three perspectives of model, training, and optimization. Specifically, we first propose a plug-and-play GAME layer that can be easily incorporated into any GNNs and enhance their adversarial learning capabilities. Second, we design a decoupling-based graph adversarial training in which the component of the model used to generate adversarial graphs is separated from the component used to update weights. Third, we introduce a graph diversity regularization that enables the model to learn diverse representation and further improves model performance. Extensive experiments demonstrate the effectiveness and advantages of GAME over the state-of-the-art adversarial training methods across various datasets given different attacks.

1. INTRODUCTION

Graph neural networks (GNNs) have been demonstrated to be effective at learning from graphs. They explore a message-passing mechanism to update node representations by iteratively aggregating information from their neighbors, allowing GNNs to achieve state-of-the-art performance (Kipf & Welling, 2017; Veličković et al., 2018; Hamilton et al., 2017) . Many real-world applications are based on GNNs, such as modeling over social networks (Fan et al., 2022; Zhang et al., 2019; Hu et al., 2020) , scene graph reasoning (Chen et al., 2020; Zhang et al., 2022) , and biological molecules (Jin et al., 2018; Xu et al., 2019; Guo et al., 2022) . Nevertheless, despite their outstanding performance, GNNs are susceptible to perturbations (Zügner et al., 2018b; Zügner & Günnemann, 2019; Zheng et al., 2021; Yue et al., 2022) , which necessitate techniques to leverage GNN's robustness against adversarial attacks. Attackers can downgrade the performance of GNNs from multiple perspectives, such as adding or removing edges (Geisler et al., 2021; Chen et al., 2023) , perturbing node properties (Zügner & Günnemann, 2019; Sun et al., 2020; Tian et al., 2023) , and injecting malicious nodes (Zou et al., 2021; Ju et al., 2023) . To enhance GNN's robustness, multiple defense methods against graph attacks have been proposed (Jin et al., 2020; Entezari et al., 2020; Zhang & Zitnik, 2020) . However, most existing methods have not uncovered the principled difficulty (i.e., the convoluted mixture distribution between clean and attacked data samples), which results in sub-optimal model design, poor robustness, and limited performance. In light of this, we study the robustness of GNNs from a more fundamental perspective by discovering the key pattern behind the adversarial attacks that jeopardizes the performance of GNNs.

