REVISITING ROBUSTNESS IN GRAPH MACHINE LEARNING

Abstract

Many works show that node-level predictions of Graph Neural Networks (GNNs) are unrobust to small, often termed adversarial, changes to the graph structure. However, because manual inspection of a graph is difficult, it is unclear if the studied perturbations always preserve a core assumption of adversarial examples: that of unchanged semantic content. To address this problem, we introduce a more principled notion of an adversarial graph, which is aware of semantic content change. Using Contextual Stochastic Block Models (CSBMs) and real-world graphs, our results uncover: i) for a majority of nodes the prevalent perturbation models include a large fraction of perturbed graphs violating the unchanged semantics assumption; ii) surprisingly, all assessed GNNs show over-robustness -that is robustness beyond the point of semantic change. We find this to be a complementary phenomenon to adversarial examples and show that including the label-structure of the training graph into the inference process of GNNs significantly reduces over-robustness, while having a positive effect on test accuracy and adversarial robustness. Theoretically, leveraging our new semantics-aware notion of robustness, we prove that there is no robustness-accuracy tradeoff for inductively classifying a newly added node. 1 

1. INTRODUCTION

Graph Neural Networks (GNNs) are seen as state of the art for various graph learning tasks (Hu et al., 2020; 2021) . However, there is strong evidence that GNNs are unrobust to changes to the underlying graph (Zügner et al., 2018; Geisler et al., 2021) . This has led to the general belief that GNNs can be easily fooled by adversarial examples and many works trying to increase the robustness of GNNs through various defenses (Günnemann, 2022) . Originating from the study of deep image classifiers (Szegedy et al., 2014) , an adversarial example has been defined as a small perturbation, usually measured using an ℓ p -norm, which does not change the semantic content (i.e. category) of an image, but results in a different prediction. These perturbations are often termed unnoticeable relating to a human observer for whom a normal and an adversarially perturbed image are nearly indistinguishable (Goodfellow et al., 2015; Papernot et al., 2016) . However, compared to visual tasks, it is difficult to visually inspect (large-scale) graphs. This has led to a fundamental question: What constitutes a small, semantics-preserving perturbation to a graph? The de facto standard in the literature is to measure small changes to the graph's structure using the ℓ 0 -pseudonorm (Zheng et al., 2021; Günnemann, 2022) . Then, the associated threat models restrict the total number of inserted and deleted edges globally in the graph and/or locally per node. However, if the observation of semantic content preservation for these kind of perturbation models transfers to the graph domain can be questioned: Due to the majority of lowdegree nodes in real-world graphs, small ℓ 0 -norm restrictions still allow to completely remove a significant number of nodes from their original neighbourhood. Only few works introduce measures beyond ℓ 0 -norm restrictions. In particular, it was proposed to additionally use different global graph properties as a proxy for unnoticeability, such as the degree distribution (Zügner et al., 2018) , degree assortativity (Li et al., 2021) , or other homophily metrics (Chen et al., 2022) . While these are important first steps, the exact relation between preserving certain graph properties and the graph's semantic content (e.g., node-categories) is unclear (see Appendix B). For instance, one can completely rewire the graph by iteratively interchanging the endpoints of two randomly selected edges and preserve the global degree distribution. As a result, current literature lacks a principled understanding of semantics-preservation in their employed notions of smallness as well as robustness studies using threat models only including provable semantics-preserving perturbations to a graph. We bridge this gap by being the first to directly address the problem of exactly measuring (node-level) semantic content preservation in a graph under structure perturbations. Surprisingly, using Contextual Stochastic Block Models (CSBMs), this leads us to discover a novel phenomenon: GNNs show strong robustness beyond the point of semantic change (see Figure 1 ). This does not contradict the existence of adversarial examples for the same GNNs. Related to the small degree of nodes, we find that common perturbation sets include both: graphs which are truly adversarial as well as graphs with changed semantic content. Our contributions are: 1. We define a semantics-aware notion of adversarial robustness (Section 3) for node-level predictions. Using this, we introduce a novel concept into the graph domain: over-robustness -that is (unwanted) robustness against admissible perturbations with changed semantic content (i.e., changed ground-truth labels). 2. Using CSBMs, we find: i) common perturbations sets, next to truly adversarial examples, include a large fraction of graphs with changed semantic content (Section 5.1); ii) all examined GNNs show significant over-robustness to these graphs (Section 5.2) and we observe similar patterns on realworld datasets (Section 5.2.1). Using ℓ 0 -norm bounded adversaries on CSBM graphs, we find a considerable amount of a conventional adversarial robustness to be in fact over-robustness. 3. Including the known label-structure through Label Propagation (LP) (Huang et al., 2021) into the inference process of GNNs significantly reduces over-robustness with no negative effects on test accuracy or adversarial robustness (Section 5.2) and similar behaviour on real-world graphs. 4. Using semantic awareness, we prove the existence of a model achieving both, optimal robustness and accuracy in classifying an inductively sampled node (Section 4.1), i.e., no robustness-accuracy tradeoff for a non-i.i.d. data setting.

2. PRELIMINARIES

Let n be the number of nodes and d the feature dimension. We denote the node feature matrix X ∈ R n×d , the (symmetric) adjacency matrix A ∈ {0, 1} n×n , and the node labels y ∈ {0, 1} n of which y L ∈ {0, 1} l , l ≤ n are known. We assume a graph has been sampled from a graph data generating distribution D n denoted as (X, A, y) ∼ D n . We study inductive node classification (Zheng et al., 2021) . Due to the non-i.i.d data generation, a node-classifier f may depend its decision on the whole known graph (X, A, y L ). As a result, we write f (X, A, y L ) v to denote the classification of a node v. All GNNs covered in this work are a function f (X, A) only depending on the node features X and adjacency matrix A. A list of the used symbols and abbreviations can be found in Appendix A. Label Propagation. We use label spreading (Zhou et al., 2004) , which builts a classifier f (A, y L ) by taking the row-wise arg max of the iterate F t = αD -1/foot_1 AD -1/2 F t-1 + (1 -α)Y , with D being the diagonal degree matrix; Y ∈ R n×c with Y ij = 1 if i ≤ l and y i L = j, otherwise Y ij = 0; and α ∈ [0, 1]. Similar to Huang et al. (2021) , we combine LP and GNNs by replacing the zero-rows for i > l in Y with GNN soft-predictions, effectively forming a function f (X, A, y L ).



Project page: https://www.cs.cit.tum.de/daml/revisiting-robustness/ CSBMs parametrized as outlined in Section 5 using K = 1.5 and ℓ2-weak attack.



Figure 1: Average degree-dependent nodeclassification robustness. Semantic boundary indicates when the semantics (i.e., the most likely class) of a node of a given degree changes on average. Data from CSBM graphs 2 . All GNNs show robustness beyond the point of semantic change.

