FAIR GRAPH MESSAGE PASSING WITH TRANSPARENCY Anonymous

Abstract

Recent advanced works achieve fair representations and predictions through regularization, adversarial debiasing, and contrastive learning in graph neural networks (GNNs). These methods implicitly encode the sensitive attribute information in the well-trained model weight via backward propagation. In practice, we not only pursue a fair machine learning model but also lend such fairness perception to the public. For current fairness methods, how the sensitive attribute information usage makes the model achieve fair prediction still remains a black box. In this work, we first propose the concept transparency to describe whether the model embraces the ability of lending fairness perception to the public or not. Motivated by the fact that current fairness models lack of transparency, we aim to pursue a fair machine learning model with transparency via explicitly rendering sensitive attribute usage for fair prediction in forward propagation . Specifically, we develop an effective and transparent Fair Message Passing (FMP) scheme adopting sensitive attribute information in forward propagation. In this way, FMP explicitly uncovers how sensitive attributes influence final prediction. Additionally, FMP scheme can aggregate useful information from neighbors and mitigate bias in a unified framework to simultaneously achieve graph smoothness and fairness objectives. An acceleration approach is also adopted to improve the efficiency of FMP. Experiments on node classification tasks demonstrate that the proposed FMP outperforms the state-of-the-art baselines in terms of fairness and accuracy on three real-world datasets.

1. INTRODUCTION

Graph neural networks (GNNs) (Kipf & Welling, 2017; Veličković et al., 2018; Wu et al., 2019; Han et al., 2022a; b) are widely adopted in various domains, such as social media mining (Hamilton et al., 2017) , knowledge graph (Hamaguchi et al., 2017) and recommender system (Ying et al., 2018) , due to remarkable performance in learning representations. Graph learning, a topic with growing popularity, aims to learn node representation containing both topological and attribute information in a given graph. Despite the outstanding performance in various tasks, GNNs often inherit or even amplify societal bias from input graph data (Dai & Wang, 2021) . The biased node representation largely limits the application of GNNs in many high-stake tasks, such as job hunting (Mehrabi et al., 2021) and crime ratio prediction (Suresh & Guttag, 2019) . Hence, bias mitigation that facilitates the research on fair GNNs is in an urgent need. Many existing works achieving fair prediction in graph either rely on regularization (Jiang et al., 2022) , adversarial debiasing (Dai & Wang, 2021) , or contrastive learning (Zhu et al., 2020; 2021b; Agarwal et al., 2021; Kose & Shen, 2022) . These methods adopt sensitive attribute information in training loss refinement. In this way, such sensitive attribute can be implicitly encoded in well-trained model weight through backward propagation. However, only achieving fair model is insufficient in practice since the fairness should also lend perception to the public (e.g., the auditors, or the maintainers of machine learning systems). In other words, the influence of sensitive attributes should be easily probed for public. We name such property for public probing as transparency. Specifically, we provide the following formal statement on transparency in fairness: Transparency in fairness: Onlookers can verify the released fair model with • Transparent influence: How and if the sensitive attribute information influence fair model prediction. • Less is more: The required resources to obtain the influence of sensitive attributes only includes well-trained model and test data samplesfoot_0 . From auditors' perspective, even though the fairness metric for machine learning model is low, such fair model is still untrustful if the auditors cannot understand how the sensitive attributes are adopted to achieve fair prediction given the well-trained model. From maintainer's perspective, it is important to understand how the model provide fair prediction. Such understanding could help maintainers improve models and further convince auditors in terms of fairness. In summary, transparency aims to make fairness implementation understandable. The transparency aims to make the process of achieving fair model via sensitive attribute informations white-boxfoot_1 . Therefore, the maintainers and auditors both get benefits from model transparency. More importantly, similar to intrinsic explainability of the modelfoot_2 , the fairness with transparency property is binary, i.e., the prediction model either embrace the fairness with transparency or not. Based on the formal statement on transparency in fairness, the key rule to determinate whether a fair model is transparent is that the model prediction difference under the cases with and without sensitive attribute information can be identified given the well-trained fair model and test data samples. Unfortunately, many existing fairness works, including regularization, adversarial debiasing, and contrastive learning, do not satisfy transparency requirements in practice. For example, the fair model trained based on existing works are not with transparent influence. This is because the sensitive attribute is implicitly encoded in well-trained model weight. Therefore, it is intractable to infer how sensitive attribute influence the well-trained model weight without access the dynamic model training process. Additionally, for fair model obtained from existing works, the required resources for transparent influence includes training data and training strategy so that the influence of sensitive attributed can be probed via detecting well-trained model weight difference. In a nutshell, the current fair models based on loss refinement lack of transparency. A natural question is raised: Can we find fair prediction model with transparency? In this work, we provide a positive answer via chasing transparency and fairness in message passing of GNNs. The key idea of achieving transparency is to explicitly adopt sensitive attribute in message passing (forward propagation). Specifically, we design an fair and transparent message passing scheme for GNNs, called fair message passing (FMP). First, we formulate an optimization problem that integrates fairness and prediction performance objectives. Then, we solve the formulated problem via Fenchel conjugate and gradient descent to generate fair-and-predictive representation. We also interpret the gradient descent as aggregation first and them debiasing. Finally, we integrate FMP in graph neural networks to achieve fair and accurate prediction for node graph classification task. Further, we demonstrate the superiority of FMP by examining its effectiveness and efficiency, where we adopt the property of softmax function to accelerate the gradient calculation over primal variables. In short, the contributions can be summarized as follows: • We consider fairness problem from a new perspective, named transparency, i.e., the sensitive attribute should be easily probed for public. We point out that many existing fairness method cannot achieve transparency. • We propose FMP to achieve fairness with transparency via using sensitive attribute information in message passing. Specifically, we use gradient descent to chasing graph smoothness and fairness in a unified optimization framework. An acceleration method is proposed to reduce gradient computational complexity with theoretical and empirical validation. • The effectiveness and efficiency of FMP are experimentally evaluated on three real-world datasets. The results show that compared to the state-of-the-art, our FMP exhibits a superior trade-off between prediction performance and fairness with negligibly computation overhead.



A naive way for many existing works (e.g., adding fair regularization, adversarialdebiasing, et.al.) to obtain the influence of sensitive attribute is to train fair and unfair model with/without the sensitive attribute information, and then get the prediction difference. Therefore, the required resources includes training data and additional (unfair) model training. Similar with the goal of model explainability, only achieving accurate prediction is insufficient, and chasing explainability can help experts understand how the model provide prediction and convince users. Decision tree is intrinsic explainable while deep neural networks is not.

