HAS IT REALLY IMPROVED? KNOWLEDGE GRAPH BASED SEPARATION AND FU-SION FOR RECOMMENDATION

Abstract

In this paper we study the knowledge graph (KG) based recommendation systems. We first design the metric to study the relationship between different SOTA models and find that the current recommendation systems based on knowledge graph have poor ability to retain collaborative filtering signals, and higher-order connectivity would introduce noises. In addition, we explore the collaborative filtering recommendation method using GNN and design the experiment to show that the information learned between GNN models stacked with different layers is different, which provides the explanation for the unstable performance of GNN stacking different layers from a new perspective. According to the above findings, we first design the model-agnostic Cross-Layer Fusion Mechanism without any parameters to improve the performance of GNN. Experimental results on three datasets for collaborative filtering show that Cross-Layer Fusion Mechanism is effective for improving GNN performance. Then we design three independent signal extractors to mine the data at three different perspectives and train them separately. Finally, we use the signal fusion mechanism to fuse different signals. Experimental results on three datasets that introduce KG show that our KGSF achieves significant improvements over current SOTA KG based recommendation methods and the results are interpretable.



The recommendation system is the important technique for information filtering, which can help users find the data they want in a large amount of data. Collaborative filtering algorithm is a classical recommendation method, and its main idea is to make recommendations by mining the collaborative signals between users and items. As a deep learning method, graph neural networks (GNN) have been used to effectively mine users' collaborative signals, such as NGCF (Wang et al., 2019d) , LightGCN (He et al., 2020) . Recent works (Wu et al., 2021; Yu et al., 2022) use LightGCN as the backbone to introduce contrastive learning and achieve better performance. From LightGCN (He et al., 2020) to SimGCL (Yu et al., 2022) , the performance is constantly improving. For the convenience of description, we assume that there are two models, M1 and M2. The overall performance of M1 is better than that of M2. In practical applications, for some specific users, the recommendation effect of M1 may be inferior to that of M2. A problem that may be overlooked is, is M1 really an improvement over M2 ? That is, what is the relationship between M1 and M2? Figure 1 shows two possible relationships between M1 and M2. (i 1 ∼ i 6 are commodities, the circle "Test" represents the range of the test set, the circle "M1" represents the range of top K commodities given by the M1 model, the cricle "M2" represents the range of the test the top K commodities given by the M2 model). In Figure 1 (a), M1 learns new things on the basis of retaining the infor-mation of M2, while in Figure 1 (b) the improvement of M1 to M2 is to lose part of the information of M2 and learn more new information. To objectively measure these two cases, we design a new metric, Intersection@N, to measure the differences between two models. Based on this indicator, we conduct experiments between different collaborative filtering models (Wu et al., 2021; Yu et al., 2022) , and between different stacked layers of the same collaborative filtering model (which are described in Section 2.1 and Appendix C), and obtain two findings:(1) The relationship between different collaborative filtering methods using GNN is shown in Figure 1 (b). (2) The relationship between models based on collaborative filtering that uses the same GNN model but stacks different layers is shown in Figure 1 (b), i.e., models that stack higher layers cannot fully "include" models that stack lower layers. Many studies (Wang et al., 2019d; He et al., 2020) pointed out that within a certain range, the more GNN layers are stacked, the higher the model performance is. The performance decreases after exceeding this range. Numerous studies (He et al., 2020; Zhao & Akoglu, 2019) attribute the poor performance to the over-smoothing of nodes caused by multi-layer stacking, and based on this reason, many methods are designed to alleviate the over-smoothing. A common feature of these works is that they all choose a model with fixed stacked L layers as the final model. The default assumption in doing so is that the model with good performance (stacked L layers) is an improvement over the model with stacked T layers of poor performance and the improvement is understood as in Figure 1(a). However our experiments show that this assumption is invalid and instead their relationship should be as shown in Figure 1 (b) (i.e. The first finding). The first observation of this paper is that the recommendation method based on collaborative filtering using GNN does not fully exploit the performance of GNN. Accompanying GNN, knowledge graphs (KG) are introduced into the recommendation systems to improve their performances with auxiliary information. The popular KG based recommendation methods are KGAT (Wang et al., 2019c) and KGIN (Wang et al., 2021) , which connect KG and user-item bipartite graphs through items, thus unify the two into one graph structure. According to KGAT and KGIN, we believe that the KG based recommendation system includes item-based collaborative signals, content signals and attribute-based collaborative signals. These three signals are mined in KGAT and KGIN. The first two signals are mined in the User-Item bipartite graph and the KG respectively, and the third signal is mined in the unified graph by higher-order connectivity (Wang et al., 2019c) . However, we have two observations with this unified graph structure: (1) Poor preservation for collaborative filtering signals. We use the Intersection@N metric to measure the KG-based recommendation method and the collaborative filtering recommendation method, and find the relationship between the two methods are as shown in Figure 1 (b). Existing methods that introduce KG discard half of the information based on collaborative filtering and learn more information introduced by KG, so the performance of the former is higher than that of the latter. (2) Unnecessary information are introduced by higherorder connectivity which makes the propagation path too long. Taking user u 2 recommending i 4 in Figure 5 as an example, a possible path of a path is u 2 like -→ a 3 r2 → i 4 , and the semantic information of this path is that u 2 likes items with a 3 attribute. However, the path given by KGAT is u 2 like → i 2 r1 → a 3 r2 → i 4 . In this path, the information of node i 2 includes content signal and user-based collaborative signal, which is not helpful for the original semantic information and will introduce unnecessary information. In addition, longer propagation paths also introduce noise. Based on above three observations, in this paper we propose a general knowledge graph based separation and fusion model. It consists of three core parts to meet the three challenges mentioned above: Cross-Layer Fusion Mechanism. We find that there are differences in the information learned between models that stack different layers, so we cannot directly select a model that stacks N layers. We design a model-agnostic, general-purpose Cross-Layer Fusion Mechanism without any trainable parameters, which fuses models stacked with different layers and can preserve the information of different models. Signal Extractor. We design three independent, separately trained signal extractors to extract the three kinds of signals in the data mentioned before, which can avoid the mutual influence of each signal. We use the existing collaborative filtering method for item-based collaborative signal extraction and the Cross-Layer Fusion Mechanism is applied to further improve the performance. For



Figure 1: Two categories of improvement

