MULTI-BEHAVIOR DYNAMIC CONTRASTIVE LEARN-ING FOR RECOMMENDATION

Abstract

Dynamic behavior modeling has become an essential task in personalized recommender systems for learning the time-evolving user preference in online platforms. However, most next-item recommendation methods follow the single type behavior learning manner, which notably limits their user representation performance in reality, since the user-item relationships are often multi-typed in real-life applications (e.g., click, tag-as-favorite, review and purchase). To offer better recommendations, this work proposes Evolving Graph Contrastive Memory Network (EGCM) to model dynamic interaction heterogeneity for multi-behavior sequential recommendation. Specifically, we first develop a multi-behavior graph encoder to capture the short-term preference heterogeneity, and preserve the dedicated relation semantics for different types of user-item interactions. In addition, we design a dynamic cross-relational memory network, empowering EGCM to distill the long-term multi-behavior preference of users and the underlying evolving cross-type behavior dependencies over time. To obtain robust and informative user representation with multi-behavior commonality and diversity, we design a multi-behavior contrastive learning paradigm with heterogeneous short-and long-term interest modeling, and provides theoretical analyses to support the modeling of commonality and diversity. Experiments on several real-world datasets show the superiority of our recommender system over various state-of-the-art baselines.

1. INTRODUCTION

Learning user's dynamic preference plays a vital role in recommender systems to predict the next items that users may be interested in Wang et al. (2019) . For example, a family may buy chicken and bread on an online platform for a long time because of their daily needs, and also buy turkeys close to Christmas. The recent advances of neural network architectures has inspired many efforts to model the transitions between temporally-ordered items, due to the strong representation capability of deep learning techniques, e.g., recurrent neural encoder Hidasi et al. (2016) In real-life recommendation scenarios, users often interact with items in various ways, based on their interests which are intrinsically time-evolving and diverse. For instance, different types of user behaviors (e.g., page view, add-to-favorite, purchase) in online retailers may reflect diverse user intentions and heterogeneous user-item relationships Guo et al. (2019); Jin et al. (2020b) . Leaving this fact untouched, single type of behavior modeling in previous chronological user embedding functions is insufficient to comprehensively capture diverse user intents with behavior heterogeneity Xia et al. (2021a) . Hence, time-evolving multi-behavior representations can characterize the various latent factors behind user-item interactions, and maintain dedicated embedding space for different types of dynamic user behaviors in recommender systems. While having realized the importance of modeling behavior-aware time-evolving user-item relationships in recommendation, some key challenges remain to be carefully tackled. Specifically, (1) How to explicitly preserve the dynamic behavior-specific semantics pertinent to each type of user-item interactions over time then delivering and retaining user preferences, is not trivial multi-behavior sequential recommendation. It is critical in for the recommender to distill such heterogeneous itemlevel dependencies with the jointly modeling of short-term and long-term user interests. (2) Learning informative and robust representations of multiplex user-item interactions requires a tailored modeling with a performant recommendation paradigm, which towards the encoding of users' multi-behavior commonality and diversity. While we can embed behavior-specific semantics into individual latent vectors, the understanding of multi-behavior commonality underlying global view of user-specific dynamic preference is critical to multi-behavior modeling. Contributions. In light of these challenges, we propose an Evolving Graph Contrastive Memory (EGCM) framework that can effectively distill the heterogeneous user intentions over time from multi-behavior data in recommendation. Specifically, we first introduce a multi-behavior graph encoder equipped with temporal context embedding for modeling the behavior-aware short-term interests of users. Furthermore, a dynamic cross-relational memory network based on self-attention is proposed to incorporate heterogeneous cross-behavior dependencies into learning user dynamic preferences with cross-behavior relational transitions. In a nutshell, this dynamic multi-behavior modeling allows us to characterize diverse user intents from the long-term perspective behind the interacted item sequence. To enhance the generalizability and robustness of our recommender, we design our multi-behavior contrastive learning paradigm to endow EGCM with the capability of encoding multi-behavior commonality and differentiating the behavior-aware preference of various users. We also provide theoretical analysis of our EGCM model in Supplementary Section. To summarize, the key contributions of this work are presented as follows: • Emphasizing the importance of jointly learning of dynamic preference heterogeneity with multibehavior data and diverse user behavior-aware interests for recommendation. • Proposing a new model EGCM, which integrates the dynamic cross-relational dependency modeling with the multi-behavior contrastive learning paradigm, so as to distill the evolving user-item relationships at the fine-grained level of user preferences. In addition, we perform the theoretical analysis of our proposed EGCM model as presented in the Supplementary Section. • Conducting experiments on three real-world datasets to demonstrate the superiority of EGCM. Further ablation studies and in-depth model analysis justify the rationality of our model design. To support the reproducibility of our experimental results, the model implementations can be found at the anonymous link: https://anonymous.4open.science/r/EGCM. 2021b)) attempt to leverage graph neural networks for encoding the multi-behavior patterns, based on their constructed relationaware heterogeneous user-item interaction graph. One major drawback of existing multi-behavior recommender systems is that they mostly focus on the stationary scenarios, while neglect the timeevolving multi-behavior dependencies from diverse user interest representation.



, convolution-based model Tang & Wang (2018) and attention mechanism Kang & McAuley (2018). More recent sequential recommender systems are built upon the Transformer Sun et al. (2019); Liu et al. (2021b) or Graph Neural Networks (GNNs) Wu et al. (2019); Ma et al. (2020); Wang et al. (2020c) to provide state-of-the-art recommendation performance. Despite their effectiveness, most of existing next-item recommendation approaches rely on only single type of user-item interaction (e.g., click or purchase data), and thus are limited to capture the item-level multi-behavior interaction patterns.

Next-item/Sequential Recommendation. Early studies (e.g., FPMC Rendle et al. (2010)) rely on the Markov chain to tackle the sequential recommendation problem. Many recent efforts have been devoted to learning users' dynamic interests with various neural network encodes, such as the RNN-based method GRU4Rec Hidasi et al. (2016) and CNN-based approach Caser Tang & Wang (2018). In addition, several self-attention relational learning models are introduced to estimate the item correlations, e.g., SASRec Kang & McAuley (2018), BERT4Rec Sun et al. (2019) and TiSASRec Li et al. (2020). Inspired by the strength of graph neural networks, some recent sequential recommender systems are built over the graph-based message passing scheme to encode the multiorder dependencies among items, including MA-GNN Ma et al. (2020), SURGE Chang et al. (2021), and GCE-GNN Wang et al. (2020c). Furthermore, self-supervised learning has been used in recent sequential recommendation methods for data augmentation, like COTREC Xia et al. (2021c) and DHCN Xia et al. (2021c). However, most of existing methods are built on single type of interactions and ignore the heterogeneous behavior-aware user preferences. Multi-Behavior Recommender System. Recently, multi-behavior recommendation has gained considerable attention due to the effectiveness of considering multi-typed user behaviors in boosting the recommendation performance Chen et al. (2021). For example, NMTR Gao et al. (2019) and DIPN Guo et al. (2019) differentiating behavior semantics with multi-task learning schemes. To encode diverse relationships between users and items, some recent studies (e.g., MBGCN Jin et al. (2020b), KHGT Xia et al. (2021a) and MBGMN Xia et al. (

