DUAL PERSONALIZATION ON FEDERATED RECOMMEN-DATION

Abstract

Federated recommendation is a new Internet service architecture that aims to 1 provide privacy-preserving recommendation services in federated settings. Ex-2 isting solutions are used to combine distributed recommendation algorithms and 3 privacy-preserving mechanisms. Thus it inherently takes the form of heavyweight 4



models at the server and hinders the deployment of on-device intelligent models 5 to end-users. This paper proposes a novel Personalized Federated Recommen-6 dation (PFedRec) framework to learn many user-specific lightweight models 7 to be deployed on smart devices rather than a heavyweight model on a server.  When the items are one-hot encoding vectors, we can simply equivalent use item embedding network and item embedding representations.



Moreover, we propose a new dual personalization mechanism to effectively learn 9 fine-grained personalization on both users and items. The overall learning process 10 is formulated into a unified federated optimization framework. Specifically, 11 unlike previous methods that share exactly the same item embeddings across 12 users in a federated system, dual personalization allows mild finetuning of item 13 embeddings for each user to generate user-specific views for item representations 14 which can be integrated into existing federated recommendation methods to 15 gain improvements immediately. Experiments on multiple benchmark datasets 16 have demonstrated the effectiveness of PFedRec and the dual personalization 17 mechanism. Moreover, we provide visualizations and in-depth analysis of the 18 personalization techniques in item embedding, which shed novel insights on the 19 design of RecSys in federated settings. 20 1 INTRODUCTION 21 Federated recommendation is a new service architecture for Internet applications, and it aims to 22 provide personalized recommendation service while preserving user privacy in the federated set-23 tings. Existing federated recommendation systems (Ammad-Ud-Din et al., 2019; Chai et al., 2020; 24 Muhammad et al., 2020; Perifanis & Efraimidis, 2022; Singhal et al., 2021) are usually to be an 25 adaptation of distributed recommendation algorithms by embodying the data locality in federated 26 setting and adding privacy-preserving algorithms with guaranteed protection. However, these im-27 plementations of federated recommendations still inherit the traditional service architecture, which 28 is to deploy large-scale models at servers. Thus it is impractical and inconsistent with the newly 29 raised on-device service architecture, which is to deploy a lightweight model on the device to pro-30 vide service independently without frequently communicating with the server. Given the challenge 31 of implementing data locality on devices in federated settings, the personalization mechanism needs 32 to be reconsidered to better capture fine-grained personalization for end-users. 33 Personalization is the core component of implementing federated recommendation systems. Inher-34 ited from conventional recommendation algorithms, existing federated recommendation frameworks 35 are usually composed of three modules: user embedding to preserve the user's profile, item embed-36 ding to maintain proximity relationships among items, and the score function to predict the user's 37 preference or rating for a given item. They usually preserve user-specific personalization in the user 38 embedding module while sharing consensus on item embeddings and score functions.

This paper proposes a new dual personalization mechanism designed to capture fine-grained two-40 fold personal preferences for users in the federated recommendation system. Inspired by human

annex

beings' decision logic, we believe all modules in the recommendation framework should be used 42 to preserve part of personalization rather than use user embedding only. For example, the score function is to mimic the user's personal decision logic that is natural to be diverse across clients. 

56

The learning procedure is also carefully tailored in a federated setting. A personalized score function 57 will be learned using its own data on the device, and then it won't be sent to the server for model 58 aggregation that usually generates a general view for all devices. Moreover, the personalized item 59 embedding will be implemented through light fine-tuning in a federated learning framework, thus it 60 can leverage both the general view from server and the personalized view from its own data.

61

In summary, we propose a novel federated recommendation framework that integrates both the per-62 sonalized score function and personalized item embedding via light finetuning from the shared item 63 embedding. Our key contributions are summarized as follows.

64

• We propose a novel federated recommendation framework which is more naturally consis-65 tent with layer-wise neural architecture which can better fit federated learning.

66

• We design a novel dual personalization to capture user preferences using a personalized 

