MATRIX DATA DEEP DECODER -GEOMETRIC LEARN-ING FOR STRUCTURED DATA COMPLETION

Abstract

In this work we present a fully convolutional end to end method to reconstruct corrupted sparse matrices of Non-Euclidean data.The classic example for such matrices is recommender systems matrices where the rows/columns represent items/users and the entries are ratings. The method we present is inspired by the surprising and spectacular success of methods like" deep image prior" and "deep decoder" for corrupted image completion. In sharp contrast to previous Matrix Completion methods wherein the latent matrix or its factors directly serve as the optimization variable, in the method we present, the matrix is parametrized as the weights of a graph neural network acting on a random noisy input. Then we are tuning the network parameters to get a result as close as possible to the initial sparse matrix (using it's factors) getting that way state of the art matrix completion result. In addition to the conceptual simplicity of our method, which is just non-Euclidean generalization of deep image priors, it holds less parameters then previously presented methods which makes the parameters more trackable and the method more computationally efficient and more applicable for the real world tasks.The method also achieves state-of-the-art results for the matrix completion task on the classical benchmarks in the field. The method also surprisingly shows that untrained convolutional neural network can use a good prior not only for image completion but also for Matrix Completion when redefined for graphs.

1. INTRODUCTION

Matrix completion (MC) consists of estimating the missing entries of an n × m matrix X (usually, of very big dimensions) given its measurements M on a (usually, very sparse) support Ω. An example of such matrices are signals on graphs/manifolds which are Non-Euclidean domains. The classical example of such data are recommender (recommendation) systems, where the ratings are signals on (user item) couple. The most known Matrix Completion problem is the Netflix problem, where a 1M $ prize was offered for the algorithm that can best predict user ratings in a dataset that contained 480k movies × 18k users (8.5B entries), with 0.011% known entries (Bell et al., 2009) . Many works focused on solutions for the MC problem. In brief, one wishes to obtain the matrix X given matrix M as the specified input on the support Ω. Then, formally the completion task amounts to the minimization problem X = argmin X A Ω • (X -M ) 2 F where A Ω is the observation mask matrix (filled with 1 where data exists in the original problem), • is the Hadamard product and .

2

F is the Frobenius norms (Rennie & Srebro, 2005) . Different approaches where presented in order to fill in matrix X. Those approached included imposing different regularization (priors) on the matrix and its factors. The most prominent approach consists of imposing a low rank (Candès & Recht, 2009; Recht, 2009) on the matrix. Then, priors based on collaborative filtering (users/items rating patterns), content based filtering (user/items profile) (Ghassemi et al., 2018; Jain & Dhillon, 2013; Xu et al., 2013; Si et al., 2016) and their combinations. Then Geometric Matrix Completion approaches appeared (Li & Yeung, 2009; Rao et al., 2015; Cai et al., 2011) and proposed describing rows/column graphs which represent similarity, then encoding the structural (geometric) information of those graphs via graph Laplacian regularization (Belkin & Partha, 2002; Belkin & Niyogi, 2003) and imposing smoothness of the data in those graphs (Kalofolias et al., 2014; Rao et al., 2015; Ma et al., 2011; Mardani et al., 2012) . Those approaches where generally related to the field of signal processing as entries signals on the rows/columns graphs (Shuman et al., 2012) . Then Geometric Deep Learning Methods where introduced to learn the domains of geometric data structures (e.g. single graphs or manifolds) (Bronstein et al., 2016; Lefkimmiatis, 2016; Defferrard et al., 2016; Niepert et al., 2016; Gilmer et al., 2017; Hamilton et al., 2017; Velickovic et al., 2017; Chen et al., 2018; W. Huang et al., 2018; Klicpera et al., 2018; Abu-El-Haija et al., 2019; Ying et al., 2018; Gao et al., 2018; Hammond et al., 2011) . The current state of the art solution for Matrix completion problem, relies on an extending classical harmonic analysis methods to non-Euclidean domains. When, the geometry of the column/row spaces and their graphs is utilised to provide a Geometric Deep Learning mechanism called the RMGCNN (Monti et al., 2017) that includes a complex combined CNN and RNN(Hochreiter & Schmidhuber, 1997) networks. In this work we present a simplified method for the MC problem: the Matrix Data Deep Decoder that contains a classical end to end GRAPH convolutional neural network and inspired by the leading methods from the field of image completion -the Deep Image Prior (Ulyanov et al., 2020) and the Deep Decoder (Heckel & Hand, 2018) . In our method, random noisy input matrix is acted upon by the weights of a neural network (parametrization). By tuning the parameters of the network and minimising the error between its output to the initial corrupted matrix, we find the best candidate for the complete matrix. This method yields state of art results for the MC task. The contributions of our work are: • A novel approach for solving the MC Problem, using deep learning with end-to-end pure convolutional network for graphs. • State-of-the-art performance for the MC problem in both prediction error (RMSE) and solution running timefoot_0 . Our method significantly outperforms the previous state of art method -the RMGCNN. • We show that a pure graph convolutional neural network is a good prior for the MC problem. This provides a correspondence of convolutionial neural networks methods to MC problems.

2.1. MATRIX COMPLETION NOTATION

The most prominent prior for the MC problem is assuming the matrix X is of low rank. Low rank is obtained by rank regularization using its nuclear (trace) norm X * -sum of the singular values of X. The canonical optimization problem with parameter λ * , is stated as: X = min X A Ω • (X -M ) 2 F + λ * X * 2.1.1 MATRIX FACTORIZATION To alleviate the computational burden for big datasets, we factorize X = W H T , where W ∈ R m×k , H T ∈ R k×n . Here, k m and n is the upper bound on the rank of X. With this factorization, the nuclear norm term can be replaced by the sum of the Frobenius norms leading to the following non-convex (but still very well-behaved) problem (Rao et al., 2015) : X = min W ,H T A Ω • W H T -M 2 F + λ * 2 W 2 F + H T 2 F 2.2 GEOMETRIC MATRIX COMPLETION We introduce the geometric matrix completion framework, using notations as in RMGCNN (Monti et al., 2017) .



evaluated on the existing classical benchmark for MC Problems

