A NEW FRAMEWORK FOR TENSOR PCA BASED ON TRACE INVARIANTS

Abstract

We consider the Principal Component Analysis (PCA) problem for tensors T ∈ (R n ) ⊗k of large dimension n and of arbitrary order k ≥ 3. It consists in recovering a spike v ⊗k 0



(related to a signal vector v 0 ∈ R n ) corrupted by a Gaussian noise tensor Z ∈ (R n ) ⊗k such that T = βv ⊗k 0 + Z where β is the signal-to-noise ratio. In this paper, we propose a new framework based on tools developed by the theoretical physics community to address this important problem. They consist in trace invariants of tensors built by judicious contractions (extension of matrix product) of the indices of the tensor T. Inspired by these tools, we introduce a new process that builds for each invariant a matrix whose top eigenvector is correlated to the signal for β sufficiently large. Then, we give examples of classes of invariants for which we demonstrate that this correlation happens above the best algorithmic threshold (β ≥ n k/4 ) known so far. This method has many algorithmic advantages: (i) it provides a detection algorithm linear in time and with only O(1) memory requirements (ii) the algorithms are very suitable for parallel architectures and have a lot of potential of optimization given the simplicity of the mathematical tools involved (iii) experimental results show an improvement of the state of the art for the symmetric tensor PCA. We provide experimental results to these different cases that match well with our theoretical findings.

1. INTRODUCTION

Powerful computers and acquisition devices have made it possible to capture and store real-world multidimensional data. For practical applications (Kolda & Bader (2009) ), analyzing and organizing these high dimensional arrays (formally called tensors) lead to the well known curse of dimensionality (Gao et al. (2017) ,Suzuki (2019)). Thus, dimensionality reduction is frequently employed to transform a high-dimensional data set by projecting it into a lower dimensional space while retaining most of the information and underlying structure. One of these techniques is Principal Component Analysis (PCA), which has made remarkable progress in a large number of areas thanks to its simplicity and adaptability (Jolliffe & Cadima (2016); Seddik et al. ( 2019)). In the Tensor PCA, as introduced by Richard & Montanari (2014), we consider a model where we attempt to detect and retrieve an unknown unit vector v 0 from noise-corrupted multi-linear measurements put in the form of a tensor T. Using the notations found below, our model consists in: T = βv ⊗k 0 + Z, with Z a pure Gaussian noise tensor of order k and dimension n with identically independent distributed (iid) standard Gaussian entries: Z i1,i2,...,i k ∼ N (0, 1) and β the signal-to-noise ratio. To solve this important problem, many methods have been proposed. However, practical applications require optimizable and parallelizable algorithms that are able to avoid the high computationally cost due to an unsatisfactory scalability of some of these methods. A summary of the time and space requirement of some existent methods can be found in Anandkumar et al. (2017) . One way to achieve this parallelizable algorithm is through methods based on tensor contractions (Kim et al. ( 2018)) which are extensions of the matrix product. These last years, tools based on tensor contractions have been developed by theoretical physicists where random tensors have emerged as a generalization of random matrices. In this paper, we investigate the algorithmic threshold of

