NEURALLY AUGMENTED ALISTA

Abstract

It is well-established that many iterative sparse reconstruction algorithms can be unrolled to yield a learnable neural network for improved empirical performance. A prime example is learned ISTA (LISTA) where weights, step sizes and thresholds are learned from training data. Recently, Analytic LISTA (ALISTA) has been introduced, combining the strong empirical performance of a fully learned approach like LISTA, while retaining theoretical guarantees of classical compressed sensing algorithms and significantly reducing the number of parameters to learn. However, these parameters are trained to work in expectation, often leading to suboptimal reconstruction of individual targets. In this work we therefore introduce Neurally Augmented ALISTA, in which an LSTM network is used to compute step sizes and thresholds individually for each target vector during reconstruction. This adaptive approach is theoretically motivated by revisiting the recovery guarantees of ALISTA. We show that our approach further improves empirical performance in sparse reconstruction, in particular outperforming existing algorithms by an increasing margin as the compression ratio becomes more challenging.

1. INTRODUCTION AND RELATED WORK

Compressed sensing deals with the problem of recovering a sparse vector from very few compressive linear observations, far less than its ambient dimension. Fundamental works of Candes et al. (Candès et al., 2006) and Donoho (Donoho, 2006) show that this can be achieved in a robust and stable manner with computationally tractable algorithms given that the observation matrix fulfills certain conditions, for an overview see Foucart & Rauhut (2017) . Formally, consider the set of s-sparse vectors in R N , i.e. Σ N s := x ∈ R N x 0 ≤ s where the size of the support of x is denoted by x 0 := |supp(x)| = |{i : x i = 0}|. Furthermore, let Φ ∈ R M ×N be the measurement matrix, with typically M N . For a given noiseless observation y = Φx * of an unknown but s-sparse x * ∈ Σ N s we therefore wish to solve: argmin x x 0 s.t. y = Φx In (Candès et al., 2006 ) it has been shown, that under certain assumptions on Φ, the solution to the combinatorial problem in (1) can be also obtained by a convex relaxation where one instead minimizes the 1 -norm of x. The Lagrangian formalism yields then an unconstrained optimization problem also known as LASSO (Tibshirani, 1996) , which penalizes the 1 -norm via the hyperparameter λ ∈ R: x = argmin x 1 2 y -Φx 2 2 + λ x 1 A very popular approach for solving this problem is the iterative shrinkage thresholding algorithm (ISTA) (Daubechies et al., 2003) , in which a reconstruction x (k) is obtained after k iterations from initial x (0) = 0 via the iteration: 1 x (k+1) = η λ/L x (k) + 1 L Φ T (y -Φx (k) )



3) * equal contribution † The work is partially funded by DFG grant JU 2795/3 and the German Federal Ministry of Education and Research (BMBF) in the framework of the international future AI lab "AI4EO -Artificial Intelligence for Earth Observation: Reasoning, Uncertainties, Ethics and Beyond" (Grant number: 01DD20001).

