ON REPRESENTING LINEAR PROGRAMS BY GRAPH NEURAL NETWORKS

Abstract

Learning to optimize is a rapidly growing area that aims to solve optimization problems or improve existing optimization algorithms using machine learning (ML). In particular, the graph neural network (GNN) is considered a suitable ML model for optimization problems whose variables and constraints are permutation-invariant, for example, the linear program (LP). While the literature has reported encouraging numerical results, this paper establishes the theoretical foundation of applying GNNs to solving LPs. Given any size limit of LPs, we construct a GNN that maps different LPs to different outputs. We show that properly built GNNs can reliably predict feasibility, boundedness, and an optimal solution for each LP in a broad class. Our proofs are based upon the recently-discovered connections between the Weisfeiler-Lehman isomorphism test and the GNN. To validate our results, we train a simple GNN and present its accuracy in mapping LPs to their feasibilities and solutions.

1. INTRODUCTION

Applying machine learning (ML) techniques to accelerate optimization, also known as Learning to Optimize (L2O), is attracting increasing attention. It has been reported that L2O shows great potentials on both continuous optimization (Monga et al., 2021; Chen et al., 2021; Amos, 2022) and combinatorial optimization (Bengio et al., 2021; Mazyavkina et al., 2021) . Many of the L2O works train a parameterized model that takes the optimization problem as input and outputs information useful to classic algorithms, such as a good initial solution and branching decisions (Nair et al., 2020) , and some even directly generate an approximate optimal solution (Gregor & LeCun, 2010). In these works, one is building an ML model to approximate the mapping from an explicit optimization instance either to its key properties or directly to its solution. The ability to achieve accurate approximation is called the representation power or expressive power of the model. When the approximation is accurate, the model can solve the problem or provide useful information to guide an optimization algorithm. This paper tries to address a fundamental but open theoretical problem for



* A major part of the work of Z. Chen was completed during his internship at Alibaba US DAMO Academy. † Corresponding author.

