A GNN-GUIDED PREDICT-AND-SEARCH FRAME-WORK FOR MIXED-INTEGER LINEAR PROGRAMMING

Abstract

Mixed-integer linear programming (MILP) is widely employed for modeling combinatorial optimization problems. In practice, similar MILP instances with only coefficient variations are routinely solved, and machine learning (ML) algorithms are capable of capturing common patterns across these MILP instances. In this work, we combine ML with optimization and propose a novel predict-and-search framework for efficiently identifying high-quality feasible solutions. Specifically, we first utilize graph neural networks to predict the marginal probability of each variable, and then search for the best feasible solution within a properly defined ball around the predicted solution. We conduct extensive experiments on public datasets, and computational results demonstrate that our proposed framework achieves 51.1% and 9.9% performance improvements to MILP solvers SCIP and Gurobi on primal gaps, respectively.

1. INTRODUCTION

Mixed-integer linear programming is one of the most widely used techniques for modeling combinatorial optimization problems, such as production planning (Pochet & Wolsey, 2006; Chen, 2010) , resource allocation (Liu & Fan, 2018; Watson & Woodruff, 2011) , and transportation management (Luathep et al., 2011; Schöbel, 2001) . In real-world settings, MILP models from the same application share similar patterns and characteristics, and such models are repeatedly solved without making uses of those similarities. ML algorithms are well-known for its capability of recognizing patterns (Khalil et al., 2022) , and hence they are helpful for building optimization algorithms. Recent works have shown the great potential of utilizing learning techniques to address MILP problems. The work of (Bengio et al., 2021) categorized ML efforts for optimization as (i) end-to-end learning (Vinyals et al., 2015; Bello* et al., 2017; Khalil et al., 2022) , (ii) learning to configuring algorithms (Bischl et al., 2016; Kruber et al., 2017; Gasse et al., 2022) and (iii) learning alongside optimization (Gasse et al., 2019; Khalil et al., 2016; Gupta et al., 2020) . In this work, for the sake of interest, we focus on the end-to-end approach. While such an approach learns to quickly identify high-quality solutions, it generally faces the following two challenges: (I) high sample collection cost. The supervised learning task for predicting solutions is to map from the instance-wise information to a high-dimensional vector. Such a learning task becomes computationally expensive since it necessitates collecting a considerable amount of optimal solutions (see, e.g., Kabir et al. (2009) ). (II) feasibility. Most of the end-to-end approaches directly predict solutions to MILP problems, ignoring feasibility requirements enforced by model constraints (e.g. Yoon (2022); Nair †Equal first authorship *Corresponding authors et al. ( 2020)). As a result, the solutions provided by ML methods could potentially violate constraints. We propose a novel predict-and-search framework to address the aforementioned challenges. In principle, end-to-end approaches for MILP problems require the collection of abundant optimal solutions. However, collecting so many training samples is computationally prohibitive since obtaining optimal solutions is excessively time-consuming. We consider to approximate the distribution by properly weighing high-quality solutions with their objective values. This reduces the cost of sample collections by avoiding gathering optimal solutions as mentioned in challenge (I). Regarding challenge (II), we implement a trust region inspired algorithm that searches for near-optimal solutions within the intersection of the original feasible region and a properly defined ball centered at a prediction point. The overall framework is outlined in Figure 1 . The commonly used end-to- Figure 1: Our approach first predicts a marginal probability of each variable utilizing a graph neural network (GNN) with graph convolution and multi-layer perceptron modules, and then searches for near optimal solutions to the original MILP problem within a well defined trust region. end approaches usually fix variables directly based on information from the prediction step, such as in Nair et al. ( 2020) and Yoon (2022). However, such approaches could lead to sub-optimal solutions or even infeasible sub-problems. Rather than forcing variables to be fixed, our search module looks for high-quality solutions within a subset of the original feasible region, which allows better feasibility while maintaining optimality. The distinct contributions of our work can be summarized as follows. • We propose a novel predict-and-search framework that first trains a GNN to predict a marginal probability of each variable and then constructs a trust region to search for highquality feasible solutions. • We demonstrate the ability of our proposed framework to provide equivalently good or better solutions than fixing-based end-to-end approaches. • We conduct comprehensive computational studies on several public benchmark datasets and the computational results show that our proposed framework achieves 51.1% and 9.9% smaller primal gaps than state-of-the-art general-purpose optimization solvers SCIP and Gurobi, respectively. We make our code publicly available at https://github.com/sribdcn/Predict-and -Search_MILP_method.

2. PRELIMINARIES

Given a vector v ∈ R n and an index set I ⊆ {1, 2, ..., n}, let v I ∈ R |I| denote a subvector that corresponds to the index set I.

2.1. MIXED-INTEGER LINEAR PROGRAMMING

MILP techniques are used to model combinatorial optimization problems, and an MILP instance can be formulated as: min x∈D c ⊤ x, where D ≡ {x ∈ Z q × R n-q : Ax ≤ b, l ≤ x ≤ u} denotes the

