LEARNING TO GENERATE COLUMNS WITH APPLICATION TO VERTEX COLORING

Abstract

We present a new column generation approach based on Machine Learning (ML) for solving combinatorial optimization problems. The aim of our method is to generate high-quality columns that belong to an optimal integer solution, in contrast to the traditional approach that aims at solving linear programming relaxations. To achieve this aim, we design novel features to characterize a column, and develop an effective ML model to predict whether a column belongs to an optimal integer solution. We then use the ML model as a filter to select high-quality columns generated from a sampling method and use the selected columns to construct an integer solution. Our method is computationally fast compared to the traditional methods that generate columns by repeatedly solving a pricing problem. We demonstrate the efficacy of our method on the vertex coloring problem, by empirically showing that the columns selected by our ML model are significantly better, in terms of the integer solution that can be constructed from them, than those selected randomly or based only on their reduced cost. Further, we show that the columns generated by our method can be used as a warm start to boost the performance of a column generation-based heuristic.

1. INTRODUCTION

Machine Learning (ML) has been increasingly used to tackle combinatorial optimization problems (Bengio et al., 2020) , such as learning to branch in Mixed Integer Program (MIP) solvers (Khalil et al., 2016; Balcan et al., 2018; Gupta et al., 2020) , learning heuristic search algorithms (Dai et al., 2017; Li et al., 2018) , and learning to prune the search space of optimization problems (Lauri & Dutta, 2019; Sun et al., 2021b; Hao et al., 2020) . Given a large amount of historical data to learn from, ML techniques can often outperform random approaches and hand-crafted methods typically used in the existing exact and heuristic algorithms. Predicting optimal solutions for combinatorial optimization problems via ML has attracted much attention recently. A series of studies (Li et al., 2018; Lauri & Dutta, 2019; Sun et al., 2021b; Grassia et al., 2019; Fischetti & Fraccaro, 2019; Lauri et al., 2020; Sun et al., 2021a; 2022; Ding et al., 2020; Abbasi et al., 2020; Zhang et al., 2020) have demonstrated that predicting optimal solution values for individual variables can achieve a reasonable accuracy. The predicted solution can be used in various ways (e.g., to prune the search space of a problem (Lauri & Dutta, 2019; Sun et al., 2021b; Hao et al., 2020) or warm-start a search method (Li et al., 2018; Zhang et al., 2020; Sun et al., 2022) ) to facilitate the solving of combinatorial optimization problems. However, for a symmetric optimization problem, predicting optimal values for individual decision variables does not provide much benefit for solving the problem. For example, in the vertex coloring problem (VCP) (See Section 2 for a formal problem definition), a random permutation of the colors in an optimal solution results in an alternative optimal solution, and thus predicting the optimal colors for individual vertices is not very useful. On the other hand, predicting a complete optimal solution for a problem directly is too difficult. This is partially due to the NP-hardness of a problem and the difficulty in designing a generic representation for solutions of different sizes. In this paper, we take an intermediate step by developing an effective ML model to predict columns (or fragments (Alyasiry et al., 2019) ) that belong to an optimal solution of a combinatorial optimization problem. To illustrate our method, we use the VCP as an example, in which a column is a

