AIA: LEARN TO DESIGN GREEDY ALGORITHM FOR NP-COMPLETE PROBLEMS USING NEURAL NETWORKS

Abstract

Algorithm design is an art that heavily requires intuition and expertise of the human designers as well as insights into the problems under consideration. In particular, the design of greedy-selection rules, the core of greedy algorithms, is usually a great challenge to designer: it is relatively easy to understand a greedy algorithm while it is always difficult to find out an effective greedy-selection rule. In the study, we present an approach, called AIA, to learn algorithm design with the aid of neural networks. We consider the minimum weighted set cover problem (WSCP), one of the NP-hard problems, as an representative example. Initially, we formulate a given WSCP as an 0-1 integer linear program (ILP): each variable x i has two options, i.e., x i = 0, which denotes abandon of the set s i , and x i = 1, which denotes selection of s i . Each option of a variable leads to a sub-problem with respect to the original ILP problem. Next, we design a generic search framework to find the optimal solution to the ILP problem. At each search step, the value of a variable is determined with the aid of neural networks. The key of our neural network is the loss function: the original ILP problem and the sub-problems generated by assigning a variable x i should satisfy the Bellman-Ford equation, which enables us to set the dissatisfication of Bellman-Ford equation as loss function of our neural network. The neural network is used as greedy-selection rule. Experimental results on representative instances suggest that using the NN-based greedy selection rule, we can successfully find the optimal solutions. More importantly, the NN-based greedy-selection rule outperform the outstanding Chavatal greedy algorithm, which was designed by human expert. The basic idea of our approach can be readily extended without significant modification to design greedy algorithm for other NP-hard problems.

1. INTRODUCTION

NP-complete problems, the hardest ones in the NP class, can be validated in polynomial time, but no polynomial-time algorithm has yet been found to solve these problems. A great variety of practical problems can be formulated as NP-complete problems, such as strategic planning, production planning, facility location problems, as well as a variety of scheduling and routing problems. Thus, despite the hardness of these problems, designing efficient solving algorithms for NP-complete problems is highly desired. The weighted set cover problem (WSCP) is a classical NP-complete problem, which aims to find a subset of columns that cover all the rows of a 0-1 matrix at minimal cost (Karp, 1972) . The algorithms to solve WSCP, say branch-and-bound and branch-and-cut, can only handle instances with limited size. Therefore, considerable efforts have been devoted to design heuristics and metaheuristics that can find optimal or near optimal solutions to large-scale WSCP problems within a reasonable time. The latest works on meta-heuristic approaches for the WSCP include genetic algorithms, ant colony optimization, simulated annealing, tabu Search. With the breakthrough of deep learning (DL) in solving practical problems, many researchers try to use DL to solve combinatorial optimization problems. Training a neural network end-to-end with supervised learning to solve theoretically complex combinatorial optimization problems is a difficult problem. On the one hand, traditional algorithms and mathematical methods have a relatively complex theoretical foundation while the neural network as a black box lacks theoretical foundation. 1

