SOLVING MIN-MAX OPTIMIZATION WITH HIDDEN STRUCTURE VIA GRADIENT DESCENT ASCENT

Abstract

Many recent AI architectures are inspired by zero-sum games, however, the behavior of their dynamics is still not well understood. Inspired by this, we study standard gradient descent ascent (GDA) dynamics in a specific class of non-convex non-concave zero-sum games, that we call hidden zero-sum games. In this class, players control the inputs of smooth but possibly non-linear functions whose outputs are being applied as inputs to a convex-concave game. Unlike general zerosum games, these games have a well-defined notion of solution; outcomes that implement the von-Neumann equilibrium of the "hidden" convex-concave game. We prove that if the hidden game is strictly convex-concave then vanilla GDA converges not merely to local Nash, but typically to the von-Neumann solution. If the game lacks strict convexity properties, GDA may fail to converge to any equilibrium, however, by applying standard regularization techniques we can prove convergence to a von-Neumann solution of a slightly perturbed zero-sum game. Our convergence guarantees are non-local, which as far as we know is a firstof-its-kind type of result in non-convex non-concave games. Finally, we discuss connections of our framework with generative adversarial networks.

1. INTRODUCTION

Traditionally, our understanding of convex-concave games revolves around von Neumann's celebrated minimax theorem, which implies the existence of saddle point solutions with a uniquely defined value. Although many learning algorithms are known to be able to compute such saddle points (Cesa-Bianchi & Lugoisi, 2006) , recently there has there has been a fervor of activity in proving stronger results such as faster regret minimization rates or analysis of the day-to-day behavior (Mertikopoulos et al., 2018; Daskalakis et al., 2018; Bailey & Piliouras, 2018; Abernethy et al., 2018; Wang & Abernethy, 2018; Daskalakis & Panageas, 2019; Abernethy et al., 2019; Mertikopoulos et al., 2019; Bailey & Piliouras, 2019; Gidel et al., 2019; Zhang & Yu, 2019; Hsieh et al., 2019; Bailey et al., 2020; Mokhtari et al., 2020; Hsieh et al., 2020; Pérolat et al., 2020) . This interest has been largely triggered by the impressive successes of AI architectures inspired by min-max games such as Generative Adversarial Networks (GANS) (Goodfellow et al., 2014a ), adversarial training (Madry et al., 2018) and reinforcement learning self-play in games (Silver et al., 2017) . Critically, however, all these applications are based upon non-convex non-concave games, our understanding of which is still nascent. Nevertheless, some important early work in the area has focused on identifying new solution concepts that are widely applicable in general min-max games, such as (local/differential) Nash equilibrium (Adolphs et al., 2019; Mazumdar & Ratliff, 2019 ), local minmax (Daskalakis & Panageas, 2018 ), local minimax (Jin et al., 2019) , (local/differential) Stackleberg equilibrium (Fiez et al., 2020 ), local robust point (Zhang et al., 2020) . The plethora of solutions concepts is perhaps suggestive that "solving" general min-max games unequivocally may be too ambitious a task. Attraction to spurious fixed points (Daskalakis & Panageas, 2018) , cycles (Vlatakis-Gkaragkounis et al., 2019) , robustly chaotic behavior (Cheung & Piliouras, 2019; Cheung & Piliouras, 2020) and computational hardness issues (Daskalakis et al., 2020) all suggest that general min-max games might inherently involve messy, unpredictable and complex behavior. Are there rich classes of non-convex non-concave games with an effectively unique game theoretic solution that is selected by standard optimization dynamics (e.g. gradient descent)?

