PRINCIPAL TRADE-OFF ANALYSIS

Abstract

The focus on equilibrium solutions in games underemphasizes the importance of understanding their overall structure. A different set of tools is needed for learning and representing the general structure of a game. In this paper we illustrate "Principle Trade-off Analysis" (PTA), a decomposition method that embeds games into a low dimensional feature space and argue that the embeddings are more revealing than previously demonstrated. Here, we develop an analogy to Principal Component Analysis (PCA). PTA represents an arbitrary two-player zero-sum game as the weighted sum of pairs of orthogonal 2D feature planes. We show that each of the feature planes represent unique strategic trade-offs (cyclic modes) and truncation of the sequence provides insightful model reduction. We demonstrate the validity of PTA on a pair of games (Blotto, Pokemon). In Blotto, PTA identifies game symmetries, and specifies strategic trade-offs associated with distinct win conditions. These symmetries reveal limitations of PTA unaddressed in previous work. For Pokemon, PTA recovers clusters that naturally correspond to Pokemon types, correctly identifies the designed tradeoff between those types, and discovers a rock-paper-scissor (RPS) cycle in the Pokemon generation type -all absent any specific information except game outcomes.

1. INTRODUCTION

In recent years algorithms have achieved superhuman performance in a number of complex games such as Chess, Go, Shogi, Poker and Starcraft (Silver et al., 2018; Heinrich & Silver, 2016; Moravčík et al., 2017; Vinyals et al., 2019) . Despite impressive game play, enhanced understanding of the game is typically only achieved by additional analysis of the algorithms game play post facto (Silver, 2018) . Current work emphasizes the "policy problem", developing strong agents, despite growing demand for a task theory which addresses the "problem problem", i.e. what games are worth study and play (Omidshafiei et al., 2020; Clune, 2019) . A task theory requires a language that characterizes and categorizes games, namely, a toolset of measures and visualization techniques that evaluate and illustrate game structure. Summary visuals and measures are especially important for complex games where direct analysis is intractable. In this vain tournaments are used to sample the game and to empirically evaluate agents. The empirical analysis of tournaments has a long history, in sports analytics (Lewis, 2004; Bozóki et al., 2016) , ecology and animal behavior (Laird & Schamp, 2006; Silk, 1999) , and biology (Stuart-Fox et al., 2006; Sinervo & Lively, 1996) . While the primary interest in these cases is typically in ranking agents/players, tournament graphs also reveal significant information about the nature of the game being played (Tuyls et al., 2018) . This paper describes mathematical techniques for extracting useful information about the underlying game structure directly from tournament data. While these methods can be applied to the various contexts in which tournaments are already employed in machine learning (e.g., population based training), they open up a range of new research questions regarding the characterization of natural games, synthesis of artificial games (c.f. Omidshafiei et al. ( 2020)), game approximation via simplified dynamics, and the strategic perturbation of games. Fine structural characteristics of a tournament graph can be represented by low dimensional embeddings that map competitive relationships to embedded geometry. We review and expand on methods introduced by Balduzzi et al. (2018b), who proposed a canonical series of maps that provide a complete description of a sample tournament in terms of a sum of simple games, namely, disc games. PTA provides a simplified global understanding of a tournament compatible with a broad set of objectives beyond finding equilibrium solutions.

