DIFFERENTIABLE APPROXIMATIONS FOR MULTI-RESOURCE SPATIAL COVERAGE PROBLEMS

Abstract

Resource allocation for coverage of physical spaces is a challenging problem in robotic surveillance, mobile sensor networks and security domains. Recent gradient-based optimization approaches to this problem estimate utilities of actions by using neural networks to learn a differentiable approximation to spatial coverage objectives. In this work, we empirically show that spatial coverage objectives with multiple-resources are combinatorially hard to approximate for neural networks and lead to sub-optimal policies. As our major contribution, we propose a tractable framework to approximate a general class of spatial coverage objectives and their gradients using a combination of Newton-Leibniz theorem, spatial discretization and implicit boundary differentiation. We empirically demonstrate the efficacy of our proposed framework on single and multi-agent spatial coverage problems.

1. INTRODUCTION

Allocation of multiple resources for efficient spatial coverage is an important component of many practical single-agent and multi-agent systems, for e.g., robotic surveillance, mobile sensor networks and security game modeling. Surveillance tasks generally involve a single agent assigning resources e.g. drones or sensors, each of which can monitor physical areas, to various points in a target domain such that a loss function associated with coverage of the domain is minimized (Renzaglia et al., 2012) . Alternatively, security domains follow a leader-follower game setup between two agents, where a defender defends a set of targets (or a continuous target density in a geographical area) with limited resources to be placed, while an attacker plans an attack after observing the defender's placement strategy using its own resources (Tambe, 2011) . Traditional methods used to solve single-agent multi-resource surveillance problems often rely on potential fields (Howard et al., 2002) , discretization based approaches (Kong et al., 2006 ), voronoi tessellations (Dirafzoon et al., 2011) and particle swarm optimization (Nazif et al., 2010; Saska et al., 2014) . Similarly, many exact and approximate approaches have been proposed to maximize the defender's expected utility in two-agent multi-resource security domains against a best responding attacker (Kiekintveld et al., 2009; Amin et al., 2016; Yang et al., 2014; Haskell et al., 2014; Johnson et al., 2012; Huang et al., 2020) . Notably, most existing traditional approaches focus on exploiting some specific spatio-temporal or symmetry structure of the domain being examined. Related Work: Since spatial coverage problems feature continuous action spaces, a common technique used across most previous works is to discretize the area to be covered into grid cells and restrict the agents' actions to discrete sets (Kong et al., 2006; Yang et al., 2014; Haskell et al., 2014; Gan et al., 2017) to find the equilibrium mixed strategies or optimal pure strategies using integer linear programming. However, discretization quickly becomes intractable when the number of each agent's resources grows large. While some games can be characterized by succinct agent strategies and can be solved efficiently via mathematical programming after discretizing the agents' actions spaces (Behnezhad et al., 2018) , this is not true for most multi-resource games. Recent works in spatial coverage domains have focused on incorporating advances from deep learning to solve the coverage problems with more general algorithms. For instance, Pham et al. ( 2018) focus on the multi-UAV coverage of a field of interest using a model-free multi-agent RL method while StackGrad (Amin et al., 2016 ), OptGradFP (Kamra et al., 2018) , PSRO (Lanctot et al., 2017) are model-free fictitious play based algorithms which can be used to solve games in continuous action spaces. However model-free approaches are sample inefficient and require many interactions with the domain (or with a simulator) to infer expected utilities of agents' actions. Secondly, they often rely

