EMERGENCE OF MAPS IN THE MEMORIES OF BLIND NAVIGATION AGENTS

Abstract

Animal navigation research posits that organisms build and maintain internal spatial representations, or maps, of their environment. We ask if machines -specifically, artificial intelligence (AI) navigation agents -also build implicit (or 'mental') maps. A positive answer to this question would (a) explain the surprising phenomenon in recent literature of ostensibly map-free neural-networks achieving strong performance, and (b) strengthen the evidence of mapping as a fundamental mechanism for navigation by intelligent embodied agents, whether they be biological or artificial. Unlike animal navigation, we can judiciously design the agent's perceptual system and control the learning paradigm to nullify alternative navigation mechanisms. Specifically, we train 'blind' agents -with sensing limited to only egomotion and no other sensing of any kind -to perform PointGoal navigation ('go to ∆x, ∆y') via reinforcement learning. Our agents are composed of navigation-agnostic components (fully-connected and recurrent neural networks), and our experimental setup provides no inductive bias towards mapping. Despite these harsh conditions, we find that blind agents are (1) surprisingly effective navigators in new environments (∼95% success); (2) they utilize memory over long horizons (remembering ∼1,000 steps of past experience in an episode); (3) this memory enables them to exhibit intelligent behavior (following walls, detecting collisions, taking shortcuts); (4) there is emergence of maps and collision detection neurons in the representations of the environment built by a blind agent as it navigates; and (5) the emergent maps are selective and task dependent (e.g. the agent 'forgets' exploratory detours). Overall, this paper presents no new techniques for the AI audience, but a surprising finding, an insight, and an explanation.

1. INTRODUCTION

Decades of research into intelligent animal navigation posits that organisms build and maintain internal spatial representations (or maps)foot_0 of their environment, that enables the organism to determine and follow task-appropriate paths (Tolman, 1948; O'keefe & Nadel, 1978; Epstein et al., 2017) . Hamsters, wolves, chimpanzees, and bats leverage prior exploration to determine and follow shortcuts they may never have taken before (Chapuis & Scardigli, 1993; Peters, 1976; Menzel, 1973; Toledo et al., 2020; Harten et al., 2020) . Even blind mole rats and animals rendered situationallyblind in dark environments demonstrate shortcut behaviors (Avni et al., 2008; Kimchi et al., 2004; Maaswinkel & Whishaw, 1999) . Ants forage for food along meandering paths but take near-optimal return trips (Müller & Wehner, 1988) , though there is some controversy about whether insects like ants and bees are capable of forming maps (Cruse & Wehner, 2011; Cheung et al., 2014) . Analogously, mapping and localization techniques have long played a central role in enabling nonbiological navigation agents (or robots) to exhibit intelligent behavior (Thrun et al., 2005; Institute, 



Throughout this work, we use 'maps' to refer to a spatial representation of the environment that enables intelligent navigation behavior like taking shortcuts. We provide a detailed discussion and contrast w.r.t. a 'cognitive map' as defined byO'keefe & Nadel (1978) in Apx. B.1.

