LAPLACIAN EIGENSPACES, HOROCYCLES AND NEU-RON MODELS ON HYPERBOLIC SPACES

Abstract

We use hyperbolic Poisson kernel to construct the horocycle neuron model on hyperbolic spaces, which is a spectral generalization of the classical neuron model. We prove a universal approximation theorem for horocycle neurons. As a corollary, we obtain a state-of-the-art result on the expressivity of f 1 a,p , which is used in the hyperbolic multiple linear regression. Our experiments get state-of-the-art results on the Poincare-embedding subtree classification task and the classification accuracy of the two-dimensional visualization of images.

1. INTRODUCTION

Conventional deep network techniques attempt to use architecture based on compositions of simple functions to learn representations of Euclidean data (LeCun et al., 2015) . They have achieved remarkable successes in a wide range of applications (Hinton et al., 2012; He et al., 2016) . Geometric deep learning, a niche field that has caught the attention of many authors, attempts to generalize conventional learning techniques to non-Euclidean spaces (Bronstein et al., 2017; Monti et al., 2017) . There has been growing interest in using hyperbolic spaces in machine learning tasks because they are well-suited for tree-like data representation (Ontrup & Ritter, 2005; Alanis-Lobato et al., 2016; Nickel & Kiela, 2017; Chamberlain et al., 2018; Nickel & Kiela, 2018; Sala et al., 2018; Ganea et al., 2018b; Tifrea et al., 2019; Chami et al., 2019; Liu et al., 2019; Balazevic et al., 2019; Yu & Sa, 2019; Gulcehre et al., 2019; Law et al., 2019) . Many authors have introduced hyperbolic analogs of classical learning tools (Ganea et al., 2018a; Cho et al., 2019; Nagano et al., 2019; Grattarola et al., 2019; Mathieu et al., 2019; Ovinnikov, 2020; Khrulkov et al., 2020; Shimizu et al., 2020) . Spectral methods are successful in machine learning, from nonlinear dimensionality reduction (Belkin & Partha, 2002) to clustering (Shi & Malik, 2000; Ng et al., 2002) to hashing (Weiss et al., 2009) to graph CNNs (Bruna et al., 2014) to spherical CNNs (Cohen et al., 2018) and to inference networks (Pfau et al., 2019) . Spectral methods have been applied to learning tasks on spheres (Cohen et al., 2018) and graphs (Bruna et al., 2014) , but not yet on hyperbolic spaces. This paper studies a spectral generalization of the FC (affine) layer on hyperbolic spaces. Before presenting the spectral generalization of the affine layer, we introduce some notations. Let (•, •) E be the inner product, | • | the Euclidean norm, and ρ an activation function. The Poincaré ball model of the hyperbolic space H n (n≥2) is a manifold {x∈R n : |x|<1} equipped with a Riemannian metric dsfoot_0  H n = n i=1 4(1-|x| 2 ) -2 dx 2 i . The boundary of H n under its canonical embedding in R n is the unit sphere S n-1 . The classical neuron y=ρ((x, w) E +b) is of input x∈R n , output y∈R, with trainable parameters w∈R n , b∈R. An affine layer R n → R m is a concatenation of m neurons. An alternative representation of the neuron x →ρ((x, w) E +b) is given by 1 x∈R n → ρ(λ(x, ω) E +b), ω∈S n-1 , λ, b∈R. (1) This neuron is constant over any hyperplane that is perpendicular to a fixed direction ω. In H n , a horocycle is a n-1 dimensional sphere (one point deleted) that is tangential to S n-1 . Horocycles are hyperbolic counterparts of hyperplanes (Bonola, 2012) . Horocyclic waves x, ω H := 1



log 1-|x| 2 |x-ω| 2 are constant over any horocycle that is tangential to S n-1 at ω. Therefore,x∈H n → ρ(λ x, ω H +b), ω∈S n-1 , λ, b∈R(2)1 if w = (0, . . . , 0), one can take ω = w/|w|, λ = |w|; else, one can take λ = 0 and any ω ∈ S n-1 .1

