SPACETIME REPRESENTATION LEARNING

Abstract

Much of the data we encounter in the real world can be represented as directed graphs. In this work, we introduce a general family of representations for directed graphs through connected time-oriented Lorentz manifolds, called "spacetimes" in general relativity. Spacetimes intrinsically contain a causal structure that indicates whether or not there exists a causal or even chronological order between points of the manifold, called events. This chronological order allows us to naturally represent directed edges via imposing the correct ordering when the nodes are embedded as events in the spacetime. Previous work in machine learning only considers embeddings lying on the simplest Lorentz manifold or does not exploit the connection between Lorentzian pre-length spaces and directed graphs. We introduce a well-defined approach to map data onto a general family of spacetimes. We empirically evaluate our framework in the tasks of hierarchy extraction of undirected graphs, directed link prediction and representation of directed graphs.

1. INTRODUCTION

Most of the machine learning literature has focused on learning representations that lie on a Riemannian manifold such as the Euclidean space, the d-sphere (e.g., 2 -normalized representations) (Wang et al., 2017; Tapaswi et al., 2019) , hyperbolic geometry to represent graphs without cycles (Nickel & Kiela, 2017) , or a statistical manifold in information geometry (Amari, 1998) . Concepts of Euclidean geometry, such as distances, are naturally generalized to Riemannian geometry which remains easy to interpret. In contrast, recent approaches have considered learning representations that lie on a pseudo-Riemannian manifold to extract hierarchies in graphs with cycles (Law & Stam, 2020; Law, 2021) or represent directed graphs (Clough & Evans, 2017; Sim et al., 2021) . Pseudo-Riemannian manifolds are generalizations of Riemannian manifolds where the constraint of positive definiteness of the nondegenerate metric tensor is relaxed. The machine learning literature on pseudo-Riemannian manifolds can be divided into two categories. The first category focuses on how to optimize a given function whose domain is a pseudo-Riemannian manifold and does not take into account whether the manifold is time-oriented or not (Law & Stam, 2020; Law, 2021) . The second category exploits the interpretation of a specific family of pseudo-Riemannian manifolds called "spacetimes" in general relativity (Clough & Evans, 2017; Sim et al., 2021) . More specifically, spacetimes are connected time-oriented Lorentz manifolds. They intrinsically contain a causal structure that indicates whether or not there exists a causal order between points of the manifold, called events. This causal structure has been utilized to represent directed graphs where each node is an event and the existence of an arc (i.e., directed edge) between two nodes depends on the causal character of the curves joining them (Bombelli et al., 1987) . In particular, Clough & Evans (2017) consider learning representations via the Minkowski spacetime which is the simplest such manifold. On the other hand, Sim et al. ( 2021) use three types of spacetimes and propose an ad hoc method based on the sign of some time coordinate difference function to determine the orientation of edges. The sign of such a function is not always meaningful as it for instance alternates periodically when the manifold is non-chronological and does not generalize to all spacetimes. Moreover, the distance function that they optimize is constant when two points cannot be joined by a geodesic. Contributions. We propose a framework inspired by Lorentzian causality theory (Kronheimer & Penrose, 1967; Minguzzi, 2019) , and in particular Lorentzian pre-length spaces (Kunzinger & Sämann, 2018) , to learn directed graph representations lying on a large family of spacetimes. To this end, we present tools to account for time-orientation and exploit distances specific to Lorentz geometry. In particular, we propose to restrict the existence of edges to pairs of nodes whose representations lie in an open globally hyperbolic convex normal neighborhood. Such a neighborhood can be defined for any spacetime (see Theorem 2.7 of Minguzzi ( 2019)) and admits simple distance and time separation functions whose sign determines the direction of edges. We experimentally show that spacetimes can extract hierarchies in social networks better than standard approaches. Our framework also outperforms existing methods in link prediction on graphs with directed cycles.

2. SPACETIME DIFFERENTIAL GEOMETRY

We introduce some differential geometry background about spacetimes. Pseudo-Riemannian Manifold. A d-dimensional pseudo-Riemannian manifold (M, g) is a smooth manifold such that every point x ∈ M has a d-dimensional tangent space T x M whose metric tensor g x : T x M × T x M → R is a nondegenerate symmetric bilinear form (called a scalar product). Nondegeneracy means that ∀v ∈ T x M, g x (u, v) = 0 =⇒ u = 0. When the context is clear and to simplify the notation, we write •, • := g x (•, •) to define the metric tensor at x. We also write M instead of (M, g). We write points x ∈ M of the manifold in bold serif font, and tangent vectors u ∈ T x M in bold sans-serif font when we want to distinguish them from points. Lorentz manifold. Every tangent space T x M of a d-dimensional pseudo-Riemannian manifold M admits an orthonormal basis {e 1 , . . . , e d } that satisfies ∀i, e i , e i = ±1 and ∀ i = j, e i , e j = 0. The index ν ≤ d of M is the number of vectors e i that satisfy e i , e i = -1. If ν = 0, M is Riemannian and its metric tensor is positive definite (i.e., ∀x ∈ M, ∀u ∈ T x M, u, u ≥ 0 and u, u = 0 ⇐⇒ u = 0). If ν = 1, M is a Lorentz manifold and T x M is a Lorentz vector space. Future timecone. A nonzero tangent vector u is called timelike (or chronological), null, spacelike or non-spacelike (or causal) if u, u is negative, zero, positive or nonpositive, respectively. The type into which u falls is called its causal character. If u = 0, then u is spacelike. Every Lorentz tangent space contains two timecones. Some timelike tangent vector t ∈ T x M can arbitrarily be used to define the future timecone as the following set: C + x (t) := {v ∈ T x M : v, v < 0, t, v < 0} whereas -t defines the past timecone C - x (t) := C + x (-t). Two timelike tangent vectors u and v are in the same timecone iff u, v < 0. They belong to different timecones if v = -u. Time-orientability and time-orientation. A continuous vector field X is a function that assigns to each point x ∈ M a tangent vector of M at x denoted by X(x) ∈ T x M. X and -X are timelike if ∀x ∈ M, X(x), X(x) < 0. A Lorentz manifold is time-orientable iff there exists a timelike vector field. If M is assigned such a timelike vector field X, it is time-oriented by X. In this case, non-spacelike tangent vectors u at each point x can be divided into two separate classes: future-directed if X(x), u < 0, and past-directed if X(x), u > 0. A curve γ x→u : I → M where I ⊆ R is defined such that its initial point is γ x→u (0) = x and its initial velocity is γ x→u (0) = u ∈ T x M. We denote it by γ when its initial conditions are clear from



Figure 1: Geodesics of the de Sitter space S d 1 (r) (left) and of the anti-de Sitter space H d 1 (r) (right).

Spacetimes have been widely studied, and we refer the reader to Hawking & Ellis (1973); Uhlenbeck (1975); O'Neill (1995); Beem et al. (1996); Wolf (2011); Gourgoulhon (2016), Chapter 5-8 & 14 of O'Neill (1983).

