CONSTRUCTIVE TT-REPRESENTATION OF THE TENSORS GIVEN AS INDEX INTERACTION FUNCTIONS WITH APPLICATIONS

Abstract

This paper presents a method to build explicit tensor-train (TT) representations. We show that a wide class of tensors can be explicitly represented with sparse TTcores, obtaining, in many cases, optimal TT-ranks. Numerical experiments show that our method outperforms the existing ones in several practical applications, including game theory problems. Theoretical estimations of the number of operations show that in some problems, such as permanent calculation, our methods are close to the known optimal asymptotics, which are obtained by a completely different type of methods.

1. INTRODUCTION

The tensor train is a powerful tool for compressing multidimensional tensors (by tensor we mean a multidimensional array of complex numbers). It allows us to circumvent the curse of dimensionality in a number of cases. In a case of d-dimensional tensor with number of indices equal to n for each dimension, direct storage of tensor involves O(n d ) memory cells, while tensor train bypasses O(ndr 2 ), where r is average rank of TT decomposition (Oseledets, 2011) . In many important applications, the average rank may be small enough so that n d ≫ ndr 2 . Tensor approximation is a hot topic in the area of machine learning. For example, in the paper (Richter et al., 2021) tensor train format is used to solve high-dimensional parabolic PDE with dimension in numerical experiments up to d ∼ 10 2 . Problems of building tensor decomposition and tensor completion are considered in (Lacroix et al., 2020; Fan, 2022; Ma & Solomonik, 2021) . The properties of tensor decompositions as applied to machine learning tasks are discussed in (Ghalamkari & Sugiyama, 2021; Kileel et al., 2021; Khavari & Rabusseau, 2021) . Existing methods allow one to build TT-decompositions by treating the tensor values as a black box. The TT-cross approximation method (Oseledets & Tyrtyshnikov, 2010) adaptively queries the points where the tensor value is evaluated. The iterative alternative schemes such as alternating least squares method (Oseledets & Dolgov, 2012) or alternative linear schemes (Holtz et al., 2012) , build a decomposition consistently updating the decomposition cores. These methods do not take into account the analytic dependence, if any, of the tensor value on its indices. At the same time, even for relatively simple tensors, these methods can build a TT decomposition for a long time and in the vast majority of cases obtain an answer with a given error greater than zero, even if the original tensor has an exact TT decomposition. In this paper, we present a fast method to directly construct cores of the TT decomposition of a tensor for which the analytical 1 dependence of the tensor value on the values of its indices is known. Technically, our method works with functions, each of which depends on tensor index and which are sequentially applied to the values of the previous functions. This functions we call derivative functions hereafter. However, this assignment covers quite a large range of functional dependences of tensor value on its indices if such a set of functions is chosen skillfully. Examples are given in Section 3 and Appendix. Our method works best in cases where the derivative functions together with the tensor itself have a small number of possible values. In the Application section and Appendix there are several examples for indicator tensors taking values only 0 and 1. TT-cores, obtained by our method, are highly sparse, which gives an additional gain in performance. In many cases our method gives the lowest possible TT-rank, so that no further rounding of the TT-cores is required. In some other applications, the ranks of the TT decomposition obtained by our method can be substantially higher than those obtained by approximate methods. However, in a large number of such cases, the sparse structure of the cores allows one to achieve performance comparable to known algorithms. The advantage of representing tensors in the TT format is not only in overcoming the curse of dimensionality, but also in the implemented tensor algebra for them: we can easily add, multiply, round TT-tensors, calculate the convolution (Oseledets, 2011) . In this way we can, for example, construct a set of indicator tensors that represent some constraints in the given problem in advance, and then combine these constraints arbitrarily by multiplying these tensors with a data tensor. As a practical use of such a scheme, we give an example of calculating the permanent of a matrix. The cooperative games examples in Application section use the well-known algorithm for quickly finding the sum of all elements of the TT-tensor. Other examples with practical problems are given in Appendix. They include: simple examples for sum, where we explicitly show sparse TT-cores; cooperative games, where we show how one can build iterative algorithm, based on our method; Knapsack problem (in several formulations), where we use existing algorithms to find the (quasi-) maximal element of the TT-tensor; Partition problem; Eight queens puzzle in several extended formulations (see Fig. 1 for the result for the case of 10 queens); sawtooth sequence; standard Boolean satisfiability problem. Python code with the examples is available in the public domainfoot_1 . In the vast majority of this examples we use products of tensors, convolution to find multidimensional sums and maximum element search to solve optimization problems. These operations are sufficient to solve a large class of problems from different areas of mathematics. Our method has a direct extension to more complex cases of tensor networks, for one of the cooperative games below and in several examples in Appendix. Such a construction is called TT-Tucker (Dolgov & Khoromskij, 2013; Oseledets, 2011) . Our main contribution and advantages of our approach • the exact and fast representation of the tensor in TT-format, which can then, if necessary, be rounded to smaller ranks with a given accuracy. In many of the given examples, this representation is optimal in the sense that the ranks of the TT decomposition cannot be reduced without loss of accuracy; • highly sparse structure of TT-decomposition cores which leads to a noticeable reduction in calculations; • a unified approach and a simple algorithmic interface to inherently different tasks and areas including those problems for which it is not immediately obvious the representation of the function specifying the tensor value in the form of consecutive functions (1)-(3); • the ability to construct an approximate TT-decomposition with a controlled error or/and with the specified maximum ranks of the TT decomposition;



By analytic dependence we mean the known symbolic formula for the tensor value, not the definition of the term within complex analysis. https://github.com/G-Ryzhakov/Constructive-TT



Figure 1: The solution of 10-queen problem, obtained by our algorithm.

