CORESET FOR RATIONAL FUNCTIONS

Abstract

We consider the problem of fitting a rational function g : R → R to a time-series f : {1, • • • , n} → R. This is by minimizing the sum of distances (loss function) ℓ(g) := n i=1 |f (i) -g(i)|, possibly with additional constraints and regularization terms that may depend on g. Our main motivation is to approximate such a time-series by a recursive sequence model G n = k i=1 θ i G n-i , e.g. a Fibonacci sequence, where θ ∈ R k are the model parameters, and k ≥ 1 is constant. For ε ∈ (0, 1), an ε-coreset for this problem is a data structure that approximates ℓ(g) up to 1 ± ε multiplicative factor, for every rational function g of constant degree. We suggest a coreset construction that runs in O(n 1+o(1) ) time and returns such a coreset that uses O(n o(1) /ε 2 ) memory words. We provide open source code as well as extensive experimental results, on both real and synthetic datasets, which compare our method to existing solvers from Numpy and Scipy.

1. BACKGROUND

The original motivation for this work was to suggest provable and efficient approximation algorithms for fitting input data by a stochastic model or its variants, such as Hidden-Markov Models  (constant) k ≥ 1 past values F (t -1), • • • , F (t -k) in the sequence, and the model's parameter θ.

1.1. AUTO-REGRESSION

Unfortunately, most existing results seem to be based on heuristics with little provable approximation guarantees. We thus investigate a simplified but fundamental version, called auto-regression, which has a provable but not so efficient solution using polynomial system solvers, after applying the technique of generating functions. This technique is strongly related to the Fourier, Laplace and z-Transform, as explained below. We define an auto-regression inspired by Ghosh et al. ( 2013  Definition 1. A time-series F : [n] → R is an auto-regression (AR for short) of degree k, if there exist a vector of coefficients θ = (θ 1 , • • • , θ k ) ∈ R k such that F (t) = θ 1 F (t-1)+• • •+θ k F (t-k). The polynomial P (x) = x k -θ k x k-1 -• • • -θ 1 is called the characteristic polynomial of F . Substituting k = 2, θ = (1, 1) and F (1) = F (2) = 1 in Definition 1 yields the Fibonacci sequence, i.e. F (t) = F (t -1) + F (t -2), where F (1) = F (2) = 1. From Auto-regression to Rational functions. In the corresponding "data science version" of Fibonacci's sequence, the input is the time-series G(1), G(2), • • • , which is based on F with additional noise. A straight forward method to recover the original model is by directly minimizing of the squared error between the given noisy time-series and the fitted values, as done e.g. in Eshragh et al. ( 2019)) using simple linear regression. However, this has a major drawback; AR time-series usually grows exponentially, like geometric sequences, and thus the loss will be dominated by the last few terms in the time-series. Moreover, small changes in the time domain have exponential



(HMM) Basu et al. (2001); McCallum et al. (2000); Murphy (2002); Park et al. (2012); Sassi et al. (2020); Yu et al. (2010) and reference therein, Baysian Networks Acar et al. (2007); Murphy (2002); Nikolova et al. (2010); Rudzicz (2010) and reference therein, auto-regression Ghosh et al. (2013), and Decision Markov Process Shanahan & den Poel (2010). Informally, and in the context of this work, a model defines a time series (sequence, discrete signal, time series) F : [n] → R where [n] := {1, • • • , n}, and the value F (t) at time (integer) t ≥ 1 is a function of only the previous

); Eshragh et al. (2019); Yuan (2009), as follows:

