ARMCMC: ONLINE MODEL PARAMETERS FULL PROBABILITY ESTIMATION IN BAYESIAN PARADIGM

Abstract

Although the Bayesian paradigm provides a rigorous framework to estimate the full probability distribution over unknown parameters, its online implementation can be challenging due to heavy computational costs. This paper proposes Adaptive Recursive Markov Chain Monte Carlo (ARMCMC) which estimates full probability density of model parameters while alleviating shortcomings of conventional online approaches. These shortcomings include: being solely able to account for Gaussian noise, being applicable to systems with linear in the parameters (LIP) constraint, or having requirements on persistence excitation (PE). In ARMCMC, we propose a variable jump distribution, which depends on a temporal forgetting factor. This allows one to adjust the trade-off between exploitation and exploration, depending on whether there is an abrupt change to the parameter being estimated. We prove that ARMCMC requires fewer samples to achieve the same precision and reliability compared to conventional MCMC approaches. We demonstrate our approach on two challenging benchmark: the estimation of parameters in a soft bending actuator and the Hunt-Crossley dynamic model. Our method shows at-least 70% improvement in parameter point estimation accuracy and approximately 55% reduction in tracking error of the value of interest compared to recursive least squares and conventional MCMC.

1. INTRODUCTION

Bayesian methods are powerful tools to not only obtain a numerical estimate of a parameter but also to give a measure of confidence (Kuśmierczyk et al., 2019; Bishop, 2006; Joho et al., 2013) . In particular, Bayesian inferences calculate the probability distribution of parameters rather than a point estimate, which is prevalent in frequentist paradigms (Tobar, 2018) . One of the main advantages of probabilistic frameworks is that they enable decision making under uncertainty (Noormohammadi-Asl & Taghirad, 2019) . In addition, knowledge fusion is significantly facilitated in probabilistic frameworks; different sources of data or observations can be combined according to their level of certainty in a principled manner (Agand & Shoorehdeli, 2019) . Nonetheless, Bayesian inferences require high computational effort for obtaining the whole probability distribution and prior general knowledge on noise distribution before estimation. One of the most effective methods for Bayesian inferences is the Markov Chain Monte Carlo (MCMC) methods. In the field of system identification, MCMC variants such as the one recently proposed by Green (2015) are mostly focused on offline system identification. This is partly due to computational challenges which prevent real-time use (Kuindersma et al., 2012) . The standard MCMC algorithm is not suitable for model variation since different candidates do not share the same parameter set. Green (1995) first introduced reversible jump Markov chain Monte Carlo (RJMCMC) as a method to address the model selection problem. In this method, an extra pseudo random variable is defined to handle dimension mismatch. There are further extensions of MCMC in the literature, however, an online implication of it has yet to be reported. Motion filtering and force prediction of robotic manipulators are important fields of study with interesting challenges suitable for Bayesian inferences to address (Saar et al., 2018) . Here, measurements are inherently noisy, which is not desirable for control purposes. Likewise, inaccuracy, inaccessibility, and costs are typical challenges that make force measurement not ideal for practical use (Agand et al., 2016) . Different environmental identification methods have been proposed in the literature

