SYNC: SAFETY-AWARE NEURAL CONTROL FOR STA-BILIZING STOCHASTIC DELAY-DIFFERENTIAL EQUA-TIONS

Abstract

Stabilization of the systems described by stochastic delay-differential equations (SDDEs) under preset conditions is a challenging task in the control community. Here, to achieve this task, we leverage neural networks to learn control policies using the information of the controlled systems in some prescribed regions. Specifically, two learned control policies, i.e., the neural deterministic controller (NDC) and the neural stochastic controller (NSC), work effectively in the learning procedures that rely on, respectively, the well-known LaSalle-type theorem and the newly-established theorem for guaranteeing the stochastic stability in SDDEs. We theoretically investigate the performance of the proposed controllers in terms of convergence time and energy cost. More practically and significantly, we improve our learned control policies through considering the situation where the controlled trajectories only evolve in some specific safety set. The practical validity of such control policies restricted in safety set is attributed to the theory that we further develop for safety and stability guarantees in SDDEs using the stochastic control barrier function and the spatial discretization. We call this control as SYNC (SafetY-aware Neural Control). The efficacy of all the articulated control policies, including the SYNC, is demonstrated systematically by using representative control problems.

1. INTRODUCTION

Stochastic delay-differential equations (SDDEs) (Mao, 1996; Lin & He, 2005; Sun & Cao, 2007; Guo et al., 2016) have been widely applied to characterize the complex dynamical behavior emergent in real-world systems with dependence on the current state, the past state, and the noise. Efficiently controlling these systems is a long-standing and crucial problem, with the consequent emphasis being placed on the design of control policies and analysis of stability in SDDEs. Traditional control methods in stochastic settings have been fully developed in the convex optimization frameworks using the control Lyapunov stability theory, e.g. the quadratic programming (QP) (Fan et al., 2020; Sarkar et al., 2020) . These methods cannot provide the analytical form of feedback controllers and own a high computational cost, requiring solving QP problems at each iteration step. To overcome these difficulties, utilizing neural networks (NNs) to automatically design controllers becomes one of the mainstream approaches in recent years (Zhang et al., 2022; Chang et al., 2019) . However, existing machine-learning-based methods either focus on controlling systems without time-delay or aim at learning the control Lyapunov function instead of the control policy (Khansari-Zadeh & Billard, 2014) . All these, therefore, motivate us to design neural controllers for general nonlinear SDDEs. The safety verification of controlled systems plays an important role in many branches of cybernetics and industry. For example, with the safety verification, one can reduce a significant economic burden and loss of life (Ames et al., 2016; Wang et al., 2016) . In particular, the dominant framework for safety control in stochastic settings is the use of stochastic control barrier function (SCBF) (Clark, 2019; 2021; Santoyo et al., 2021) . The core idea of designing a candidate SCBF is that its value tends to explode as the system's state leaves the safe region, implying a safety guarantee as long as one could design a controller such that the SCBF is always finite within the controlled time duration. Unfortunately, the existing theories of SCBF either require a lot of inequality constraints or are limited in handling systems without any time delay. In this paper, we utilize neural networks (NNs) to learn control policies for SDDEs based on the corresponding stability theories. Additionally, we develop a simplified SCBF theory for SDDEs and then use it to construct the neural controller with a safety guarantee, named SYNC. All these control policies are intuitively depicted in Figure 1 . The major contributions of this paper include: • designing a novel and practical framework of neural deterministic control based on the existing LaSalle-Type stability theory, • proposing a simplified stability theorem and designing the second novel neural stochastic control framework that can benefit from noise according to this theorem, • establishing an SCBF theory for SDDEs as well as a theory of safety guarantee and stability guarantee using neural network settings, • providing theoretical estimation for the proposed neural controller in terms of convergence time and energy cost based on the developed theory of safety and stability guarantees, and • demonstrating the efficacy of the proposed neural control methods through numerical comparisons with the typical existing control methods on several representative physical systems.

2. PRELIMINARIES

To begin with, we consider the SDDE in a general form of dx(t) = F (x(t), x(t -τ ), t)dt + G(x(t), x(t -τ ), t)dB t , t ≥ 0, τ > 0, x(t) ∈ R d , where x(t) = ξ(t) ∈ C F0 ([-τ, 0]; R d ) is the initial function, the drift term F : R d × R d × R + → R d and the diffusion term G : R d × R d × R + → R d×r are Borel-measurable functions, and B t is a standard r-dimensional (r-D) Brownian motion defined on probability space (Ω, F, {F t } t≥0 , P) with a filtration {F t } t≥0 satisfying the regular conditions. Without loss of generality, we assume that F (0, 0, t) = 0 and G(0, 0, t) = 0. This assumption guarantees that the zero solution x(t) ≡ 0 with t ≥ 0 is an equilibrium of Eq. (1). Additionally, the following notations and assumptions are used throughout the paper.



Figure 1: Overall work flow. Sketches of SYNC. Both the NDC and NSC can stabilize the SDDEs to the target unstable equilibrium x * . The safety-aware controlled state trajectories are restricted in the safe region.

