ON THE UNIVERSAL APPROXIMABILITY AND COMPLEXITY BOUNDS OF DEEP LEARNING IN HYBRID QUANTUM-CLASSICAL COMPUTING

Abstract

With the continuously increasing number of quantum bits in quantum computers, there are growing interests in exploring applications that can harvest the power of them. Recently, several attempts were made to implement neural networks, known to be computationally intensive, in hybrid quantum-classical scheme computing. While encouraging results are shown, two fundamental questions need to be answered: (1) whether neural networks in hybrid quantum-classical computing can leverage quantum power and meanwhile approximate any function within a given error bound, i.e., universal approximability; (2) how do these neural networks compare with ones on a classical computer in terms of representation power? This work sheds light on these two questions from a theoretical perspective.

1. INTRODUCTION

Quantum computing has been rapidly evolving (e.g., IBM (2020) recently announced to debut quantum computer with 1,121 quantum bits (qubits) in 2023), but the development of quantum applications is far behind; in particular, it is still unclear what and how applications can take quantum advantages. Deep learning, one of the most prevalent applications, is well known to be computationintensive and therefore their backbone task, neural networks, is regarded as an important task to potentially take quantum advantages. Recent works (Francesco et al., 2019; Tacchino et al., 2020; Jiang et al., 2020) have demonstrated that the shallow neural networks with limited functions can be directly implemented on quantum computers without interfering with classical computers, but as pointed by Broughton et al. (2020) , the near-term Noisy Intermediate-Scale Quantum (NISQ) can hardly disentangle and generalize data in general applications, using quantum computers alone. This year, Google (2020) has put forward a library for hybrid quantum-classical neural networks, which attracts attention from both industry and academia to accelerate quantum deep learning. In a hybrid quantum-classical computing scheme, quantum computers act as hardware accelerators, working together with classical computers, to speedup the neural network computation. The incorporation of classical computers is promising to conduct operations that are hard or costly to be implemented on quantum computers; however, it brings high data communication costs at the interface between quantum and classical computers. Therefore, instead of contiguous communication during execution, a better practice is a "prologue-acceleration-epilogue" scheme: the classical computer prepares data and post-processes data at prologue and epilogue, while only the quantum computer is active during the acceleration process for the main computations. Without explicit explanation, "hybrid model" refers to the prologue-acceleration-epilogue scheme in the rest of the paper. In a classical computing scheme, the universal approximability, i.e., the ability to approximate a wide class of functions with arbitrary small error, and the complexity bounds of different types of neural networks have been well studied (Cybenko, 1989; Hornik et al., 1989; Mhaskar & Micchelli, 1992; Sonoda & Murata, 2017; Yarotsky, 2017; Ding et al., 2019; Wang et al., 2019; Fan et al., 2020) . However, due to the differences in computing paradigms, not all types of neural networks can be directly implemented on quantum computers. As such, it is still unclear whether those can work with hybrid quantum-classical computing and still attain universal approximability. In addition, as quantum computing limits the types of computations to be handled, it is also unknown whether the hybrid quantum-classical neural networks can take quantum advantage over the classical networks under the same accuracy. This work explores these questions from a theoretical perspective. In this work, we first illustrate neural networks that are feasible in hybrid quantum-classical computing scheme. Then we use the method of bound-by-construction to demonstrate their universal approximability for a wide class of functions and the computation bounds, including network depth, qubit cost and gate cost, under a given error bound. In addition, compared with some of the lower complexity bounds for neural networks on classical computers, our established upper bounds are of lower asymptotic complexity, showing the potential of quantum advantage.

2.1. NEURAL NETWORKS IN QUANTUM COMPUTING

Although the research on neural networks in quantum computing can trace back to the 1990s (Kak, 1995; Purushothaman & Karayiannis, 1997; Ezhov & Ventura, 2000) , but only recently, along with the revolution of quantum computers, the implementation of neural networks on actual quantum computer emerges (Francesco et al., 2019; Jiang et al., 2020; Bisarya et al., 2020) . There are mainly three different directions to exploit the power of quantum computers: (1) applying the Quantum Random Access Memory (QRAM) (Blencowe, 2010); (2) employing pure quantum computers; (3) bridging different platforms for a hybrid quantum-classical computing (McClean et al., 2016) . 2019) is a typical work to implement neural networks with QRAM. Using QRAM provides the highest flexibility, such as implementing non-linear functions using lookup tables. But QRAM itself has limitations: instead of using the widely applied superconducting qubits (Arute et al., 2019; IBM, 2016) , QRAM needs the support of spin qubit (Veldhorst et al., 2015) to provide relatively long lifetime. To make the system practical, there is still a long way to go. Alternatively, there are works which encode data to either qubits (Francesco et al., 2019) or qubit states (Jiang et al., 2020) and use superconducting-based quantum computers to run neural networks. These methods also have limitations: Due to the short decoherence times in current quantum computers, the condition statement is not supported, making it hard to implement some non-linear functions such as the most commonly used Rectified Linear Unit (ReLU). But the advantages are also obvious: (1) the designs can be directly evaluated on actual quantum computers; (2) little communication is needed between quantum and classical computers, which may otherwise be expensive.

Kerenidis et al. (

Hybrid quantum-classical computing tries to address the limitations of QRAM and pure quantum computer based approaches. Broughton et al. ( 2020) establishes a computing paradigm where different neurons can be implemented on either quantum or classical computers. This brings the flexibility in implementing functions (e.g., ReLU), while at the same time, it calls for fast interfaces for massive data transfer between quantum and classical computers. In this work, we focus on the hybrid quantum-classical computing scheme and follow the "prologueacceleration-epilogue" computing scheme. It offers the flexibility of implementation and at the same time requires minimal quantum-classical data transfer, as demonstrated in Figure 2 .

2.2. UNIVERSAL APPROXIMATION AND COMPLEXITY BOUND

Universal approximability of neural network indicates that for any given continuous function or a wide class of functions satisfying some constraints, and arbitrarily small error bound > 0, there exists a neural network model which can approximate the function with no more than error. On classical computing, different types of neural networks have been proved to have universal approximability: multi-layer feedforward neural networks (Cybenko, 1989; Hornik et al., 1989) ; ReLU neural networks (Mhaskar & Micchelli, 1992; Sonoda & Murata, 2017; Yarotsky, 2017) ; quantized neural networks (Ding et al., 2019; Wang et al., 2019) ; and quadratic neural networks (Fan et al., 2020) . In addition, many of these works also establishes complexity bounds in terms of the number of weights, number of layers, or number of neurons needed for approximation with error bound . When it comes to quantum computing, in recent years, Delgado (2018) demonstrated quantum circuit with an additional trainable diagonal matrix can approximate the given functions, and Schuld et al. (2020) shown that the Fourier-type sum-based quantum models can be universal function approximators if the quantum circuit is measured enough many times. Most recently, we are wit-

