NEURALEQ: NEURAL-NETWORK-BASED EQUALIZER FOR HIGH-SPEED WIRELINE COMMUNICATION

Abstract

Massive data processing is required by various applications such as ML, video streaming, cloud services, etc. In such systems, I/O bandwidth must be scaled up to prevent any performance degradation due to the limited data transfer rates. To meet this demand, wireline communication recently started adopting PAM4 signaling and DSP-based equalizers. However, multi-level signaling and conventional equalizing techniques degrade the bit-error-rate (BER) performance significantly. To mitigate this problem, this paper proposes a novel neural network architecture that mimics the forward-backward algorithm estimating the posterior probabilities in Hidden Markov Models. The proposed neural network overcomes the existing equalizer performance, such as feed-forward equalizers or decisionfeedback equalizers, while reducing the complexity of the forward-backward algorithm.

1. INTRODUCTION

Recent advances in ML/AI technologies have benefited our daily life by enabling services like searching, recommendation, and translation. The rapid growth of ML applications, increasing the amount of data and computation, has driven enormous demands on high-performance computing systems. Accordingly, the I/O bandwidth and computing power of such systems must be scaled up to support large data transactions between computing cores without experiencing performance degradation due to their limited data transfer rates. To meet such high bandwidth demand for highperformance computing systems, the PCIe standard is being developed up to 64 Gbps for the next generation, and data-rate of NVlink specializing in GPU communication reaches 50 Gbps (Temuc ¸in et al., 2021) . Data-rate of the ethernet protocol is also being developed up to 112 Gbps to handle the high network load. Although demands for higher data-rate for wireline communication keep increasing, limited bandwidth of wireline channels poses problems in data transmission. Different from wireless communication, wireline channels are time-invariant. In addition, pursuing low latency and good I/O energy efficiency, wireline communication has relied on simple modulation like PAM2 and simple equalizers, followed by symbol-by-symbol detection. However, data-rate is increasing so rapidly even a short channel causes severe signal distortion, making it difficult to restore data at the receiver. To increase the data-rate further, recently, wireline communication started adopting multi-level signaling like PAM4 and DSP-based equalizers along with analog-to-digital converters (ADCs) on the receive side. Although PAM4 signaling enables data-rate to reach over 100 Gbps, as (Cordova et al., 2021; de Abreu Farias Neto et al., 2020; Krupnik et al., 2020) , due to the peak power constraint, PAM4 degrades SNR and makes the signal more vulnerable to inter-symbol interference (ISI) caused by limited channel bandwidth, resulting in bit-error-rate (BER) degradation. Under these circumstances, equalizers to compensate for the channel loss play a critical role in high-speed wireline communication. Equalizers can be primarily divided into a linear equalizer, e.g., feed-forward equalizer (FFE), and a nonlinear equalizer, e.g., decision-feedback equalizers (DFE). FFE is a discrete-time finite-impulseresponse (FIR) filter that boosts the input in the frequency range where channel loss is high. It is simple to implement, but since incoming noise is also amplified, symbol detection after FFE shows limited BER performance. On the other hand, DFE does not boost noise and shows better BER, but the timing constraint of the feedback loop makes it difficult to design at a high speed. The loop-unrolling, or speculative, technique mitigates the timing constraint but increases hardware complexity significantly, especially in PAM4, resulting in a loss of energy efficiency. Moreover, DFE alone cannot remove the pre-cursor ISI. Therefore, high-speed wireline communication typically uses both FFE and DFE to improve BER (Dikhaminjia et al., 2018) . Recently, thanks to the development of deep learning, receivers using neural networks, especially recurrent neural networks (RNNs), have been studied, (Ye et al., 2018; Zhou et al., 2019; Kechriotis et al., 1994; Kim et al., 2020; Gomez Diaz et al., 2022) , showing excellent performance for sequence detection. However, for wireline communication, RNNs also have disadvantages that 1) high-speed implementation is difficult due to the timing constraint (past computation results are required for the current operation), and 2) sequence detection incurs large latency. In this paper, we propose a novel neural network architecture that is amenable to high-speed implementation and shows better BER than existing equalizers. The proposed neural network employs feed-forward architecture without any data-dependency-induced timing constraints, which allows efficiently pipelined hardware architecture for high-speed receivers. For better BER performance, the proposed network is trained to mimic the forward-backward (FB) algorithm that has been used to estimate posterior probabilities of the states in Hidden Markov Models (HMMs), while reducing the computational complexity of the FB algorithm. This paper makes the following contributions: • Inspired by the forward-backward algorithm, a novel neural network, NeuralEQ, whose structure is similar to that of the forward-backward algorithm is proposed. • We verify that the computational complexity of the proposed NeuralEQ is lower than that of the forward-backward algorithm. While having much lower complexity, NeuralEQ shows superior BER performance compared to conventional equalizers. • More than half the weights of NeuralEQ can be pruned without performance degradation, which reduces the complexity significantly. Moreover, NeuralEQ is found to be robust against ISI variation.

2. BACKGROUND

The core idea of this paper is to develop a novel neural network architecture that mimics the forwardbackward (FB) algorithm in order to overcome the existing equalizer (EQ) performance such as FFE and DFE while reducing the complexity of the FB algorithm. In the following, a brief description of the FB algorithm and how it can be applied to wireline communication will be provided.

2.1. FORWARD-BACKWARD ALGORITHM

The FB algorithm, which is also known as BCJR (Bahl et al., 1974) or MAP decoder, is widely used in modern wireless communication with its excellent performance. However, since wireline communication requires a much higher data rate, it is challenging to run a computationally heavy algorithm such as FB. It would be helpful to look at the formula in detail to understand the complexity of the FB algorithm. The FB algorithm infers the maximum posterior of all hidden states in a hidden Markov model. Let S t be a hidden state variable, which is an element of a set of hidden states S, and X 1:t be an observed sequence from time 1 to t. Then the FB algorithm is summarized as follows. γ t = P (S t , X 1:t )P (X t+1:T |S t ), 1 ≤ t ≤ T P (S t , X 1:t ) is a probability of hidden states with a sequence from time 1 to t, which refers to forward probability. And P (X t+1:T |S t ) is a conditional probability of sequence occurs when the state at time t is given, which refers to backward probability. Each forward and backward probability can be computed using dynamic programming with the following equations. Note that P (S t = s i , X 1:t = x 1:t ) is replaced with α t i , P (X t+1:T = x t+1:T |S t = s i ) is replaced with β t i for simplicity and a ji is the transition probability from j-th state to i-th state. α t i = j∈|S| α t-1 j a ji P (X t = x t |S t = s i ), β t-1 i = j∈|S| β t j a ji P (X t = x t |S t = s i )

