REWRITER-EVALUATOR FRAMEWORK FOR NEURAL MACHINE TRANSLATION

Abstract

Encoder-decoder architecture has been widely used in neural machine translation (NMT). A few methods have been proposed to improve it with multiple passes of decoding. However, their full potential is limited by a lack of appropriate termination policy. To address this issue, we present a novel framework, Rewriter-Evaluator. It consists of a rewriter and an evaluator. Translating a source sentence involves multiple passes. At every pass, the rewriter produces a new translation to improve the past translation and the evaluator estimates the translation quality to decide whether to terminate the rewriting process. We also propose a prioritized gradient descent (PGD) method that facilitates training the rewriter and the evaluator jointly. Though incurring multiple passes of decoding, Rewriter-Evaluator with the proposed PGD method can be trained with similar time to that of training encoder-decoder models. We apply the proposed framework to improve the general NMT models (e.g., Transformer). We conduct extensive experiments on two translation tasks, Chinese-English and English-German, and show that the proposed framework notably improves the performances of NMT models and significantly outperforms previous baselines.

1. INTRODUCTION

Encoder-decoder architecture (Sutskever et al., 2014) has been extensively used in neural machine translation (NMT) (Vaswani et al., 2017; Zhang et al., 2019) . Given a source sentence, an encoder firstly converts it into hidden representations, which are then conditioned by a decoder to generate the target sentence. Attention mechanism (Bahdanau et al., 2015) is very effective in learning the alignment between a source sentence and a target sentence. Hence, attention mechanism is usually used in the architecture to improve its capability, such as capturing long-distance dependencies. Similar to traditional machine learning efforts (Zhang & Gildea, 2008) , some recent approaches in deep learning attempt to improve encoder-decoder architecture with multiple passes of decoding (Xia et al., 2017; Zhang et al., 2018; Geng et al., 2018) . NMT refers this to polish mechanism (Niehues et al., 2016) . Under this scheme, more than one translations are generated for a source sentence and, except for the first translation, each of them is based on the translation from the previous decoding pass. While these methods have achieved promising results, they lack a proper termination policy to the multi-turn process. Xia et al. ( 2017); Zhang et al. ( 2018) adopt a fixed number of decoding passes that can be inflexible in deciding the optimal number of decoding passes. Geng et al. ( 2018) use reinforcement learning (RL) (Sutton et al., 2000) to automatically decide the optimal number of decoding passes. However, RL is unstable due to its high variance of gradient estimation and objective instability (Boyan & Moore, 1995) . Since these methods may have premature termination or over translation, their potential can be limited. To address this problem, we propose a novel framework, Rewriter-Evaluator, in this paper. It consists of a rewriter and an evaluator. The translation process involves multiple passes. Given a source sentence, at every pass, the rewriter generates a new target sequence aiming at improving the translation from prior passes, and the evaluator measures the translation quality to determine whether to terminate the rewriting process. We also propose a prioritized gradient descent (PGD) method that facilitates training the rewriter and the evaluator jointly. The essential idea is using a priority queue to improve sampling efficiency by collecting the translation cases that yield low scores from the evaluator for next-pass rewriting. The size of the queue is a few times larger than the batch size.

