GAMMA SAMPLING: FINE-GRAINED CONTROLLING LANGUAGE MODELS WITHOUT TRAINING

Abstract

The dominant approaches for controlling language models achieve prominence in controlling high-level attributes (e.g., topic and sentiment). However, these methods often require condition-specific data or are computationally expensive. We propose a new simple guided decoding method, GAMMA SAMPLING, which does not require any training data to achieve fine-grained controllable text generation while maintaining a fast generation speed. GAMMA SAMPLING introduces attribute-related information (provided by humans or language models themselves) into the sampling process to guide language models to generate texts with desired attributes. Since no training is involved, GAMMA SAMPLING can be easily applied to any language model for controllable text generation. Through experiments, we show that GAMMA SAMPLING-steered GPT2 generally outperforms all the representative baselines for controllable generation in terms of diversity, attribute relevance, and overall quality of generated samples.

1. INTRODUCTION

Benefiting from large-scale text data crawled from the web, the state-of-the-art large language models (LMs) achieve great success in language generation. However, although existing models can generate high-quality texts, we have little control over the attributes (e.g., topic and sentiment) of generated outputs. This limitation makes it difficult to apply unconditional LMs to scenarios that require good control over the generated text. How to steer unconditional LMs, i.e., controllable text generation, becomes a topic of real-world significance. Despite great advances in controllable text generation (Weng 2021), it remains an open question what the ideal method for controlling the attributes of the generated language is (Mireshghallah et al. 2022 , Yang & Klein 2021 , Meng et al. 2022) . No matter whether training a conditional LM from scratch (Keskar et al. 2019) or fine-tuning an LM (Ziegler et al. 2019 , Xu et al. 2021) , the need for condition-specific data makes these approaches not easily applicable to unconditional LMs. On the other hand, although some methods (Shin et al. 2020 , Zou et al. 2021 , Ghazvininejad et al. 2017 , Pascual et al. 2020 , Lu et al. 2021 ) are data-free, they are often very limited in steerability or computationally intensive. In this paper, we propose GAMMA SAMPLING for fine-grained controlling LMs, which does not require any training data and is computationally efficient. This method is inspired by gamma correction (Applebaum 1952), a nonlinear operation used to encode and decode luminance in video or still image systems. The basic assumption of GAMMA SAMPLING is that some attributes of the generated text are closely related to the occurrences of certain tokens. Therefore, we can increase or decrease the probability of these attribute-related tokens to control the attributes of generated text. Our key contributions are as follows. • GAMMA SAMPLING, as a data-free approach, requires no training on LM or additional discriminators. It can be readily used to achieve controllable text generation for any LM by selecting attribute-related tokens manually or automatically. • GAMMA SAMPLING supports combinations of multiple controllable attributes, and its control strength is fine-grained. Users can determine how strong the attribute relevance should be, and the control can be turned off at any time. • We compared GAMMA SAMPLING-steered GPT2 in all sizes (i.e., Small, Medium, Large, and XL) with several common methods for controllable generation through both automatic and human evaluations. The results show that our method generally outperformed all the baselines in terms of diversity, attribute relevance, and overall quality.

2. BACKGROUND

The typical controllable generation is about modelling a probabilistic model p(x|a), that is, generating text x based on an attribute a. In contrast, for unconditional LMs, only p(x) can be obtained directly. However, by using certain methods for controlling unconditional LMs, it is still possible to make the generated text x have an attribute a. There are several common approaches for controllable generation, each with its pros and cons. Conditional Language Model Conditions are introduced to models during training phases, which can be obtained from the metadata of the training data. However, as the entire LM needs to be trained from scratch, it requires a large amount of condition-specific data as well as Guided Decoding Although decoding does not affect any trainable parameters of LMs, it is a critical part of text generation. Guided decoding (Ghazvininejad et al. 2017 , Ghosh et al. 2017 , Pascual et al. 2020 , Lu et al. 2021 , Liu et al. 2021a , Pascual et al. 2021 ) introduces attribute-related



Common methods of controllable generation. Modules that introduce attribute-related information are marked in red. • GAMMA SAMPLING is computationally efficient, as it requires only a slight modification at decoding time. Unlike other guided decoding methods (Pascual et al. 2021), the time cost of GAMMA SAMPLING is insensitive to the number of controlled tokens. Compared to PPLM (Dathathri et al. 2020), the generation speed of GAMMA SAMPLING is at least 100× faster.

considerable training costs. Furthermore, conditional LMs such as CTRL (Keskar et al. 2019) fall short in controlling what not to generate, e.g., detoxification and anti-degeneration (Gehman et al. 2020,Ma et al. 2020).Fine-tuned Language Model Fine-tuned LMs(Ziegler et al. 2019, Xu et al. 2021) usually strike a good balance between training cost and generation quality. These models are based on existing large models with all the weights in them fine-tuned, limiting the fine-tuning to the top or additional layers only, or introducing discriminators(Krause et al. 2021, Liu et al. 2021a). However, finetuned LMs still require condition-specific data. Furthermore, models such as PPLM(Dathathri et al.  2020), which combine multiple small attribute models with a large LM, could cause computational efficiency to become unacceptable by requiring multiple passes at every decoding step.Shin et al. 2020, Zou et al. 2021), unconditional LMs can be used to solve a wide range of downstream tasks. Although prompt engineering has become a recent research hotspot(Liu et al. 2021b), minor differences in prompt usually have a big impact on the performance of downstream tasks(Kojima et al. 2022).

