GAMMA SAMPLING: FINE-GRAINED CONTROLLING LANGUAGE MODELS WITHOUT TRAINING

Abstract

The dominant approaches for controlling language models achieve prominence in controlling high-level attributes (e.g., topic and sentiment). However, these methods often require condition-specific data or are computationally expensive. We propose a new simple guided decoding method, GAMMA SAMPLING, which does not require any training data to achieve fine-grained controllable text generation while maintaining a fast generation speed. GAMMA SAMPLING introduces attribute-related information (provided by humans or language models themselves) into the sampling process to guide language models to generate texts with desired attributes. Since no training is involved, GAMMA SAMPLING can be easily applied to any language model for controllable text generation. Through experiments, we show that GAMMA SAMPLING-steered GPT2 generally outperforms all the representative baselines for controllable generation in terms of diversity, attribute relevance, and overall quality of generated samples.

1. INTRODUCTION

Benefiting from large-scale text data crawled from the web, the state-of-the-art large language models (LMs) achieve great success in language generation. However, although existing models can generate high-quality texts, we have little control over the attributes (e.g., topic and sentiment) of generated outputs. This limitation makes it difficult to apply unconditional LMs to scenarios that require good control over the generated text. How to steer unconditional LMs, i.e., controllable text generation, becomes a topic of real-world significance.

Despite great advances in controllable text generation (Weng 2021), it remains an open question

what the ideal method for controlling the attributes of the generated language is (Mireshghallah et al. 2022 , Yang & Klein 2021 , Meng et al. 2022) . No matter whether training a conditional LM from scratch (Keskar et al. 2019) or fine-tuning an LM (Ziegler et al. 2019 , Xu et al. 2021) , the need for condition-specific data makes these approaches not easily applicable to unconditional LMs. On the other hand, although some methods (Shin et al. 2020 , Zou et al. 2021 , Ghazvininejad et al. 2017 , Pascual et al. 2020 , Lu et al. 2021 ) are data-free, they are often very limited in steerability or computationally intensive. In this paper, we propose GAMMA SAMPLING for fine-grained controlling LMs, which does not require any training data and is computationally efficient. This method is inspired by gamma correction (Applebaum 1952), a nonlinear operation used to encode and decode luminance in video or still image systems. The basic assumption of GAMMA SAMPLING is that some attributes of the generated text are closely related to the occurrences of certain tokens. Therefore, we can increase or decrease the probability of these attribute-related tokens to control the attributes of generated text. Our key contributions are as follows. • GAMMA SAMPLING, as a data-free approach, requires no training on LM or additional discriminators. It can be readily used to achieve controllable text generation for any LM by selecting attribute-related tokens manually or automatically. • GAMMA SAMPLING supports combinations of multiple controllable attributes, and its control strength is fine-grained. Users can determine how strong the attribute relevance should be, and the control can be turned off at any time.

