SELECTIVE ANNOTATION MAKES LANGUAGE MODELS BETTER FEW-SHOT LEARNERS

Abstract

Many recent approaches to natural language tasks are built on the remarkable abilities of large language models. Large language models can perform in-context learning, where they learn a new task from a few task demonstrations, without any parameter updates. This work examines the implications of in-context learning for the creation of datasets for new natural language tasks. Departing from recent in-context learning methods, we formulate an annotation-efficient, two-step framework: selective annotation that chooses a pool of examples to annotate from unlabeled data in advance, followed by prompt retrieval that retrieves task examples from the annotated pool at test time. Based on this framework, we propose an unsupervised, graph-based selective annotation method, vote-k, to select diverse, representative examples to annotate. Extensive experiments on 10 datasets (covering classification, commonsense reasoning, dialogue, and text/code generation) demonstrate that our selective annotation method improves the task performance by a large margin. On average, vote-k achieves a 12.9%/11.4% relative gain under an annotation budget of 18/100, as compared to randomly selecting examples to annotate. Compared to state-of-the-art supervised finetuning approaches, it yields similar performance with 10-100× less annotation cost across 10 tasks. We further analyze the effectiveness of our framework in various scenarios: language models with varying sizes, alternative selective annotation methods, and cases where there is a test data domain shift. We hope that our studies will serve as a basis for data annotations as large language models are increasingly applied to new tasks. 1 



Here we experiment with GPT-J and Codex-davinci-002. Two selective annotation methods are presented: random selection and our vote-k method. We observe that an appropriate selective annotation method largely improves the in-context learning performance with smaller variance over random selection under varying annotation budgets.

1. INTRODUCTION

Much recent work builds approaches to natural language tasks on the impressive abilities of large language models (e.g., GPT-3; Brown et al., 2020) . Large language models can perform downstream tasks by conditioning generation on a few task demonstrations, thereby avoiding the need for any parameter updates. This new, few-shot learning paradigm is called in-context learning and has become an attractive alternative to supervised finetuning (Liu et al., 2021) . In this work, we study the implications of this remarkable capability of large language models for dataset creation and annotation. We extensively examine how to reduce the manual annotation cost while retaining high in-context learning performance. Although in-context learning was originally proposed for few-shot learning, recent works show that retrieving prompts from a large set of annotated examples is necessary to achieve good performances (Liu et al., 2022; Rubin et al., 2022) . In particular, they show that the performance substantially Distinct from these recent efforts, we establish a two-step framework to better understand and improve the annotation efficiency (Fig. 1 ): the first step is selective annotation that picks a small number of instances to get annotated before test time, followed by prompt retrieval that retrieves in-context examples for each test instance from the annotated data. The total annotation budget is the number of examples selected and annotated in the first step. The second step is bounded by the number of examples that can fit as input to a language model. Based on this framework, we propose an unsupervised, graph-based selective annotation method, named vote-k, that selects diverse and representative instances to be annotated. Our extensive experiments over 10 datasets across diverse tasks (covering classification, commonsense reasoning, dialogue, and text/code generation; see Tab. 2) demonstrate that our graph-based selective annotation method, vote-k ( §2.1), substantially improves the in-context learning performance by balancing the diversity and representativeness of annotated samples. For instance, vote-k, combined with similarity-based prompt retrieval (Liu et al., 2022; Rubin et al., 2022) , achieves a 11.4% relative gain under a budget of 100 annotated examples and a 12.9% relative gain when only 18 examples are annotated; 18 samples can fit into language models' input, meaning the prompt retrieval step is not needed. Moreover, the improvement is consistent across language models with varying sizes (2B-175B parameters) ( §4.2). This finding is in contrast with finetuning, where we cannot see the effectiveness of selective annotation over random baseline, due to outliers (Karamcheti et al., 2021) or training instability (D'Arcy & Downey, 2022) . We hypothesize that in-context learning with similarity-based prompt retrieval is more robust to small annotation sizes and outliers because only the most similar examples are retrieved for each test instance. Indeed, we observe that random prompt retrieval fails to benefit from selective annotation ( §4.4), providing support for our hypothesis. Besides performance comparisons within a fixed annotation budget, we show that selective annotation provides better few-shot performance with 5-100× less annotation cost for new natural language tasks. In-context learning with 18 examples selected by vote-k achieves higher performance than 100 randomly selected examples on 6 out of the 10 tasks. It also outperforms strong finetuning methods by a large margin (Fig. 2 ) and requires 10-100× less annotations for similar performance ( §4.1). We observe that in-context learning quickly (100 or 300 samples are annotated) converges to decent performance when vote-k selective annotation is applied. These results suggest that large language models do not require large annotated datasets (e.g., 10K) due to their ability to adapt to new tasks through simple prompting. Selective annotation also makes in-context learning much more stable. In real-world scenarios, even collecting unlabeled data is non-trivial and introduces randomness. We simulate such randomness in our experimental setting by subsampling the original unlabeled data multiple times. Our results suggest that vote-k selective annotation largely reduces the variance of in-context learning even in this setting (Tab. 2). Further analysis shows larger improvements when there is a domain shift between training and test data (e.g., text from different Amazon users; Koh et al., 2021; §4.3) . Finally, when compared with previous selective annotation methods designed for supervised training/finetuning, we demonstrate that vote-k selective annotation consistently improves the performance ( §4.5). As



Our code is available at https://github.com/HKUNLP/icl-selective-annotation.



Figure 1: Left: two-step framework for in-context learning. Instead of assuming access to large labeled data, we first select a small number of (diverse and representative) unlabeled examples to annotate before test time. At test time, we retrieve in-context examples from the small annotated pool. Right: In-context learning performance over varying annotation budgets averaged over three representative tasks (HellaSwag commonsense reasoning, MRPC paraphrase detection, and MWOZ dialogue state tracking).Here we experiment with GPT-J and Codex-davinci-002. Two selective annotation methods are presented: random selection and our vote-k method. We observe that an appropriate selective annotation method largely improves the in-context learning performance with smaller variance over random selection under varying annotation budgets.

improves when similar examples (under some embedding function) are retrieved as in-context examples specifically for each test input (Liu et al., 2022). Each test sample only requires a few in-context examples in its prompt. Different test instances, however, require different in-context examples with their associated annotations, necessitating a large set of annotated examples.

