GRADUATED NON-CONVEXITY FOR ROBUST SELF-TRAINED LANGUAGE UNDERSTANDING Anonymous

Abstract

Self-training has been proven to be an efficient strategy for unsupervised finetuning of language models using unlabeled data and model-generated pseudolabels. However, the performance of self-trained models is unstable under different conditions of the training and evaluation data, influenced by both data distribution and pseudo-label accuracy. In this work, we propose an outlier robust self-training method based on graduated non-convexity (GNC) to mitigate the problem. We construct self-training as a non-convex optimization problem with outlier training examples. The models are self-trained with robust cost functions according to Black-Rangarajan Duality. The algorithm learns slack variables as the loss weights for all training samples. The slack variables are used to calibrate the loss items during training to update the model parameters. The calibrated loss items lead to more robust self-trained models against different training and evaluation data and tasks. We conduct experiments on few-shot natural language understanding tasks with labeled and unlabeled data examples. Experimental results show that the proposed loss calibration method improves the performance and stability of self-training on different tasks, benefiting the robustness against incorrect pseudo-labels, imbalanced training data, overfitting, and adversarial evaluation data.

1. INTRODUCTION

Recent developments in large-scale pretrained language models has significantly improved the performance of natural language understanding tasks (Devlin et al., 2018; Liu et al., 2019; Clark et al., 2020; He et al., 2020; Brown et al., 2020) . After pretraining, these models are typically fine-tuned on task-specific training data with human-generated labels. However, human-generated labels are not available (or large enough) for all tasks of interest. When there is a significant number of unlabeled examples, a pretrained model can utilize techniques such as self-training to improve performance (He et al., 2019; Zoph et al., 2020 ). An issue with this approach is that the generated pseudo-labels can be noisy (Zhao et al., 2021; Lang et al., 2022; Zhang & Zhou, 2011) , since a pretrained model can make wrong predictions for unseen examples. We propose a learning-based loss calibration strategy that tunes the loss weights for each data example during self-training. In this approach, a subset of training data are assigned low weights for calculating the overall training loss, leading to less influence in parameter updates. To learn the loss weights, we employ the graduated non-convexity (GNC) strategy (Yang et al., 2020) based on Black-Rangarajan Duality (Black & Rangarajan, 1996) . Under the fully supervised and outlier-free setting, the model parameters are updated to optimize the total cost of all training samples, while under the self-training setting, the training set usually contains outliers with wrong pseudo-labels. We thus optimize a robust cost function L r = i ρ[l(ŷ i , x i ); θ] where l(•) is the selected loss function and D train = {(x i , y i )|i ∈ [0, N ]} is the training set, and θ stands for the set of trainable parameters. ρ(•) is a robust cost function and ŷi stands for an element of a noisy pseudo label set. Such methods have been widely applied in computer vision (Black & Rangarajan, 1996) and robot perception (Yang et al., 2020) where there exists a ground-truth

