OVERPARAMETERISATION AND WORST-CASE GENER-ALISATION: FRIEND OR FOE?

Abstract

Overparameterised neural networks have demonstrated the remarkable ability to perfectly fit training samples, while still generalising to unseen test samples. However, several recent works have revealed that such models' good average performance does not always translate to good worst-case performance: in particular, they may perform poorly on subgroups that are under-represented in the training set. In this paper, we show that in certain settings, overparameterised models' performance on under-represented subgroups may be improved via post-hoc processing. Specifically, such models' bias can be restricted to their classification layers, and manifest as structured prediction shifts for rare subgroups. We detail two post-hoc correction techniques to mitigate this bias, which operate purely on the outputs of standard model training. We empirically verify that with such post-hoc correction, overparameterisation can improve average and worst-case performance.

1. INTRODUCTION

Overparameterised neural networks have demonstrated the remarkable ability to perfectly fit training samples, while still generalising to unseen test samples (Zhang et al., 2017; Neyshabur et al., 2019; Nakkiran et al., 2020) . However, several recent works have revealed that overparameterised models' good average performance does not translate to good worst-case performance (Buolamwini & Gebru, 2018; Hashimoto et al., 2018; Sagawa et al., 2020a; b) . In particular, the test performance of such models may be poor on certain subgroups that are under-represented in the training data. Worse still, such degradation can be exacerbated as model complexity increases. This indicates the unsuitability of such models in ensuring fairness across subgroups, a topical concern given the growing societal uses of machine learning (Dwork et al., 2012; Hardt et al., 2016; Buolamwini & Gebru, 2018) . Why does overparameterisation induce such unfavourable bias, and how can one correct for it? Sagawa et al. ( 2020a) demonstrated how such models may fit to spurious correlations that explain under-represented samples, which can generalise poorly. Sagawa et al. (2020b) further posited that overparameterised models have an inductive bias towards memorising labels for as few samples as possible, which are invariably those from under-represented subgroups. To mitigate such bias, existing approaches include subsampling majority subgroups (Sagawa et al., 2020b) , and modifying the training objective (Sagawa et al., 2020a; Nam et al., 2020; Zhang et al., 2020; Goel et al., 2020) . This suggests two important points regarding overparameterised models' performance: In this paper, we establish that while overparameterised models are biased against under-represented examples, in certain settings, such bias may be easily corrected via post-hoc processing of the model outputs. Specifically, such models' bias can be largely restricted to their classification layers, and manifest as structured shifts in predictions for rare subgroups. We thus show how two simple techniques applied to the model outputs -classifier retraining based on the learned representations, and correction of the classification threshold -can help overparameterised models improve worstsubgroup performance over underparameterised counterparts. Consequently, even with standard training, overparameterised models can learn sufficient information to model rare subgroups.



(a) with standard training, increasing model complexity exacerbates degradation on rare subgroups; (b) controlling this degradation may require alternate training objectives or procedures.

