MULTI-DOMAIN LONG-TAILED LEARNING BY AUG-MENTING DISENTANGLED REPRESENTATIONS

Abstract

There is an inescapable long-tailed class-imbalance issue in many real-world classification problems. Existing long-tailed classification methods focus on the single-domain setting, where all examples are drawn from the same distribution. However, real-world scenarios often involve multiple domains with distinct imbalanced class distributions. We study this multi-domain long-tailed learning problem and aim to produce a model that generalizes well across all classes and domains. Towards that goal, we introduce TALLY, which produces invariant predictors by balanced augmenting hidden representations over domains and classes. Built upon a proposed selective balanced sampling strategy, TALLY achieves this by mixing the semantic representation of one example with the domain-associated nuisances of another, producing a new representation for use as data augmentation. To improve the disentanglement of semantic representations, TALLY further utilizes a domain-invariant class prototype that averages out domain-specific effects. We evaluate TALLY on four long-tailed variants of classical domain generalization benchmarks and two real-world imbalanced multi-domain datasets. The results indicate that TALLY consistently outperforms other state-of-the-art methods in both subpopulation shift and domain shift.



. This long-tailed setting arises frequently in practice, such as wildlife recognition (Beery et al., 2020) . Classifiers tend to be biased towards majority classes and perform poorly on classbalanced test distributions, i.e. when there is a shift in the label distribution between training and test. Existing approaches address the long-tailed problem by modifying the data sampling strategy (Chawla et al., 2002; Zhang & Pfister, 2021) , adjusting the loss function for different classses (Cao et al., 2019; Hong et al., 2021) , or augmenting minority classes (Chou et al., 2020; Zhong et al., 2021) . Unlike these works, which focus on single-domain long-tailed learning, we study multi-domain long-tailed learning, where each domain has its own long-tailed distribution. Take wildlife recognition as an example (Figure 1 ): images of wildlife are collected from various locations, and the distribution over species (classes) at each location is typically imbalanced and the class distribution also varies between locations. In multi-domain long-tailed classification, the classifiers need to handle distribution shift amidst class imbalance. Here, we focus on two types of distribution shift: subpopulation shift and domain shift. In subpopulation shift, we train a model on data from multiple domains and evaluate the model on a test set with balanced domain-class pairs. In the wildlife recognition example, species are often concentrated at only a few locations, creating a spurious correlation between the label (species) and the domain (location). A machine learning model trained on the entire population may fail on the test set when this correlation does not hold anymore. In domain shift, we expect the trained model to generalize well to completely new test domains. For example, in wildlife recognition, we train a model on data from a fixed set of training locations and then deploy the model to new test locations.



Figure 1: Illustration of imbalanced class distributions across domains in iWildCam, a wildlife recognition benchmark (Beery et al., 2020). Both subpopulation shift and domain shift settings are illustrated. Deep classification models can struggle when the number of examples per class varies dramatically (Beeryet al., 2020; Zhang et al., 2021). This long-tailed setting arises frequently in practice, such as wildlife recognition(Beery et al., 2020). Classifiers tend to be biased towards majority classes and perform poorly on classbalanced test distributions, i.e. when there is a shift in the label distribution between training and test. Existing approaches address the long-tailed problem by modifying the data sampling strategy(Chawla et al.,  2002; Zhang & Pfister, 2021), adjusting the loss function for different classses(Cao et al., 2019; Hong et al.,  2021), or augmenting minority classes(Chou et al.,  2020; Zhong et al., 2021). Unlike these works, which focus on single-domain long-tailed learning, we study multi-domain long-tailed learning, where each domain has its own long-tailed distribution. Take wildlife recognition as an example (Figure1): images of wildlife are collected from various locations, and the distribution over species (classes) at each location is typically imbalanced and the class distribution also varies between locations.

