NEIGHBOR CLASS CONSISTENCY ON UNSUPERVISED DOMAIN ADAPTATION

Abstract

Unsupervised domain adaptation (UDA) is to make a prediction for unlabeled data in a target domain with labeled data from the source domain available. Recent advances exploit entropy minimization and self-training to align the feature of two domains. However, as decision boundary is largely biased towards source data, class-wise pseudo labels generated by target predictions are usually very noisy, and trusting those noisy supervisions might potentially deteriorate the intrinsic target discriminative feature. Motivated by agglomerative clustering which assumes that features in the near neighborhood should be clustered together, we observe that target features from source pre-trained model are highly intrinsic discriminative and have a high probability of sharing the same label with their neighbors. Based on those observations, we propose a simple but effective method to impose Neighbor Class Consistency on target features to preserve and further strengthen the intrinsic discriminative nature of target data while regularizing the unified classifier less biased towards source data. We also introduce an entropybased weighting scheme to help our framework more robust to the potential noisy neighbor supervision. We conduct ablation studies and extensive experiments on three UDA image classification benchmarks. Our method outperforms all existing UDA state-of-the-art.

1. INTRODUCTION

Recent advances in deep neural network have dominated many computer vision tasks, such as image recognition He et al. (2016 ), object detectionGirshick (2015) , and semantic segmentationLong et al. (2015) . However, collection and manual annotation need no trivial human effort, especially for vision tasks like semantic segmentation where dense annotations are required. Thanks to the growth of computer graphics field, it is now possible to leverage CNN to synthetic images with computergenerated annotations (Richter et al. (2016); Ros et al. (2016) ), so unlimited amount of data with free annotation is available for training network in scale. However, directly applying the model trained on synthetic source data to unlabeled target data leads to performance degradation and Unsupervised Domain Adaptation (UDA) aims to tackle this domain shift problem. 2019) methods aim to reduce this discrepancy between two domain by minimizing the adversarial loss. However, the major limitation of adversarial learning is that it only aligns the global feature distribution of two domains without considering the class labels. As the result, a small H H distance does not guarantee the small error on ideal joint hypothesis on the features of two domains Liu et al. (2019) . To alleviate this issue, Entropy minimization (Grandvalet & Bengio (2005) ; Vu et al. ( 2019)) and Self-Training (Lee (2013); Zou et al. (2018) ) are the two dominant methods to enforce the cluster assumption such that network can learn a discriminative feature space by pushing the decision boundary away from densely-distributed area. However, as decision boundary is largely biased to-



widespread of UDA methods were proposed to align the domain-invariant representations by simultaneously minimizing the source error and discrepancy(e.g. H-divergence Ben-David et al. (2010); Hoffman et al. (2016) H H-divergenceBen-David et al. (2010)) between two domain such as the maximum mean discrepancy Tzeng et al. (2014), correlation distanceSun et al. (2016) and etc. Further, adversarial learning-based UDA Ganin & Lempitsky (2015); Tzeng et al. (2017); Radford et al. (2015); Hoffman et al. (2018); Tsai et al. (2018); Sankaranarayanan et al. (2018); Luo et al. (

