LABEL DISTRIBUTION LEARNING VIA IMPLICIT DIS-TRIBUTION REPRESENTATION

Abstract

In contrast to multi-label learning, label distribution learning characterizes the polysemy of examples by a label distribution to represent richer semantics. In the learning process of label distribution, the training data is collected mainly by manual annotation or label enhancement algorithms to generate label distribution. Unfortunately, the complexity of the manual annotation task or the inaccuracy of the label enhancement algorithm leads to noise and uncertainty in the label distribution training set. To alleviate this problem, we introduce the implicit distribution in the label distribution learning framework to characterize the uncertainty of each label value. Specifically, we use deep implicit representation learning to construct a label distribution matrix with Gaussian prior constraints, where each row component corresponds to the distribution estimate of each label value, and this row component is constrained by a prior Gaussian distribution to moderate the noise and uncertainty interference of the label distribution dataset. Finally, each row component of the label distribution matrix is transformed into a standard label distribution form by using the self-attention algorithm. In addition, some approaches with regularization characteristics are conducted in the training phase to improve the performance of the model.

1. INTRODUCTION

Label distribution learning (LDL) ( Geng ( 2016)) is a novel learning paradigm that characterizes the polysemy of examples. In LDL, the relevance of each label to an example is given by an exact numerical value between 0 and 1 (also known as description degree), and the description degree of all labels forms a distribution to fully characterize the polysemy of an example. Compared with traditional learning paradigms, LDL is a more generalizable and representational learning paradigm that provides richer semantic information. LDL has been successful in several application domains (Gao et al. (2018) 2022)). To obtain the label distribution for learning, there are mainly two ways: one is expert labeling, but labeling is expensive and there is no objective labeling criterion, and the resulting label distribution is highly subjective and ambiguous. The other is to convert a multi-label dataset into a label distribution dataset through a label enhancement algorithm (Xu et al. (2019; 2020) ; Zheng et al. (2021a); Zhao et al. ( 2022b)). However, label enhancement lacks a reliable theory to ensure that the label distribution recovered from logical labels converges to the true label distribution, because logical labels provide a very loose solution space for the label distribution, making the solution less stable and less accurate. In summary, the label distribution dataset used for training has a high probability of inaccuracy and uncertainty, which significantly limits the performance of LDL algorithms. To characterize and mitigate the uncertainty of the label distribution, we propose a novel LDL method based on the implicit label distribution representation. Our work is inspired by recent work on implicit neural representation in 2D image reconstruction (Sitzmann et al. (2020) ). The key idea of implicit neural representation is to represent an object as a function that maps a sequence of coordinates to the corresponding signal, where the function is de-parameterized by a deep neural network. In this paper, we start with a deep network to extract the latent features of input information. Then, the latent features are looked up against the encoded coordinate matrix to generate a label distribution matrix (implicit distribution representation). Finally, the label distribution matrix is computed by a self-attention module to yield a standard label distribution. Note that the goal of the proposed implicit distribution representation is to generate a label distribution matrix with Gaussian distribution constraints as a customized representation pattern.



; Zhao et al. (2021); Chen et al. (2021a); Si et al. (

