SUBSPACE CLUSTERING VIA ROBUST SELF-SUPERVISED CONVOLUTIONAL NEURAL NETWORK

Abstract

Deep subspace clustering (SC) algorithms recently gained attention due to their ability to successfully handle nonlinearities in data. However, the insufficient capability of existing SC methods to deal with data corruption of unknown (arbitrary) origin hinders their generalization ability and capability to address realworld data clustering problems. This paper proposes the robust formulation of the self-supervised convolutional subspace clustering network (S 2 ConvSCN) that incorporates the fully connected (FC) layer and, with an additional spectral clustering module, is capable of estimating the clustering error without using the ground truth. Robustness to data corruptions is achieved by using the correntropy induced metric (CIM) of the error that also enhanced the generalization capability of the network. The experimental finding showed that CIM reduces sensitivity to overfitting during the learning process and yields better clustering results. In a truly unsupervised training environment, Robust S 2 ConvSCN outperforms its baseline version by a significant amount for both seen and unseen data on four well-known datasets.

1. INTRODUCTION

Subspace clustering approaches have achieved encouraging performance when compared with the clustering algorithms that rely on proximity measures between data points. The main idea behind the subspace model is that the data can be drawn from low-dimensional subspaces which are embedded in a high-dimensional ambient space (Lodhi & Bajwa, 2018) . Grouping such data associated with respective subspaces is known as the subspace clustering (Vidal, 2011) . That is, each low-dimensional subspace corresponds to a class or category. Up to now, two main approaches for recovering lowdimensional subspaces are developed: models that are based on the self-representation property, and non-linear generalization of subspace clustering called union of subspaces (UoS) (Lodhi & Bajwa, 2018; Lu & Do, 2008; Wu & Bajwa, 2014; 2015) . UoS algorithms are out of the scope of this work. Self-representation subspace clustering is achieved in two steps: (i) learning representation matrix C from data X and building corresponding affinity matrix A = |C| + |C T |; (ii) clustering the data into k clusters by grouping the eigenvectors of the graph Laplacian matrix that correspond with the leading k eigenvalues. This second step is known as spectral clustering (Ng et al., 2002; Von Luxburg, 2007) . Owning to the presumed subspace structure, the data points obey the self-expressiveness or self-representation property (Elhamifar & Vidal, 2013; Peng et al., 2016b; Liu et al., 2012; Li & Vidal, 2016; Favaro et al., 2011) . In other words, each data point can be represented as a linear combination of other points in a dataset: X=XC. The self-representation approach is facing serious limitations regarding real-world datasets. One limitation relates to the linearity assumption because in a wide range of applications samples lie in nonlinear subspaces, e.g. face images acquired under non-uniform illumination and different poses (Ji et al., 2017) . Standard practice for handling data from nonlinear manifolds is to use the kernel trick on samples mapped implicitly into high dimensional space. Therein, samples better conform to linear subspaces (Patel et al., 2013; Patel & Vidal, 2014; Xiao et al., 2015; Brbić & Kopriva, 2018) . However, identifying an appropriate kernel function for a given data set is quite a difficult task (Zhang et al., 2019b) . The second limitation of existing deep SC methods relates to their assumption that the origin of data corruption is known, in which case the proper error model can be employed. In real-word applications origin of data corruption is unknown. That can severely harm the algorithm's learning process if the non-robust loss function is used. Furthermore, validation

