CONTRASTIVE LEARNING FOR UNSUPERVISED DOMAIN ADAPTATION OF TIME SERIES

Abstract

Unsupervised domain adaptation (UDA) aims at learning a machine learning model using a labeled source domain that performs well on a similar yet different, unlabeled target domain. UDA is important in many applications such as medicine, where it is used to adapt risk scores across different patient cohorts. In this paper, we develop a novel framework for UDA of time series data, called CLUDA. Specifically, we propose a contrastive learning framework to learn contextual representations in multivariate time series, so that these preserve label information for the prediction task. In our framework, we further capture the variation in the contextual representations between source and target domain via a custom nearest-neighbor contrastive learning. To the best of our knowledge, ours is the first framework to learn domain-invariant, contextual representation for UDA of time series data. We evaluate our framework using a wide range of time series datasets to demonstrate its effectiveness and show that it achieves state-of-the-art performance for time series UDA.

1. INTRODUCTION

Many real-world applications of machine learning are characterized by differences between the domains at training and deployment (Hendrycks & Dietterich, 2019; Koh et al., 2021) . Therefore, effective methods are needed that learn domain-invariant representations across domains. For example, it is well known that medical settings suffer from substantial domain shifts due to differences in patient cohorts, medical routines, reporting practices, etc. (Futoma et al., 2020; Zech et al., 2018) . Hence, a machine learning model trained for one patient cohort may not generalize to other patient cohorts. This highlights the need for effective domain adaptation of time series. Unsupervised domain adaptation (UDA) aims to learn a machine learning model using a labeled source domain that performs well on a similar yet different, unlabeled target domain (Ganin et al., 2016; Long et al., 2018) . So far, many methods for UDA have been proposed for computer vision (Chen et al., 2020a; Ganin et al., 2016; Huang et al., 2021; Kang et al., 2019; Long et al., 2018; Pei et al., 2018; Shu et al., 2018; Singh, 2021; Sun & Saenko, 2016; Tang et al., 2021; Tzeng et al., 2014; 2017; Xu et al., 2020; Zhu et al., 2020) . These works can -in principle -be applied to time series (with some adjustment of their feature extractor); however, they are not explicitly designed to fully leverage time series properties. In contrast, comparatively few works have focused on UDA of time series. Here, previous works utilize a tailored feature extractor to capture temporal dynamics of multivariate time series, typically through recurrent neural networks (RNNs) (Purushotham et al., 2017) , long short-term memory (LSTM) networks (Cai et al., 2021) , and convolutional neural networks (Liu & Xue, 2021; Wilson et al., 2020; 2021) . Some of these works minimize the domain discrepancy of learned features via adversarial-based methods (Purushotham et al., 2017; Wilson et al., 2020; 2021; Jin et al., 2022) or restrictions through metric-based methods (Cai et al., 2021; Liu & Xue, 2021) . Another research stream has developed time series methods for transfer learning from the source domain to the target domain (Eldele et al., 2021; Franceschi et al., 2019; Kiyasseh et al., 2021; Tonekaboni et al., 2021; Yang & Hong, 2022; Yèche et al., 2021; Yue et al., 2022) . These methods pre-train a neural network model via contrastive learning to capture the contextual representation of time series from unlabeled source domain. However, these methods operate on a labeled target

