LEARNING LISTWISE DOMAIN-INVARIANT REPRE-SENTATIONS FOR RANKING

Abstract

Domain adaptation aims to transfer the knowledge acquired by models trained on (data-rich) source domains to (low-resource) target domains, for which a popular method is invariant representation learning. While they have been studied extensively for problems including classification and regression, how they would apply to ranking problems, where the data and metrics follow a list structure, is not well understood. Theoretically, we establish a domain adaptation generalization bound for ranking under listwise metrics such as MRR and NDCG, that naturally suggests an adaptation method via learning listwise invariant feature representations. Empirically, we demonstrate the benefits of listwise invariant representations by experiments for unsupervised domain adaptation on real-world ranking tasks, including passage reranking. The main novelty of our results is that they are tailored to listwise ranking: the invariant representations are learned at the list level rather than at the item level.

1. INTRODUCTION

Learning to rank applies machine learning to solve ranking problems that are at the core of many everyday products and applications, including and not limited to search engines and recommendation systems (Liu, 2009) . The availability of ever-increasing amounts of training data has enabled larger and larger models to improve the state-of-the-art on more ranking tasks. A prominent example is text retrieval and ranking, where neural language models with billions of parameters easily outperform traditional ranking models, e.g., BM25 (Nogueira et al., 2020) . But the need for abundant data means that large neural models may not benefit tasks with little to no annotated data, where they could actually fare worse than baselines such as gradient boosted decision trees (Qin et al., 2021) . Techniques for extending the benefits of large models to low-resource domains include zero-shot learning and domain adaptation. In the former, instead of directly optimizing for the task of interest with limited data, referred to as the target domain, the model is trained on a data-rich source domain that has a similar data distribution. The latter considers the scenario where (abundant unlabeled) data from the target domain is available, which can be leveraged to estimate the domain shift and improve transferability, e.g., by learning invariant feature representations. This setting and its algorithms are studied extensively for problems including classification and regression (Blitzer et al., 2008; Ganin et al., 2016; Zhao et al., 2018) . For ranking problems, however, existing methods are mostly limited to specific tasks and applications. In fact, due to the inherent list structure of the metrics and data, theoretical explorations of domain adaptation for ranking are only nascent. To this end, we provide the first analysis of domain adaptation for listwise rankingfoot_0 via domaininvariant representations (Section 3), building on the foundational work by Ben-David et al. (2007) for domain adaptation in the binary classification setting. One of the results from our theory is that, when the domain shift is small in terms of the Wasserstein distance, a ranking model optimized on the source is transferrable to the target domain, whose performance under metrics such as MRR and NDCG can be bounded.



In this paper, by listwise ranking, we mean that the data are defined over lists and the ranking metric is listwise. It is not a statement of the ranking model, e.g., whether the predicted rank assignments are obtained in a pointwise, pairwise or listwise manner (Joachims, 2006; Cao et al., 2007), or the interactions between items in the same list are modeled(Pang et al., 2020).

