HIDDEN INCENTIVES FOR AUTO-INDUCED DISTRIBU-TIONAL SHIFT

Abstract

Decisions made by machine learning systems have increasing influence on the world, yet it is common for machine learning algorithms to assume that no such influence exists. An example is the use of the i.i.d. assumption in content recommendation. In fact, the (choice of) content displayed can change users' perceptions and preferences, or even drive them away, causing a shift in the distribution of users. We introduce the term auto-induced distributional shift (ADS) to describe the phenomenon of an algorithm causing a change in the distribution of its own inputs. Our goal is to ensure that machine learning systems do not leverage ADS to increase performance when doing so could be undesirable. We demonstrate that changes to the learning algorithm, such as the introduction of meta-learning, can cause hidden incentives for auto-induced distributional shift (HI-ADS) to be revealed. To address this issue, we introduce 'unit tests' and a mitigation strategy for HI-ADS, as well as a toy environment for modelling real-world issues with HI-ADS in content recommendation, where we demonstrate that strong meta-learners achieve gains in performance via ADS. We show meta-learning and Q-learning both sometimes fail unit tests, but pass when using our mitigation strategy.

1. INTRODUCTION

Consider a content recommendation system whose performance is measured by accuracy of predicting what users will click. This system can achieve better performance by either 1) making better predictions, or 2) changing the distribution of users such that predictions are easier to make. We propose the term auto-induced distributional shift (ADS) to describe this latter kind of distributional shift, caused by the algorithm's own predictions or behaviour (Figure 1 ). ADS are not inherently bad, and are sometimes even desirable. But they can cause problems if they occur unexpectedly. It is typical in machine learning (ML) to assume (e.g. via the i.i.d. assumption) that (2) will not happen. However, given the increasing real-world use of ML algorithms, we believe it is important to model and experimentally observe what happens when assumptions like this are violated. This is the motivation of our work. In many cases, including news recommendation, we would consider (2) a form of cheating-the algorithm changed the task rather than solving it as intended. We care which means the algorithm used to solve the problem (e.g. ( 1) and/or (2)), but we only told it about the ends, so it didn't know 1



Figure 1: Distributions of users over time. Left: A distribution which remains constant over time, following the i.i.d assumption. Right: Auto-induced Distributional Shift (ADS) results in a change in the distribution of users in our content recommendation environment. (see Section 5.2 for details).

