CONTROLLABLE PARETO MULTI-TASK LEARNING

Abstract

A multi-task learning (MTL) system aims at solving multiple related tasks at the same time. With a fixed model capacity, the tasks would be conflicted with each other, and the system usually has to make a trade-off among learning all of them together. Multiple models with different preferences over tasks have to be trained and stored for many real-world applications where the trade-off has to be made online. This work proposes a novel controllable Pareto multi-task learning framework, to enable the system to make real-time trade-off switch among different tasks with a single model. To be specific, we formulate the MTL as a preference-conditioned multiobjective optimization problem, for which there is a parametric mapping from the preferences to the Pareto stationary solutions. A single hypernetwork-based multi-task neural network is built to learn all tasks with different trade-off preferences among them, where the hypernetwork generates the model parameters conditioned on the preference. For inference, MTL practitioners can easily control the model performance based on different trade-off preferences in real-time. Experiments on different applications demonstrate that the proposed model is efficient for solving various multi-task learning problems.

1. INTRODUCTION

Figure 1 : Controllable Pareto MTL allows practitioners to control the trade-offs among tasks in real time with a single model, which could be desirable for many MTL applications. Multi-task learning (MTL) is important for many real-world applications, such as computer vision (Kokkinos, 2017), natural language processing (Subramanian et al., 2018) , and reinforcement learning (Van Moffaert & Nowé, 2014) . In these problems, multiple tasks are needed to be learned at the same time. An MTL system usually builds a single model to learn several related tasks together, in which the positive knowledge transfer could improve the performance for each task. In addition, using one model to conduct multiple tasks is also good for saving storage costs and reducing the inference time, which could be crucial for many applications (Standley et al., 2020) . However, with fixed learning capacity, different tasks could be conflicted with each other, and can not be optimized simultaneously (Zamir et al., 2018) . The practitioners might need to carefully design and train the tasks into different groups to achieve the best performance (Standley et al., 2020) . A considerable effort is also needed to find a set of suitable weights to balance the performance of each task (Kendall et al., 2018; Chen et al., 2018b; Sener & Koltun, 2018) . For some applications, it might need to train and store multiple models for different trade-off preferences among the tasks (Lin et al., 2019) . In many real-world MTL applications, the system will need to make a trade-off among different tasks in real time, and it is desirable to have the whole set of optimal trade-off solutions. For example, in a self-driving system, multiple tasks must be conducted simultaneously but also compete for a fixed resource (e.g., fixed total inference time threshold), and their preferences could change in real time for different scenarios (Karpathy, 2019). A recommendation system needs to balance multiple criteria among different stakeholders simultaneously, and making trade-off adjustment would be a crucial component (Milojkovic et al., 2019) . Consider the huge storage cost, it is far from ideal to In this paper, we propose a novel controllable Pareto multi-task learning framework, to learn the whole trade-off curve for all tasks with a single model. As shown in Fig. 1 , MTL practitioners can easily control the trade-off among tasks based on their preferences. To our best knowledge, this is the first approach to learn the MTL trade-off curve. The main contributions are: • We formulate solving an MTL problem as a preference-conditioned multiobjective optimization problem, and propose a novel Pareto solution generator to learn the whole tradeoff curve of Pareto stationary solutions. The proposed Pareto solution generator is also a novel contribution to multiobjective optimization. • We propose a general hypernetwork-based multi-task neural network framework, and develop an end-to-end optimization algorithm to train a single model concerning different trade-off preferences among all tasks simultaneously. With the proposed model, MTL practitioners can control the trade-offs among different tasks in real time.

2. RELATED WORK

Multi-Task Learning. The current works on deep multi-task learning mainly focus on designing novel network architecture and constructing efficient shared representation among tasks (Zhang & Yang, 2017; Ruder, 2017) . Different deep MTL networks, with hard or soft parameters sharing structures, haven been proposed in the past few years (Misra et al., 2016; Long et al., 2017; Yang & Hospedales, 2017) . However, how to properly combine and learn different tasks together remains a basic but challenging problem for MTL applications. Although it has been proposed for more than two decades, the simple linear tasks scalarization approach is still the current default practice to combine and train different tasks in MTL problems (Caruana, 1997) . Some adaptive weight methods have been proposed to better combine all tasks in MTL problems with a single model (Kendall et al., 2018; Chen et al., 2018b; Liu et al., 2019; Yu et al., 2020) . However, analysis on the relations among tasks in transfer learning (Zamir et al., 2018) and multitask learning (Standley et al., 2020) show that some tasks might conflict with each other and can not be optimized at the same time. Multiobjective Optimization. Multiobjective optimization itself is a popular research topic in the optimization community. Many gradient-based and gradient-free algorithms have been proposed in the past decades (Fliege & Svaiter, 2000; Désidéri, 2012; Miettinen, 2012) . In addition to MTL, they also can be used in reinforcement learning (Van Moffaert & Nowé, 2014) and neural architecture search (NAS) (Elsken et al., 2019) . However, most methods directly use or modify well-studied multiobjective algorithms to find a single solution or a finite number of Pareto stationary solutions. Recently, Lu et al. ( 2020) proposed a novel supernet-based multi-objective neural architecture transfer framework. This method simultaneously optimizes a set of neural networks, which are sampled from a large supernet, to approximate the optimal trade-off curve among different objectives via a single run. The obtained supernet also supports fast adaption to new tasks and domains. Some efforts have been made to learn the entire trade-off curve for different machine learning applications. Parisi et al. (2016) proposed to learn the Pareto manifold for a multi-objective reinforcement learning problem. However, since this method does not consider the preference for generation, it does not support real-time preference-based adjustment. Very recently, some efforts have been made to learn preference-based solution adjustment for multi-objective reinforcement learning (Yang et al., 2019) and image generation (Dosovitskiy & Djolonga, 2020) with simple linear combinations. There is a concurrent work (Anonymous, 2021) that also proposes to learn the entire trade-off curve for MTL problems by hypernetwork. While this work emphasizes the runtime efficiency on training for multiple preferences, we highlight the benefit of scalability and real-time preference adjustment.



Sener and Koltun (Sener & Koltun, 2018)  propose to treat MTL as a multiobjective optimization problem, and find a single Pareto stationary solution among different tasks. Pareto MTL(Lin et al., 2019)  generalizes this idea, and proposes to generate a set of Pareto stationary solutions with different trade-off preferences. Recent works focuses on generating diverse and dense Pareto stationary solutions(Mahapatra & Rajan, 2020; Ma et al., 2020).

