MULTI-TASK MULTICRITERIA HYPERPARAMETER OPTIMIZATION

Abstract

We present a new method for searching optimal hyperparameters among several tasks and several criteria. Multi-Task Multi Criteria method (MTMC) provides several Pareto-optimal solutions, among which one solution is selected with given criteria significance coefficients. The article begins with a mathematical formulation of the problem of choosing optimal hyperparameters. Then, the steps of the MTMC method that solves this problem are described. The proposed method is evaluated on the image classification problem using a convolutional neural network. The article presents optimal hyperparameters for various criteria significance coefficients.

1. INTRODUCTION

Hyperparameter optimization (Hutter et al., 2009) is an important component in the implementation of machine learning models (for example, logistic regression, neural networks, SVM, gradient boosting, etc.) in solving various tasks, such as classification, regression, ranking, etc. The problem is how to choose the optimal parameters when a trained model is evaluated using several sets and several criteria. This article describes a method to solving the above problem. We will present the results of experiments on the selection of hyperparameters obtained using the proposed approach (MTMC) with various criteria significance coefficients. The article is organized as follows. First, we discuss related work in Section 2. Section 3 describes the proposed method. Section 4 presents the results of experiments on the selection of optimal hyperparameters. Section 5 contains the conclusion and future work.

2. RELATED WORK

Hyperparameter optimization is applied to solve various problems such as computer vision (Bergstra et al., 2013; Dong et al., 2019 ), robotics (Mahmood et al., 2018; Tran et al., 2020) , natural language processing (Wang et al., 2015; Dernoncourt & Lee, 2016) and speech synthesis (Koriyama et al., 2014) . The problem of choosing optimal hyperparameters has long been known. Existing solutions can be considered for the following features: 1. Number of optimal solutions. 2. Number of tasks to be solved.

3.. Number of criteria for choosing the optimal solution.

In this article, task means a set of images with a number of classes N classes and a number of images N images . There are examples of the same classes between tasks, the difference is how the images are made (different lighting, background and used cameras). Criteria is a quantitative characteristic of training / evaluation a neural network on a task (e.g. accuracy, latency or epoch of training convergence).

