Computer Laboratory

CAMOPT: Optimisation for Computer Systems in High Dimensional and Dynamic Parameter Space:
Bayesian Optimisation and Reinforcement Learning with Structural Information

Home

Contact

  Overview

The project investigates how to apply Structured Bayesian Optimisation (SBO) and Deep Reinforcement Learning (DRL) towards automatic performance tuning of systems, where parameter space is high-dimensional and complex. Modern computer systems often have many tunable parameters to accommodate a range of workloads including dynamic change of parameters. Even for experts, a lengthy trial-and-error process is needed to obtain sufficient understanding about customer's workloads. First, tuning complex and high dimensional is a resource extensive task with standard optimisation methods, and individual experiments may take hours. We attempt to alleviate these constraints by injecting expert knowledge into the optimisation procedure, thus guiding the model to high performing configuration regions using our novel Structured Bayesian Optimisation (SBO). Second, we investigate Reinforcement Learning (RL) and aim to bring similar improvements to the control of dynamically evolving tasks such as scheduling or resource management. SBO or RL based optimisation can build an adaptive and robust tuner to achieve optimal performance.  

   Goal and Objectives

The goal is to adopt ML-based optimisation for computer systems in a complex, large parameter space. A key novel aspect is the model decomposition method that subsequently accelerates the process. We investigate innovative optimisation and control mechanisms to minimise the need for human-designed heuristics, while injecting a small amount of expert knowledge into the optimisation procedure.

  Structured Bayesian Optimisation and BOAT Framework

Our BespOke Auto-Tuner (BOAT) framework allows developers to build efficient bespoke auto-tuners, which can achieve their iterative evaluation much faster. The core of BOAT is a novel extension of Bayesian Optimisation, called Structured Bayesian Optimisation (SBO), which leverages contextual information in the form of a probabilistic model of systems behaviour. BOAT provides a Probabilistic C++ library for building such a model. Adding structural information in a probabilistic model of the objective function in Bayesian optimisation outperforms standard Gaussian processes by orders of magnitude. An initial case study has been the hyper-parameters of a convolutional neural network to optimise the accuracy of the model and the relationship between these hyper-parameters. This probabilistic model will be used to obtain optimal values of hyper-parameters for target tasks. Diverse range of objectives for optimisation could be studied, such as model size or robustness on adversarial images.

  RLgraph: Flexible Computation Graphs for Deep Reinforcement Learning

RL has distinct advantages for computer systems. We have already identified combinatorial optimisation and discrete decision making problems as promising avenues, because they are difficult to address with optimisation methods targeting continuous functions (e.g. Bayesian Optimisation). We have also identified the lack of standard software and tools as an obstacle in contemporary research, which results in the need to re-implement over and over the same set of standard algorithms or to use poorly understood or motivated open-source algorithms. Computer systems experiments are distinct from typical RL research domains. They are more expensive to execute than common RL benchmarks such as games, and are also easier to parallelise and restart than other traditional RL domains such as robotics. Moreover, contemporary computer systems provide vast amounts of real-time monitoring and performance information, which can be used in the form of historical traces to extract initial model information, thus shortening training times. This project will generate a software stack addressing the algorithmic level as a standard interface to common algorithms and the system model level. This approach will help to connect typical systems, such as databases or stream engines, to a reinforcement learning training and execution cycle. An initial case study could be resource management/scheduling in stream processing.  

  Publications / Code

[1] O. Hope and E. Yoneki: GDDR: GNN-based Data-Driven Routing, ICDCS, 2021. (arXiv preprint)

[2] S. Alabed and E. Yoneki: High-Dimensional Bayesian Optimization with Multi-Task Learning for RocksDB, EuroMLSys at EuroSys, 2021. (arXiv preprint)

[3] H. Brown, K. Fricke and E. Yoneki: World-Models for Bitrate Streaming, MDPI Journal of Applied Science: Special Issue on AI in Mobile Networks, 2020. (preprint)

[4] M. Schaarschmidt, K. Fricke and E. Yoneki: Wield: Systematic Reinforcement Learning with Progressive Randomization, https://arxiv.org/abs/1909.068448, 2019.

[5] J. Welborn, M. Schaarschmidt and E. Yoneki: Learning Index Selection with Structured Action Spaces, https://arxiv.org/abs/1909.07440, 2019.

[6] M. Schaarschmidt, S. Mika, K. Fricke and E. Yoneki: RLgraph: Flexible Computation Graphs for Deep Reinforcement Learning, https://arxiv.org/abs/1810.09028, SysML, 2019.

[7] V. Dalibard, M. Schaarschmidt, and E. Yoneki: BOAT: Building Auto-Tuners with Structured Bayesian Optimization. World Wide Web Conference, Systems and Infrastructure Track, 2017 (PDF).