MPhil, Part III, and Part II Project Suggestions (2019-2020)
|
Please contact Eiko Yoneki (email: eiko.yoneki@cl.cam.ac.uk) if you are interested in any project below. * indicates high recommendation! 1. Tuning Neural Networking Compression with Hierarchical Reinforcement Learning * Contact:
Eiko Yoneki (with Michael Schaarschmidt) Executing neural network tasks on mobile and edge
devices (e.g. facial recognition on smart phones) is gaining increasing focus in
the systems community due to constraints in CPU, memory and energy usage on such
devices. Consequently, neural networks trained on large servers usually need to
be compressed to be fit for low power mobile execution. Recent work has explored
various techniques to automatically combine various compression and quantization
techniques [1, 2]. This project will investigate hierarchical reinforcement
learning [3] to improve on these existing search techniques by attempting to
model fine-grained dependencies between different compression and quantization
methods. [1]
https://www.tik.ee.ethz.ch/file/79a7dd6f6370f809e6180c0746232283/mobisys18-liu.pdf [2]
https://arxiv.org/pdf/1806.03723.pdf [3]
https://ai.google/research/pubs/pub46646 2. Model-based Reinforcement Learning in Computer Systems * Contact:
Eiko Yoneki (with Michael Schaarschmidt) Reinforcement learning (RL) is gaining interest as a
generic optimisation and control method in data management tasks such as
resource management/scheduling, database tuning, or stream processing. A central
challenge in applying RL to such problems are long decision evaluation times. A
single step in a real system may take multiple minutes, as opposed to simulators
for standard benchmark tasks (e.g. Atari games), which can execute thousands of
steps per second.
[2]
https://worldmodels.github.io [3]
https://arxiv.org/abs/1810.09028
3. Deep Reinforcement Learning in Networking ** Contact:
Eiko Yoneki Modern network infrastructure is increasingly driven by
software-defined components. This opens up ample opportunities to explore modern
data-driven machine learning methods in traffic protocols. Early work has
explored deep reinforcement learning in routing [0]. Until recently, standard purpose network simulators
(e.g. Ns-3 [1]) were not available via common reinforcement learning benchmark
interfaces such as OpenAI gym. An open source bridge between a gym-like
interface and Ns-3 has now become available [1], and several example tasks are
provided. The aim of this project is to evaluate a baseline deep reinforcement
learning algorithm (e.g. high-performance DQN implementation available in RLgraph
[2]) on some of these tasks, and subsequently implement and explore approaches
from one of the following: Hierarchical RL, model-based RL, Neural combinatorial
optimization/optimal transport theory (guidance can be given depending on
background and student interests). [0]
http://www.cs.huji.ac.il/~schapiram/Learning_to_Route%20(NIPS).pdf [1]
https://arxiv.org/pdf/1810.03943.pdf [2]
https://github.com/rlgraph/rlgraph
4. Optimisation of JVM Garbage Collection using ML (Bayesian Optimisation) * Contact:
Eiko Yoneki This project extends the Cassandra case study [1,2]
in incremental ways:
1) Cassandra database was
only executed in one setting. We could create an auto-tuner that is able to deal
with a wider variety of settings. This includes:
-
Changing the heap size of the JVM
-
Changing the underlying hardware of Cassandra
-
Changing the underlying hardware of YCSB (Yahoo! Cloud Serving Benchmark)
2) Applying it to other JVM
applications (such as other databases, anything where we have a rigorous
benchmark for throughput and latency)
3) Looking at some of the
other JVM flags (we only tuned three) as well as the other JVM garbage collector For evaluation following two points are evaluated.
1) No GC option as baseline
2) Various GC
implementations (i.e. some are good on certain workloads) [1]
https://www.cl.cam.ac.uk/~ey204/pubs/2017_WWW.pdf [2]
https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-900.html 5. Dynamic Task Scheduling on Heterogeneous CPU/GPU Environment using ML for Parallel Processing Contact:
Eiko Yoneki In this project, various aspects of parallel
processing will be explored using a new generation of CPU/GPU integrated board.
We use NVIDIA’s Jetson X2. This new GPUs makes it possible to cluster the GPU
nodes for different scale of parallel processing. Various applications running
over X2 will be investigated including graph processing and neural network
training using the Tensorflow framework. Graph processing can take advantage of
processor heterogeneity to adapt to structural data patterns. The overall aim of
graph processing can be seen as scheduling irregular tasks to optimise
data-parallel heterogeneous graph processing, by analysing the graph at runtime
and dispatching graph elements to appropriate computation. Various task
scheduling strategies over NN applications could be explored such as mixture of
model parallelism and data parallelism. In comparison to the above approach, several types of
GPU machines in cluster computing environment will be explored. Furthermore, our
recent work, BOAT [1], could be used for optimisation of complex parameter
space. [1] V. Dalibard, M. Schaarschmidt, and E. Yoneki:
BOAT: Building Auto-Tuners with Structured Bayesian Optimization, WWW, 2017.
https://github.com/VDalibard/BOAT.
6. Reinforcement Learning for Automatic Index Selection in SQL Databases Contact:
Eiko Yoneki Index selection is a standard database administration
task which is often performed by human experts, typically assisted by profiling
tools [0]. Selecting the optimal set of indices is difficult as it depends on
query structure, data distribution, anad workload distribution. As each index
incurs memory and maintenance cost when inserting new data, the aim of index
selection is to identify the minimal number of indices meeting performance
requirements (query latency). Previous work in this group has implemented a framework
to select compound indices in NoSQL databases such as MongoDB using deep
reinforcement learning and pre-existing log data. The goal of the project is to
investigate the applicability of this framework to indexing semantics [1] in
relational databases such as PostgreSQL.
In general, labelling training data is increasingly the bottleneck in deploying machine learning systems and generating labelling function based on heuristics is a growing research topic. Possibly you can extend a project for the general model building using a weak supervision [3].
[0]
https://stratos.seas.harvard.edu/files/a4-idreos.pdf [1]
https://www.postgresql.org/docs/10/static/indexes-multicolumn.html [2]
https://arxiv.org/pdf/1801.05643.pdf [3] A. Ratner, S. H. Bach, H. Ehrenberg, J. Fries, S. Wu, and C. Re. Snorkel: Rapid training data creation with weak supervision, VLDB, 2017. https://dawn.cs.stanford.edu/2017/05/08/snorkel/
7. Optimising Neural Network Model (Hyperparameter Tuning) over Diffrent Objective Functions Contact:
Eiko Yoneki In this project, you would tune neural network model itself using Bayesian optimisation based approaches [4]. Pick one typical Deep Neural Network application (e.g. image classification), and play with various hyperparameters, where hyperparameters might take substantially different values for the best accuracy or performance. Especially you can set an objective function on reducing the power consumption. You would explore various parameters based experiments and build up a probabilistic model expressing the parameter relationship applying the BOAT technique (see [1] and [2]). Also you would explore recent Google’s hyper parameter tuning on Google Cloud Platform [3], in comparison to the above approach. If time allows, you can explore Python based optimisation [5].
[1] http://www.cl.cam.ac.uk/~ey204/pubs/2017_WWW.pdf [2] http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-900.html [3] https://cloud.google.com/blog/products/gcp/hyperparameter-tuning-on-google-cloud-platform-is-now-faster-and-smarter[4] https://towardsdatascience.com/a-conceptual-explanation-of-bayesian-model-based-hyperparameter-optimization-for-machine-learning-b8172278050f [5] https://towardsdatascience.com/an-introductory-example-of-bayesian-optimization-in-python-with-hyperopt-aae40fff4ff0 8. Optimisation of Parallel SSSP over dynamically changing road networks Contact:
Eiko Yoneki The road network is static but the edge weight changes. In this project, the number of nodes in the road is 8000. The goal is calculating SSSP for 10,000 agents, where source to destination for each agent is static). Simply calculating SSSP sequentially takes a long time. Writing multi-threading code can only increase the parallelism depending on the number of cores. The following is the basic goal and depending on how your idea evolves, it can get additional functionality.
'Reduce the number of agents to calculate path and/or reduce the number of update on the edge weight'. In either case, dividing/clustering the road graph to sub-graphs. The following is a brief idea. a) Reduce the number of agents to calculate - prioritise the subgraph to recalculate based on the density of agents (i.e. <10 agents - no update of subgraph) - adding prediction/trend of agents' movement to identify potential high density subgraphs
b) Reduce the number of update on the edge weight - prioritise the subgraph to recalculate based on the volume of the edge weight update (i.e. <10 update of weight - no update of subgraph)
9. Building Permutation-Structured Action Spaces in Reinforcement Learning Contact:
Eiko Yoneki In this project, you will
explore indexing tasks with Reinforcement Learning. The park project has a
simulator for indexing task [3] and you can see a
series of indexing projects at
https://github.com/park-project/park. You would apply the algorithm used in
our previous work [1] to one of the park projects for
transforming action space to permutation-structured action spaces. You could
plug the agent code from [1] into the simulator. This
will enable a scalable experiments. There is some other query processing tasks
with a simulated query optimiser [2]. If time allows,
you could combine it with other approaches such as graph networks. [1] Learning Index Selection with Structured Action
Spaces, arxiv
https://arxiv.org/abs/1909.07440. [2]
https://github.com/park-project/park/tree/master/park/envs
Contact EmailPlease email to eiko.yoneki@cl.cam.ac.uk for any question. |