Cloud Computing

Aims

This module aims to teach students the fundamentals of Cloud Computing covering topics such as virtualization, data centres, cloud resource management, cloud storage and popular cloud applications including batch and data stream processing. Emphasis is given on the different backend technologies to build and run efficient clouds and the way clouds are used by applications to realise computing on demand. The course will include practical tutorials on cloud infrastructure technologies. Students will be assessed via a Cloud-based coursework project.

Lectures

• Introduction to Cloud Computing
• Data centres
• Virtualization I
• Virtualization II
• MapReduce
• MapReduce advanced
• Resource management for virtualized data centres
• Cloud storage
• Cloud-based data stream processing

Objectives

By the end of the course students should:

• understand how modern clouds operate and provide computing on demand;
• understand about cloud availability, performance, scalability and cost;
• know about cloud infrastructure technologies including virtualization, data centres, resource management and storage;
• know how popular applications such as batch and data stream processing run efficiently on clouds;

Assessment

• Assignment 1, worth 55% of the final mark. Develop batch processing applications running on a cluster assessed through automatic testing, code inspection and a report.
• Assignment 2, worth 30% of the final mark: Design and develop a framework for running the batch processing applications from Assignment 1 on a cluster according to certain resource allocation policies. This assigment will be assessed by an expert, testing, code inspection and a report.
• Assignment 3, worth 15% of the final mark: Write two individual project reports describing your work for assignments 1 and 2. Write two sets of scripts to manage and run your code. Reports and scripts will be submitted with each assignment.

Recommended reading

Marinescu, D.C. Cloud Computing, Theory and Practice. Morgan Kaufmann.
Barham, P., et. al. (2003). “Xen and the Art of Virtualization”. In Proceedings of SOSP 2003.
Charkasova, L., Gupta, D. and Vahdat, A. (2007). “Comparison of the Three CPU Schedulers in Xen”. In SIGMETRICS 2007.
Dean, J. and Ghemawat, S. (2004). “MapReduce: Simplified Data Processing on Large Clusters”. In Proceedings of OSDI 2004.
Zaharia, M, et al. (2008). “Improving MapReduce Performance in Heterogeneous Environments”. In Proceedings of OSDI 2008.
Hindman, A., et al. (2011). “Mesos: A Platform for Fine-Grained Resource Sharing in Data Center”. In Proceedings of NSDI 2011.
Schwarzkopf, M., et al. (2013). “Omega: Flexible, Scalable Schedulers for Large Compute Clusters”. In EuroSys 2013.
Ghemawat, S. (2003). “The Google File System”. In Proceedings of SOSP 2003.
Chang, F. (2006). “Bigtable: A Distributed Storage System for Structured Data”. In Proceedings of OSDI 2006.
Fernandez, R.C., et al. (2013). “Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management”. In SIGMOD 2013.