Technical reports
New approaches to operating system security extensibility
April 2012, 184 pages
This technical report is based on a dissertation submitted October 2010 by the author for the degree of Doctor of Philosophy to the University of Cambridge, Wolfson College.
DOI: 10.48456/tr-818
Abstract
This dissertation proposes new approaches to commodity computer operating system (OS) access control extensibility that address historic problems with concurrency and technology transfer. Access control extensibility addresses a lack of consensus on operating system policy model at a time when security requirements are in flux: OS vendors, anti-virus companies, firewall manufacturers, smart phone developers, and application writers require new tools to express policies tailored to their needs. By proposing principled approaches to access control extensibility, this work allows OS security to be “designed in” yet remain flexible in the face of diverse and changing requirements.
I begin by analysing system call interposition, a popular extension technology used in security research and products, and reveal fundamental and readily exploited concurrency vulnerabilities. Motivated by these failures, I propose two security extension models: the TrustedBSD Mandatory Access Control (MAC) Framework, a flexible kernel access control extension framework for the FreeBSD kernel, and Capsicum, practical capabilities for UNIX.
The MAC Framework, a research project I began before starting my PhD, allows policy modules to dynamically extend the kernel access control policy. The framework allows policies to integrate tightly with kernel synchronisation, avoiding race conditions inherent to system call interposition, as well as offering reduced development and technology transfer costs for new security policies. Over two chapters, I explore the framework itself, and its transfer to and use in several products: the open source FreeBSD operating system, nCircle’s enforcement appliances, and Apple’s Mac OS X and iOS operating systems.
Capsicum is a new application-centric capability security model extending POSIX. Capsicum targets application writers rather than system designers, reflecting a trend towards security-aware applications such as Google’s Chromium web browser, that map distributed security policies into often inadequate local primitives. I compare Capsicum with other sandboxing techniques, demonstrating improved performance, programmability, and security.
This dissertation makes original contributions to challenging research problems in security and operating system design. Portions of this research have already had a significant impact on industry practice.
Full text
PDF (1.2 MB)
BibTeX record
@TechReport{UCAM-CL-TR-818,
author = {Watson, Robert N. M.},
title = {{New approaches to operating system security extensibility}},
year = 2012,
month = apr,
url = {https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-818.pdf},
institution = {University of Cambridge, Computer Laboratory},
doi = {10.48456/tr-818},
number = {UCAM-CL-TR-818}
}