Department of Computer Science and Technology

CHERI

Qemu-CHERI

Qemu-CHERI is an adaptation of the popular Qemu ISA emulator to implement the CHERI-MIPS instruction set. Qemu-CHERI boots CheriBSD and passes all of the CheriBSD MIPS and CheriABI tests. It implements 256-bit capabilities, as well as CHERI-128 (ISAv5, ISAv6) compressed capabilities, and "magic" 128-bit capabilities (in which architectural 256-bit capabilities can be compressed to 128 bits without loss when loaded and stored from memory). This latter mode permits easier diagnosis of software bugs associated with 128-bit-sized capabilities vs. those associated specifically with capability compression (e.g., changed alignment requirements). More information on Qemu itself can be found on the Qemu Wiki.

Contents

Building and Installing qemu-system-cheri

On FreeBSD, qemu-system-cheri can be installed using the package command: 

# pkg install qemu-cheri

For 128-bit capabilities use the following instead: 

# pkg install qemu-cheri128

It can also be installed using FreeBSD ports: 

# cd /usr/ports/emulators/qemu-cheri; make install

For 128-bit capabilities use: 

# cd /usr/ports/emulators/qemu-cheri128; make install

Building QEMU from git

The latest version can be built from the source in the CTSRD-CHERI Github repository. We strongly recommend that you use the cheribuild.py script to build QEMU and a CheriBSD disk image. The simplest way of getting CheriBSD running is by running cheribuild.py run -d (-d means build all dependencies). Further information on how to use cheribuild and configure build paths and options can be found in the GitHub README

Manual build instructions

FreeBSD users will need to use gmake instead of make when building Qemu manually: 

# git clone http://github.com/CTSRD-CHERI/qemu
# cd qemu
# ./configure --target-list=cheri-softmmu --prefix=/usr/local --disable-linux-user --disable-linux-aio --disable-kvm --disable-xen --extra-cflags=-g
# make
# make install

For compressed 128-bit capabilities, add the "-DCHERI_128" flag to CFLAGS: 

[...]
# ./configure --target-list=cheri-softmmu --prefix=/usr/local --disable-linux-user --disable-linux-aio --disable-kvm --disable-xen --extra-cflags="-g -DCHERI_128"

For "magic" 128-bit capabilities, add "-DCHERI_MAGIC128" flag to CFLAGS: 

[...]
# ./configure --target-list=cheri-softmmu --prefix=/usr/local --disable-linux-user --disable-linux-aio --disable-kvm --disable-xen --extra-cflags="-g -DCHERI_MAGIC128"

Building CheriBSD Kernel and Disk Image

To build a suitable kernel and disk image follow the detailed instructions in the CHERI Programmer's Guide. More information can also be found on the CheriBSD web page. Use the CHERI_MALTA64 (or CHERI128_MALTA64) kernconf to build the kernel. In short, on a FreeBSD host:

  1. Install dependencies, if not already installed: 
    # pkg install git gettext cmake ninja
  2. Build CHERI clang cross compiler (also see Chapter 2 of CHERI Programmer's Guide): 
    % git clone git://github.com/CTSRD-CHERI/llvm.git
% cd llvm/tools
% git clone git://github.com/CTSRD-CHERI/clang.git
% cd ..
% mkdir Build
% cd Build
% cmake -G Ninja -DCMAKE_BUILD_TYPE:STRING=Release -DBUILD_SHARED_LIBS:BOOL=ON -DLLVM_DEFAULT_TARGET_TRIPLE:STRING=cheri-unknown-freebsd ..
% ninja

    or for 128-bit capabilities add "-DLLVM_CHERI_IS_128=ON" to the cmake options: 

    % cmake -G Ninja -DCMAKE_BUILD_TYPE:STRING=Release -DLLVM_CHERI_IS_128=ON -DBUILD_SHARED_LIBS:BOOL=ON -DLLVM_DEFAULT_TARGET_TRIPLE:STRING=cheri-unknown-freebsd ..

    Remove the incompatible std*.h and limits.h files included with Clang/LLVM: 

    % rm lib/clang/3.*/include/std* lib/clang/3.*/include/limits.h
  3. Build CheriBSD kernel and disk image (also see Chapter 7 of CHERI Programmer's Guide): 
    % git clone git://github.com/CTSRD-CHERI/cheribsd.git
% cd cheribsd
% git submodule init
% git submodule update
% setenv MAKEOBJDIRPREFIX /var/tmp/obj
% make CHERI=256 CHERI_CC=/path/to/cheri-clang -j16 buildworld
% make CHERI=256 KERNCONF=CHERI_MALTA64 -j16 buildkernel

    Where /path/to/cheri-clang is the path to the clang cross compiler built above.

    Or for 128-bit capabilities, use "CHERI=128" and "KERNCONF=CHERI128_MALTA64": 

    % git clone git://github.com/CTSRD-CHERI/cheribsd.git
% git submodule init
% git submodule update
% cd cheribsd
% setenv MAKEOBJDIRPREFIX /var/tmp/obj
% make CHERI=128 CHERI_CC=/path/to/cheri-clang -j16 buildworld
% make CHERI=128 KERNCONF=CHERI128_MALTA64 -j16 buildkernel

    As root, create a disk image: 

    # setenv MAKEOBJDIRPREFIX /var/tmp/obj
# mkdir /var/tmp/root
# make CHERI=256 CHERI_CC=/path/to/cheri-clang installworld DESTDIR=/var/tmp/root
# make CHERI=256 KERNCONF=CHERI_MALTA64 installkernel DESTDIR=/var/tmp/root
# make CHERI=256 CHERI_CC=/path/to/cheri-clang distribution DESTDIR=/var/tmp/root
# echo '/dev/ada0 / ufs rw 1 1' > /var/tmp/root/etc/fstab
# makefs -M 2g -B be /var/tmp/disk.img /var/tmp/root

    Again, use "CHERI=128" and "KERNCONF=CHERI128_MALTA64" for 128-bit capabilities: 

    # setenv MAKEOBJDIRPREFIX /var/tmp/obj
# mkdir /var/tmp/root
# make CHERI=128 CHERI_CC=/path/to/cheri-clang installworld DESTDIR=/var/tmp/root
# make CHERI=128 KERNCONF=CHERI128_MALTA64 installkernel DESTDIR=/var/tmp/root
# make CHERI=128 CHERI_CC=/path/to/cheri-clang distribution DESTDIR=/var/tmp/root
# echo '/dev/ada0 / ufs rw 1 1' > /var/tmp/root/etc/fstab
# makefs -M 2g -B be /var/tmp/disk.img /var/tmp/root
  4. Copy the disk image (/var/tmp/disk.img) and kernel (/var/tmp/root/boot/kernel) to the host system that has qemu-system-cheri installed. Pre-built disk images and CheriBSD kernels can be found at: https://www.cl.cam.ac.uk/research/security/ctsrd/mips64-packages/qemu-cheri-images/

Running CheriBSD with Qemu-Cheri

CheriBSD Startup and Regression Testing

Using the kernel and disk image created above you can boot CheriBSD into multiuser and run the CheriBSD regression tests. Use the 'malta' system model (-M malta) with 2G of system memory (-m 2048): 

$ qemu-system-cheri -M malta -kernel ./kernel -nographic -hda ./disk.img -m 2048
[...]
Copyright (c) 1992-2015 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #0 199c896(master)-dirty: Wed Jan  6 16:38:14 UTC 2016
    [Javascript required]:/var/tmp/obj/mips.mips64/home/sson/src/cheribsd/sys/CHERI_MALTA64 mips
gcc version 4.2.1 20070831 patched [FreeBSD]
Preloaded elf kernel "kernel" at 0xffffffff806bbb10.
CHERI: compiled for 256-bit capabilities
real memory  = 2147483648 (2097152K bytes)
[...]
FreeBSD/mips (cheri) (ttyu0)
login:

At this point you can login as the root user (no password) and run the CheriBSD regression tests: 

login: root
[...]
root@:~ # cheritest -a
[...]
TEST: test_string_memmove_c: Test explicit capability memmove
PASS: test_string_memmove_c
SUMMARY: passed 142 failed 1 (1 expected)
root@:~ # cheriabitest -a
[...]
TEST: test_string_memmove_c: Test explicit capability memmove
PASS: test_string_memmove_c
SUMMARY: passed 142 failed 1 (1 expected)
root@:~ #

Note that Qemu can be exited using the key sequence "control-a x" at any time.

Instruction, Register and Memory Tracing

Qemu-Cheri also has support for simple instruction, register and memory tracing. This is very useful for debugging and can be turned on (and off) by the command-line when Qemu is started or via the Qemu monitor. Note that turning on instruction tracing will add a lot of overhead to the Qemu emulation and can generate very large trace log files.

In addition to the disassembled instructions the trace includes changes to registers and memory. Here is a small sample of what the trace log looks like (e.g. start of an exception handler): 

0xffffffff80000194:  csetdefault        c30
    Write C00|v:1 s:0 p:7fffffff b:0000000000000000 l:ffffffffffffffff
             |o:0000000000000000 t:0
[...]
0xffffffff8051d0ec:  sd a3,296(k1)
    Memory Write [c000000000143588] = 000000016004f5a0
0xffffffff8051d0f0:  cgetcause  k0
    Write k0 = 00000000000006ff
0xffffffff8051d0f4:  daddiu     t4,k1,608
    Write t4 = c0000000001436c0
0xffffffff8051d0f8:  csc        c28,t4,192(c30)
    Cap Memory Write [c0000000001436c0] = v:1 tps:00000000ffff00fa
    c:0000000000000000 b:0000000000000000 l:0000010000000000

Starting Instruction Tracing on Start Up

To start instruction tracing on start up add "-D <logfile> -d instr" to the Qemu command-line. For example: 

$ qemu-system-cheri -M malta -kernel ./kernel -nographic -hda ./disk.img -m 2048 -D /var/tmp/instr.log -d instr

Starting and Stopping Instruction Tracing via CheriBSD command

CheriBSD also has a tool /usr/bin/qtrace that can be used to toggle QEMU tracing. This will log the traces to the file specified using the -D flag on QEMU startup. For example: 

$ qtrace -u exec /bin/true  # trace user-space instructions for cat
$ qtrace exec /bin/true     # trace all instructions for true
$ qtrace exec /bin/true     # trace all instructions for true
$ qtrace start              # turn on instruction tracing
$ qtrace stop               # turn off instruction tracing

Starting and Stopping Instruction Tracing via Qemu Monitor

Instruction tracing can also be started and stopped using the Qemu Monitor. To do this toggle into the Qemu Monitor using the "control-a c" key sequence. At the Qemu Monitor prompt, to start instruction tracing, do: 

(qemu) logfile /var/tmp/instr.log
(qemu) log instr

Toggle back to the console with another "control-a c" key sequence.

To stop instruction tracing using the Qemu Monitor: 

(qemu) log none

qemu-system-cheri also can create instruction traces that are compatible with the libcheritrace format that is used by the tracedump utility and CheriVis. To generate libcheritrace instruction traces use '-d cvtrace' instead of '-d instr'. For example: 

$ qemu-system-cheri -M malta -kernel ./kernel -nographic -hda ./disk.img -m 2048 -D /var/tmp/cvtrace.bin -d cvtrace
[...]
$ tracedump -t /var/tmp/cvtrace.bin
[...]