SOSP 2023 CHERI Exercises

Presenters: Jonathan Woodruff (University of Cambridge), Paul Metzger (University of Cambridge)

These exercises are based on the CHERI Exercises repository. This repository contains a series of skills development and adversarial exercises for CHERI, specifically aimed at the CHERI-RISC-V implementation. The original exercise text was adapted by Paul Metzger (University of Cambridge) for this tutorial.

Original authors:
Robert N. M. Watson (University of Cambridge), Brooks Davis (SRI International), Wes Filardo (Microsoft Research), Jessica Clarke (University of Cambridge) and John Baldwin (Ararat River Consulting).

Acknowledgements

The authors gratefully acknowledge Reuben Broadfoot, Lawrence Esswood, Brett Gutstein, Joe Kiniry, Alex Richardson, Austin Roach, and Daniel Zimmerman for their feedback and support in developing these exercises.

Some portions of this document remain a work-in-progress. Feedback and contributions are welcomed. Please see our GitHub Repository for the source code and an issue tracker.

The test program is written in conventional C, and can be compiled to RISC-V or CHERI-RISC-V targets:

  1. Compile print-pointer.c for the baseline architecture with the print-pointer-baseline make target.

print-pointer.c:

/*
 * SPDX-License-Identifier: BSD-2-Clause-DARPA-SSITH-ECATS-HR0011-18-C-0016
 * Copyright (c) 2020 SRI International
 */
#include <stdio.h>

int
main(void)
{
    printf("size of pointer: %zu\n", sizeof(void *));
    /* XXX: ideally we'd use ptraddr_t below */
    printf("size of address: %zu\n", sizeof(size_t));

    return (0);
}

  1. Run the binary.
  2. Compile print-pointer.c for the CHERI-aware architecture with the print-pointer-cheri make target.
  3. Run the binary: it should print a pointer size of 16 and address size of 8.
  4. Inspect both binaries with llvm-readelf -h and compare the Flags rows with each other.

The second test program is written in CHERI C:

  1. Compile include print-capability.c for the CHERI-aware architecture with the print-capability make target.
/*
 * SPDX-License-Identifier: BSD-2-Clause-DARPA-SSITH-ECATS-HR0011-18-C-0016
 * Copyright (c) 2020 SRI International
 */
#include <stdio.h>
#include <cheriintrin.h>

int
main(void)
{
    int i;
    char *c;
    void *cap_to_int = &i;
    void *cap_to_cap = &c;

    printf("cap to int length: %lu\n", cheri_length_get(cap_to_int));
    printf("cap to cap length: %lu\n", cheri_length_get(cap_to_cap));

    return (0);
}

  1. Run the binary: note how the length of the capability depends on the size of the type it points to.

Answers - Compile and run RISC-V and CHERI-RISC-V programs

This exercise explores the difference in size between addresses and pointers, drawing attention to the pointer-focused nature of CHERI memory protection.

  1. Expected output:
# ./print-pointer-riscv
size of pointer: 8
size of address: 8
  1. Expected output:
# ./print-pointer-cheri
size of pointer: 16
size of address: 8
  1. Expected output:
# ./print-capability
cap to int length: 4
cap to cap length: 16

Demonstrate CHERI Tag Protection

This exercise demonstrates CHERI's capability provenance tags, in particular by showing that capabilities and their constituent bytes are subtly different things!

  1. Compile corrupt-pointer.c for the baseline architecture with the corrupt-pointer-baseline make target and for the CHERI-aware architecture with the corrupt-pointer-cheri target.

  2. Run both programs and observe the output.

  3. Use gdb to inspect the SIGPROT thrown to the CHERI program.

    Print out the pseudoregister $_siginfo. si_signo 34 is SIGPROT, a new signal introduced for conveying CHERI traps to user programs. The si_code values for SIGPROT signals are defined as the various PROT_CHERI_* values in <sys/signal.h> (which can be found in /usr/include in a CheriBSD system).

  4. Examine the disassembly of the construction of q,

    uint8_t *q = (uint8_t*)(((uintptr_t)p.ptr) & ~0xFF);
    

    and the byte-wise mutation of p.ptr to construct r,

    p.bytes[0] = 0;
    uint8_t *r = p.ptr;
    

    in both baseline and CHERI-enabled programs.

    What stands out?

  5. Given that q and r appear to have identical byte representation in memory, why does the CHERI version crash when dereferencing r?

Source

corrupt-pointer.c

/*
 * SPDX-License-Identifier: BSD-2-Clause
 * Copyright (c) 2022 Microsoft Corporation
 */
#include <assert.h>
#include <stdint.h>
#include <stdio.h>

#ifdef __CHERI_PURE_CAPABILITY__
#define PRINTF_PTR "#p"
#else
#define PRINTF_PTR "p"
#endif

int
main(void)
{
    char buf[0x1FF];

    volatile union {
        char *ptr;
        char bytes[sizeof(char*)];
    } p;

    for (size_t i = 0; i < sizeof(buf); i++) {
        buf[i] = i;
    }
    p.ptr = &buf[0x10F];

    printf("buf=%" PRINTF_PTR " &p=%" PRINTF_PTR "\n", buf, &p);
    printf("p.ptr=%" PRINTF_PTR " (0x%zx into buf) *p.ptr=%02x\n",
        p.ptr, p.ptr - buf, *p.ptr);

    /* One way to align the address down */
    char *q = (char*)(((uintptr_t)p.ptr) & ~0xFF);
    printf("q=%" PRINTF_PTR " (0x%zx into buf)\n", q, q - buf);

    printf("*q=%02x\n", *q);

    /* Maybe another, assuming a little-endian machine. */
    p.bytes[0] = 0;
    char *r = p.ptr;

    printf("r=%" PRINTF_PTR " (0x%zx)\n", r, r - buf);
    printf("*r=%02x\n", *r);

    return 0;
}

Courseware

This exercise has presentation materials available.

Answers

  1. Example output for the baseline program:

    buf=0x8085ba59 &p=0x8085ba50
    p.ptr=0x8085bb68 (0x10f into buf) *p.ptr=0f
    q=0x8085bb00 (0xa7 into buf)
    *q=a7
    r=0x8085bb00 (0xa7)
    *r=a7
    

    And for the CHERI-enabled program:

    buf=0x3fffdffd71 [rwRW,0x3fffdffd71-0x3fffdfff70] &p=0x3fffdffd60 [rwRW,0x3fffdffd60-0x3fffdffd70]
    p.ptr=0x3fffdffe80 [rwRW,0x3fffdffd71-0x3fffdfff70] (0x10f into buf) *p.ptr=0f
    q=0x3fffdffe00 [rwRW,0x3fffdffd71-0x3fffdfff70] (0x8f into buf)
    *q=8f
    r=0x3fffdffe00 [rwRW,0x3fffdffd71-0x3fffdfff70] (invalid) (0x8f)
    In-address space security exception
    
  2. gdb should report something like

    Program received signal SIGPROT, CHERI protection violation
    Capability tag fault caused by register cs1.
    main () at ./src/exercises/cheri-tags/corrupt-pointer.c:45
    45      ./src/exercises/cheri-tags/corrupt-pointer.c: No such file or directory.
    
    Thread 1 (LWP 100057 of process 1231):
    #0  main () at ./src/exercises/cheri-tags/corrupt-pointer.c:45
    

    We can ask gdb to print out the faulting instruction:

    (gdb) x/i $pcc
    => 0x101d84 <main+244>: clbu    a1,0(cs1)
    

    We can also ask gdb for more information about the signal we received:

    (gdb) p $_siginfo
    $1 = {si_signo = 34, si_errno = 0, si_code = 2, si_pid = 0, si_uid = 0,
     si_status = 0,
     si_addr = 0x101d84 <main+244> [rxR,0x100000-0x104120] (invalid), si_value = {
       sival_int = 0, sival_ptr = 0x0}, _reason = {_fault = {si_trapno = 28,
         si_capreg = 9}, _timer = {si_timerid = 28, si_overrun = 9}, _mesgq = {
         si_mqd = 28}, _poll = {si_band = 38654705692}, __spare__ = {
         __spare1__ = 38654705692, __spare2__ = {0, 0, 0, 0, 0, 0, 0}}}}
    

    As said, si_signo = 34 is SIGPROT, for which si_code = 2 is PROT_CHERI_TAG, indicating a missing (clear) tag as an input to a capability instruction. gdb in fact does this decoding for you, in the reported line Capability tag fault caused by register cs1. It will be helpful to look for similar reports associated with SIGPROTs throughout this book.

  3. Constructing r is very similar on the two targets, differing only by the use of integer- or capability-based memory instructions:

    Baseline CHERI
    Store sb zero, 0(sp) csb zero, 32(csp)
    Load ld s0, 0(sp) clc cs1, 32(csp)

    The significant difference is in the construction of q. On the baseline architecture, it is a direct bitwise and of a pointer loaded from memory:

    ld   a0, 0(sp)
    andi s0, a0, -256
    

    On CHERI, on the other hand, the program makes explicit use of capability manipulation instructions to...

    Instruction Action
    clc ca0, 32(csp) Load the capability from memory
    cgetaddr a1, ca0 Extract its address field to a register
    andi a1, a1, -256 Perform the mask operation
    csetaddr cs1, ca0, a1 Update the address field

    This longer instruction sequence serves to prove to the processor that the resulting capability (in cs1) was constructed using valid transformations. In particular, the csetaddr allows the processor to check that the combination of the old capability (in ca0) and the new address (in a1) remains representable.

  4. While the in-memory, byte representation of q and r are identical, r has been manipulated as bytes rather than as a capability and so has had its tag zeroed. (Specifically, the csb zero, 32(csp) instruction cleared the tag associated with the 16-byte granule pointed to by 32(csp); the subsequent clc transferred this zero tag to cs1.)

Exercise an inter-stack-object buffer overflow

This exercise demonstrates an inter-object buffer overflow on baseline and CHERI-enabled architectures, and asks you to characterize and fix the bug detected by CHERI bounds enforcement. It also asks you to use GDB for debugging purposes.

This example uses two stack objects to demonstrate the overflow. We will be able to see the CHERI C compiler generate code to apply spatial bounds on the capability used for the buffer pointer we pass around.

  1. Compile buffer-overflow-stack.c for the baseline architecture with the buffer-overflow-stack-baseline make target and for the CHERI-aware architecture with the buffer-overflow-stack-cheri target.

  2. Run both programs and observe their outputs.

  3. Using GDB on the core dump (or run the CHERI program under gdb): Why has the CHERI program failed?

  4. Compare and contrast the disassembly of the baseline and CHERI programs. In particular, focus on the write_buf function and main's call to it and the information flow leading up to it.

Source

buffer-overflow-stack.c

/*
 * SPDX-License-Identifier: BSD-2-Clause
 * Copyright (c) 2022 Microsoft Corporation
 */
#include <assert.h>
#include <stddef.h>
#include <stdio.h>

#pragma weak write_buf
void
write_buf(char *buf, size_t ix)
{
    buf[ix] = 'b';
}

int
main(void)
{
    char upper[0x10];
    char lower[0x10];

    printf("upper = %p, lower = %p, diff = %zx\n",
        upper, lower, (size_t)(upper - lower));

    /* Assert that these get placed how we expect */
    assert((ptraddr_t)upper == (ptraddr_t)&lower[sizeof(lower)]);

    upper[0] = 'a';
    printf("upper[0] = %c\n", upper[0]);

    write_buf(lower, sizeof(lower));

    printf("upper[0] = %c\n", upper[0]);

    return 0;
}

Courseware

This exercise has presentation materials available.

Answers - Exercise an inter-stack-object buffer overflow

  1. Expected output:

    # ./buffer-overflow-stack-baseline
    upper = 0x80d879d0, lower = 0x80d879c0, diff = 10
    upper[0] = a
    upper[0] = b
    # ./buffer-overflow-stack-cheri
    upper = 0x3fffdfff50, lower = 0x3fffdfff40, diff = 10
    upper[0] = a
    In-address space security exception
    
  2. An example session of gdb -ex run ./buffer-overflow-stack-cheri on CHERI-RISC-V:

    Reading symbols from ./buffer-overflow-stack-cheri...
    Starting program: /mnt/buffer-overflow-stack-cheri
    upper = 0x3fffdfff50, lower = 0x3fffdfff40, diff = 10
    upper[0] = a
    
    Program received signal SIGPROT, CHERI protection violation
    Capability bounds fault caused by register ca0.
    0x0000000000101cf0 in write_buf (buf=<optimized out>, ix=<optimized out>) at buffer-overflow-stack.c:13
    13              buf[ix] = 'b';
    
    Thread 1 (LWP 100055 of process 829):
    #0  0x0000000000101cf0 in write_buf (buf=<optimized out>, ix=<optimized out>) at buffer-overflow-stack.c:13
    #1  0x0000000000101d7a in main () at buffer-overflow-stack.c:31
    (gdb) disass
    Dump of assembler code for function write_buf:
       0x0000000000101ce8 <+0>:     cincoffset      ca0,ca0,a1
       0x0000000000101cec <+4>:     li      a1,98
    => 0x0000000000101cf0 <+8>:     csb     a1,0(ca0)
       0x0000000000101cf4 <+12>:    cret
    End of assembler dump.
    

    Asking gdb about the registers with info registers and focusing on the ones involved here, we see

    a0             0x3fffdfff50     274875809616
    a1             0x62     98
    
    ca0            0xd17d000007d5bf440000003fffdfff50       0x3fffdfff50 [rwRW,0x3fffdfff40-0x3fffdfff50]
    ca1            0x62     0x62
    

    The capability in ca0, which is a pointer into the lower buffer, has been taken beyond the end of the allocation, as out of bounds store has been attempted (Capability bounds fault).

    But where did those bounds originate? Heading up a stack frame and disassembling, we see (eliding irrelevant instructions):

    (gdb) up
    #1  0x0000000000101d7a in main () at buffer-overflow-stack.c:31
    31              write_buf(lower, sizeof(lower));
    (gdb) disass
    Dump of assembler code for function main:
       0x0000000000101cf8 <+0>:     cincoffset      csp,csp,-144
    
       0x0000000000101d14 <+28>:    cincoffset      ca0,csp,48
       0x0000000000101d18 <+32>:    csetbounds      cs0,ca0,16
    
       0x0000000000101d6c <+116>:   li      a1,16
       0x0000000000101d6e <+118>:   cmove   ca0,cs0
       0x0000000000101d72 <+122>:   auipcc  cra,0x0
       0x0000000000101d76 <+126>:   cjalr   -138(cra)
    => 0x0000000000101d7a <+130>:   clbu    a0,0(cs1)
    

    The compiler has arranged for main to allocate 144 bytes on the stack by decrementing the capability stack pointer register (csp) by 144 bytes. Further, the compiler has placed lower 48 bytes up into that allocation: ca0 is made to point at its lowest address and then the pointer to lower is materialized in cs0 by bounding the capability in ca0 to be 16 (sizeof(lower)) bytes long. This capability is passed to write_buf in ca0.

  3. The code for write_buf function is only slightly changed. On RISC-V it compiles to

    <write_buf>:
     add     a0, a0, a1
     addi    a1, zero, 98
     sb      a1, 0(a0)
     ret
    

    while on CHERI-RISC-V, it is

    <write_buf>:
     cincoffset      ca0, ca0, a1
     addi    a1, zero, 98
     csb     a1, 0(ca0)
     cret
    

    In both cases, it amounts to displacing the pointer passed in a0 (resp. ca0) by the offset passed in a1 and then performing a store-byte instruction before returning. In the baseline case, the store-byte takes an integer address for its store, while in the CHERI case, the store-byte takes a capability authorizing the store. There are no conditional branches or overt bounds checks in the CHERI instruction stream; rather, the csb instruction itself enforces the requirement for authority to write to memory, in the shape of a valid, in-bounds capability.

    We have already seen the CHERI program's call site to write_buf in main, and the derivation of the capability to the lower buffer, above. In the baseline version, the corresponding instructions are shown as

    (gdb) disass main
    Dump of assembler code for function main:
       0x0000000000011b44 <+0>:       addi    sp,sp,-48
    
       0x0000000000011b8a <+70>:      mv      a0,sp
       0x0000000000011b8c <+72>:      li      a1,16
       0x0000000000011b8e <+74>:      auipc   ra,0x0
       0x0000000000011b92 <+78>:      jalr    -86(ra) # 0x11b38 <write_buf>
    

    Here, the compiler has reserved only 48 bytes of stack space and has placed the lower buffer at the lowest bytes of this reservation. Thus, to pass a pointer to the lower buffer to write_buf, the program simply copies the stack pointer register (an integer register, holding an address) to the argument register a0. The subsequent address arithmetic derives an address out of bounds, clobbering a byte of the upper register.

Explore Subobject Bounds

In the CheriABI run-time environment, bounds are typically associated with memory allocations rather than C types. For example, if a heap memory allocation is made for 1024 bytes, and the structure within it is 768 bytes, then the bounds associated with a pointer will be for the allocation size rather than the structure size.

Subobject Overflows

With subobject bounds, enforcement occurs on C-language objects within allocations. This exercise is similar to earlier buffer-overflow exercises, but is for such an intra-object overflow. In our example, we consider an array within another structure, overflowing onto an integer in the same allocation.

  1. Compile buffer-overflow-subobject.c with the baseline make target buffer-overflow-subobject-baseline, and with the CHERI-enabled target buffer-overflow-subobject-cheri.

  2. As in the prior exercises, run the binaries.

  3. Explore why the CHERI binary didn't fail. Run buffer-overflow-subobject-cheri under gdb and examine the bounds of the buffer argument to fill_buf(). To what do they correspond?

  4. Compile the buffer-overflow-subobject.c with the buffer-overflow-subobject-cheri-subobject-safe make target. This target is CHERI-enabled and adds the compiler flags -Xclang -cheri-bounds=subobject-safe.

  5. Run the program to demonstrate that the buffer overflow is now caught.

  6. Run the program under gdb and examine the bounds again. What has changed?

Deliberately Using Larger Bounds

Operations like &object->field that move from super-object to sub-object are very natural in C, and there is no similarly concise syntax for the reverse operation. Nevertheless, C programs occasionally do make use of containerof constructs to do exactly that: derive a pointer to the superobject given a pointer to a subobject within.

A common example is intrusive linked lists, as found, for example, in the BSD <sys/queue.h>. subobject-list.c is an extremely minimal example of such, which we will use to explore the behavior of CHERI C here.

  1. Compile subobject-list.c with the CHERI-enabled make target subobject-list-cheri and run it.

  2. What is the length (limit - base) for capabilities to...

    • the sentinel node (&l)
    • a next pointer (ile_next) to a non-sentinel element
    • a previous-next pointer (ile_prevnp) to a non-sentinel element
  3. Compile this program now with the subobject-list-cheri-subobject-safe target, which adds -Xclang -cheri-bounds=subobject-safe and run the result. What happens and why?

  4. The CheriBSD system headers have been extended so that examples like this which use the <sys/cdefs.h> definition of __containerof (or things built atop that) will trip static assertions. Try compiling again with the subobject-list-cheri-subobject-safe-use-cdefs target and observe what the compiler tells you. This target adds the flags -Xclang -cheri-bounds=subobject-safe -DUSE_CDEFS_CONTAINEROF.

  5. Make the suggested change, marking struct ilist_elem as __subobject_use_container_bounds and recompile once again (with the same flags as just above). Run the resulting program and observe its output. Which bounds have not been narrowed? Which have? Why is that OK?

Source Files

Subobject Overflows

buffer-overflow-subobject.c

/*
 * SPDX-License-Identifier: BSD-2-Clause-DARPA-SSITH-ECATS-HR0011-18-C-0016
 * Copyright (c) 2020 SRI International
 */
#include <stdio.h>

struct buf {
    char buffer[128];
    int i;
} b;

#pragma weak fill_buf
void
fill_buf(char *buf, size_t len)
{
    for (size_t i = 0; i <= len; i++)
        buf[i] = 'b';
}

int
main(void)
{
    b.i = 'c';
    printf("b.i = %c\n", b.i);

    fill_buf(b.buffer, sizeof(b.buffer));

    printf("b.i = %c\n", b.i);

    return 0;
}

#include "asserts.inc"

asserts.inc

/*
 * SPDX-License-Identifier: BSD-2-Clause-DARPA-SSITH-ECATS-HR0011-18-C-0016
 * Copyright (c) 2020 SRI International
 */
#include <stddef.h>

_Static_assert(sizeof(b.buffer) == offsetof(struct buf, i),
    "There must be no padding in struct buf between buffer and i members");

Deliberately Using Larger Bounds

subobject-list.c

/*
 * SPDX-License-Identifier: BSD-2-Clause
 * Copyright (c) 2022 Microsoft Corporation
 *
 * This exercise investigates a circular doubly-linked list with sentinels.
 */
#include <stdio.h>

/*
 * A list element is an intrusive structure (subobject) with a pointer to the
 * next list element and a pointer to the previous node's next pointer.  In the
 * case of an empty list, ile_prevnp points to ile_next.
 */
struct ilist_elem {
    struct ilist_elem **ile_prevnp;
    struct ilist_elem *ile_next;
}; // __subobject_use_container_bounds;

static void
ilist_init_sentinel(struct ilist_elem *s) {
    s->ile_next = s;
    s->ile_prevnp = &s->ile_next;
}

static void
ilist_insert_after(struct ilist_elem *p, struct ilist_elem *n) {
    n->ile_next = p->ile_next;
    p->ile_next = n;
    n->ile_next->ile_prevnp = &n->ile_next;
    n->ile_prevnp = &p->ile_next;
}

static void
ilist_remove(struct ilist_elem *e) {
    e->ile_next->ile_prevnp = e->ile_prevnp;
    *(e->ile_prevnp) = e->ile_next;
}

#define ILIST_FOREACH(h, c) \
    for(c = (h)->ile_next; c != (h); c = c->ile_next)

#ifdef USE_CDEFS_CONTAINEROF
#define ILIST_CONTAINER(elem, type, field) \
    (((elem) == NULL) ? ((type *)NULL) : __containerof((elem), type, field))
#else
#define ILIST_CONTAINER(elem, type, field) \
    (((elem) == NULL) ? ((type *)NULL) : \
        __DEQUALIFY(type*, (const volatile char *)(elem) - \
            __offsetof(type, field)))
#endif

struct obj {
    int val;
    struct ilist_elem ilist; // __subobject_use_container_bounds;
};

struct ilist_elem l; /* Sentinel element serves as list head */
struct obj obj1 = {1, {}};
struct obj obj2 = {2, {}};
struct obj obj3 = {3, {}};

int
main() {
    struct ilist_elem *cursor;

    ilist_init_sentinel(&l);
    ilist_insert_after(&l, &obj2.ilist);
    ilist_insert_after(&obj2.ilist, &obj3.ilist);
    ilist_insert_after(&l, &obj1.ilist);
    ilist_remove(&obj2.ilist);

    printf("Traversing list=%#p first=%#p lastnp=%#p\n",
        &l, l.ile_next, l.ile_prevnp);
    ILIST_FOREACH(&l, cursor) {
        struct obj *co = ILIST_CONTAINER(cursor, struct obj, ilist);
        printf(" Ilist cursor=%#p\n  next=%#p\n  prevnp=%#p\n",
            cursor, cursor->ile_next, cursor->ile_prevnp);
        printf("  val field at %#p\n",
            /*
             * This ugly bit of syntax is unfortunate, but avoids
             * a subobject-bounds-induced trap that isn't the first
             * one you should think about.  I'm sorry.  Just pretend
             * this says "&co->val" and, for extra credit, later,
             * explain why it isn't spelled like that.
             */
            ((char *)co) + __offsetof(struct obj, val));
    }

    printf("Traversing list again, accessing superobject field...\n");
    ILIST_FOREACH(&l, cursor) {
        struct obj *co = ILIST_CONTAINER(cursor, struct obj, ilist);
        printf(" Ilist cursor=%#p value=%d (at %#p)\n", cursor,
            co->val, &co->val);
    }
}

Courseware

This exercise has presentation materials available.

Answers - Explore Subobject Bounds

Exercise a subobject buffer overflow

This exercise demonstrates subobject bounds with an array in a structure.

  1. Expected output:

    # ./buffer-overflow-subobject-riscv
    b.i = c
    b.i = b
    # ./buffer-overflow-subobject-cheri
    b.i = c
    b.i = b
    
  2. Example session:

    (gdb) b fill_buf
    Breakpoint 1 at 0x1bae: file buffer-overflow-subobject.c, line 17.
    (gdb) r
    Starting program: /root/buffer-overflow-subobject-cheri
    b.i = c
    
    Breakpoint 1, fill_buf (buf=0x103f50 <b> [rwRW,0x103f50-0x103fd4] "", len=128)
        at buffer-overflow-subobject.c:17
    17                      buf[i] = 'b';
    

    The bounds are 132 bytes corresponding to the size of the underlying object.

  3. Expected output:

    # ./buffer-overflow-subobject-cheri
    b.i = c
    In-address space security exception (core dumped)
    
  4. Example session:

    (gdb) b fill_buf
    Breakpoint 1 at 0x1bae: file buffer-overflow-subobject.c, line 17.
    (gdb) r
    Starting program: /root/buffer-overflow-subobject-cheri
    b.i = c
    
    Breakpoint 1, fill_buf (buf=0x103f50 <b> [rwRW,0x103f50-0x103fd0] "", len=128) at buffer-overflow-subobject.c:17
    17                      buf[i] = 'b';
    

    The pointer to the buffer is now bounded to the array rather than the object.

    Investigating further will reveal that the compiler has inserted a bounds-setting instruction prior to the call to fill_buf in main, that is, when the pointer to b.buffer is materialized.

    (gdb) up
    #1  0x0000000000101c0c in main () at buffer-overflow-subobject.c:26
    26              fill_buf(b.buffer, sizeof(b.buffer));
    (gdb) disassemble
    Dump of assembler code for function main:
       0x0000000000101bc0 <+0>:     cincoffset      csp,csp,-64
    ...
       0x0000000000101bfc <+60>:    csetbounds      ca0,cs1,128
       0x0000000000101c00 <+64>:    li      a1,128
       0x0000000000101c04 <+68>:    auipcc  cra,0x0
       0x0000000000101c08 <+72>:    cjalr   -92(cra)
    => 0x0000000000101c0c <+76>:    clw     a0,128(cs1)
    

Deliberately Using Larger Bounds

  1. Example output:

    Traversing list=0x104320 [rwRW,0x104320-0x104340] first=0x1040e0 [rwRW,0x1040d0-0x104100] lastnp=0x104150 [rwRW,0x104130-0x104160]
     Ilist cursor=0x1040e0 [rwRW,0x1040d0-0x104100]
      next=0x104140 [rwRW,0x104130-0x104160]
      prevnp=0x104330 [rwRW,0x104320-0x104340]
      val field at 0x1040d0 [rwRW,0x1040d0-0x104100]
     Ilist cursor=0x104140 [rwRW,0x104130-0x104160]
      next=0x104320 [rwRW,0x104320-0x104340]
      prevnp=0x1040f0 [rwRW,0x1040d0-0x104100]
      val field at 0x104130 [rwRW,0x104130-0x104160]
    Traversing list again, accessing superobject field...
     Ilist cursor=0x1040e0 [rwRW,0x1040d0-0x104100] value=1 (at 0x1040d0 [rwRW,0x1040d0-0x104100])
     Ilist cursor=0x104140 [rwRW,0x104130-0x104160] value=3 (at 0x104130 [rwRW,0x104130-0x104160])
    
  2. In turn:

    • All capabilities referencing the sentinel or its fields (including &l->ile_next) have length 0x20, corresponding to sizeof(struct ilist_elem).

    • The next pointers in the sentinel, 0x1040d0 [rwRW,0x1040c0-0x1040f0], and in the first list element, 0x104130 [rwRW,0x104120-0x104150], have legth 0x30, corresponding to sizeof(struct obj).

    • The previous-next pointers in the sentinel, 0x104140 [rwRW,0x104120-0x104150] and in the last list element, 0x1040e0 [rwRW,0x1040c0-0x1040f0] also have length 0x30.

  3. Example output:

    Traversing list=0x104350 [rwRW,0x104350-0x104370] first=0x104120 [rwRW,0x104120-0x104140] lastnp=0x104190 [rwRW,0x104190-0x1041a0]
     Ilist cursor=0x104120 [rwRW,0x104120-0x104140]
      next=0x104180 [rwRW,0x104180-0x1041a0]
      prevnp=0x104360 [rwRW,0x104360-0x104370]
      val field at 0x104110 [rwRW,0x104120-0x104140]
     Ilist cursor=0x104180 [rwRW,0x104180-0x1041a0]
      next=0x104350 [rwRW,0x104350-0x104370]
      prevnp=0x104130 [rwRW,0x104130-0x104140]
      val field at 0x104170 [rwRW,0x104180-0x1041a0]
    Traversing list again, accessing superobject field...
    In-address space security exception
    

    Notice the line val field at 0x104110 [rwRW,0x104120-0x104140]. This is out of bounds!

    The compiler has taken our use of &obj1.ilist as an argument to ilist_insert_after as license to narrow the bounds to just the subobject. Indeed, the length of all ile_next pointers is now 0x20. Further, all ile_lastnp pointers now have length 0x10, the size of just the capability they point to!

  4. The compiler will emit a pair of warnings about the uses of ILIST_CONTAINER:

    ./subobject-list.c:75:20: error: static_assert failed due to requirement '__builtin_marked_no_subobject_bounds(struct obj) || __builtin_marked_no_subobject_bounds(struct ilist_elem)' "this type is unsafe for use in containerof() with sub-objectbounds. Please mark the member/type with __subobject_use_container_bounds"
    
  5. We can take the compiler's advice in at least two ways:

    1. We could mark the struct ilist_elem type itself:

      struct ilist_elem {
        struct ilist_elem **ile_prevnp;
        struct ilist_elem *ile_next;
      } __subobject_use_container_bounds;
      

      When we run the program now, we will find that we are largely back in the case where no sub-object bounds were applied: pointers to the sentinel have length 0x20 and pointers to list elements have length 0x30. However, the &co->val pointers are still bounded:

      Traversing list=0x104300 [rwRW,0x104300-0x104320] first=0x1040d0 [rwRW,0x1040c0-0x1040f0] lastnp=0x104140 [rwRW,0x104120-0x104150]
       Ilist cursor=0x1040d0 [rwRW,0x1040c0-0x1040f0]
        next=0x104130 [rwRW,0x104120-0x104150]
        prevnp=0x104310 [rwRW,0x104300-0x104320]
        val field at 0x1040c0 [rwRW,0x1040c0-0x1040f0]
       Ilist cursor=0x104130 [rwRW,0x104120-0x104150]
        next=0x104300 [rwRW,0x104300-0x104320]
        prevnp=0x1040e0 [rwRW,0x1040c0-0x1040f0]
        val field at 0x104120 [rwRW,0x104120-0x104150]
      Traversing list again, accessing superobject field...
       Ilist cursor=0x1040d0 [rwRW,0x1040c0-0x1040f0] value=1 (at 0x1040c0 [rwRW,0x1040c0-0x1040c4])
       Ilist cursor=0x104130 [rwRW,0x104120-0x104150] value=3 (at 0x104120 [rwRW,0x104120-0x104124])
      
    2. We can mark the ilist field of struct obj:

      struct obj {
        int val;
        struct ilist_elem ilist __subobject_use_container_bounds;
      };
      

      In this case, we find that the ile_next pointers are offset and not bounded, while the ile_prevnp pointers are tightly bounded:

      Traversing list=0x104340 [rwRW,0x104340-0x104360] first=0x104110 [rwRW,0x104100-0x104130] lastnp=0x104180 [rwRW,0x104180-0x104190]
       Ilist cursor=0x104110 [rwRW,0x104100-0x104130]
        next=0x104170 [rwRW,0x104160-0x104190]
        prevnp=0x104350 [rwRW,0x104350-0x104360]
        val field at 0x104100 [rwRW,0x104100-0x104130]
       Ilist cursor=0x104170 [rwRW,0x104160-0x104190]
        next=0x104340 [rwRW,0x104340-0x104360]
        prevnp=0x104120 [rwRW,0x104120-0x104130]
        val field at 0x104160 [rwRW,0x104160-0x104190]
      Traversing list again, accessing superobject field...
       Ilist cursor=0x104110 [rwRW,0x104100-0x104130] value=1 (at 0x104100 [rwRW,0x104100-0x104104])
       Ilist cursor=0x104170 [rwRW,0x104160-0x104190] value=3 (at 0x104160 [rwRW,0x104160-0x104164])
      

Exercise heap overflows

This exercise demonstrates inter-object heap buffer overflows on baseline and CHERI-enabled architectures, and asks you to characterize and fix the bug detected by CHERI bounds enforcement.

  1. Compile buffer-overflow-heap.c for the baseline architecture with the buffer-overflow-heap-baseline make target and for the CHERI-aware architecture with the buffer-overflow-heap-cheri target.

  2. Run both versions, passing 0x20 as the (sole) command line argument. Observe that the CHERI version crashes with "In-address space security exception".

  3. Run the CHERI version, again with 0x20, under gdb and examine the crash in more detail. Where must the bounds on the capability implementing b1 have come from?

  4. Run both programs again, but now with 0x1001 as the argument. Draw a picture of the portion of the heap containing (the end of) b1 and (the start of) b2. There are, in some sense, two different ends of b1 in the baseline program and three in the CHERI program! What are they and how do they arise?

  5. While this program does crash on CHERI, again of a bounds violation, this happens slightly later than might be expected looking at the program's source. In particular, this program actually commits two out of bounds stores using the b1 capability. Examine the output carefully and describe, merely in terms of the mechanism, without venturing philosophical, why the first does not trigger a trap.

  6. Now consider the bigger picture. Since CHERI uses compressed capability bounds, what additional steps must be taken, and by whom, to ensure spatial safety of a C program?

Source Files

buffer-overflow-heap.c

/*
 * SPDX-License-Identifier: BSD-2-Clause
 * Copyright (c) 2022 Microsoft Corporation
 */
#include <assert.h>
#include <stddef.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

int
main(int argc, char **argv)
{
    char *b1, *b2;

    assert(argc == 2);
    char *p;
    size_t sz = (size_t)strtoull(argv[1], &p, 0);

    assert(sz > 0);
    assert(sz <= 0x8000);

    b1 = malloc(sz);
    assert(b1 != NULL);

    b2 = malloc(sz);
    assert(b2 != NULL);

#ifdef __CHERI_PURE_CAPABILITY__
    printf("sz=%zx, CRRL(sz)=%zx\n", sz,
        __builtin_cheri_round_representable_length(sz));
    printf("b1=%#p b2=%#p diff=%tx\n", b1, b2, b2 - b1);
#else
    printf("b1=%p b2=%p diff=%tx\n", b1, b2, b2 - b1);
#endif

    /*
     * The default CheriBSD malloc uses "size classes" for allocations.
     * Check that we've landed "nearby".
     */
    assert((ptraddr_t)(b1 + sz) <= (ptraddr_t)b2); 
    assert((ptraddr_t)(b1 + sz + sz/2) > (ptraddr_t)b2); 

    memset(b2, 'B', sz);

    printf("Overflowing by 1\n");
    memset(b1, 'A', sz + 1);

    printf("b2 begins: %.4s\n", b2);


    /* And now let's definitely make trouble */
    const size_t oversz = b2 - b1 + 2 - sz;
    printf("Overflowing by %zx\n", oversz);
    memset(b1 + sz, 'A', oversz);

    printf("b2 begins: %.4s\n", b2);
}

Courseware

This exercise has presentation materials available.

Answers - Exercise heap overflows

  1. Example output:

    # ./buffer-overflow-heap-baseline 0x20
    b1=0x83e82000 b2=0x83e82020 diff=20
    Overflowing by 1
    b2 begins: ABBB
    Overflowing by 2
    b2 begins: AABB
    
    # ./buffer-overflow-heap-cheri 0x20
    sz=20, CRRL(sz)=20
    b1=0x407c7000 [rwRW,0x407c7000-0x407c7020] b2=0x407c7020 [rwRW,0x407c7020-0x407c7040] diff=20
    Overflowing by 1
    In-address space security exception
    
  2. Example session (abridged):

    # gdb -ex run -ex bt --args ./buffer-overflow-heap-cheri 0x20
    
    Starting program: ./buffer-overflow-heap-cheri 0x20
    sz=20, CRRL(sz)=20
    b1=0x407c7000 [rwRW,0x407c7000-0x407c7020] b2=0x407c7020 [rwRW,0x407c7020-0x407c7040] diff=20
    Overflowing by 1
    
    Program received signal SIGPROT, CHERI protection violation
    Capability bounds fault caused by register ca4.
    memset (dst0=0x407c7000 [rwRW,0x407c7000-0x407c7020], c0=65, length=<optimized out>) at /cheri/source/mainline/cheribsd/lib/libc/string/memset.c:131
    131     /cheri/source/mainline/cheribsd/lib/libc/string/memset.c: No such file or directory.
    
    Thread 1 (LWP 100057 of process 960):
    #0  memset (dst0=0x407c7000 [rwRW,0x407c7000-0x407c7020], c0=65, length=<optimized out>) at /cheri/source/mainline/cheribsd/lib/libc/string/memset.c:131
    #1  0x00000000001020d2 in main (argc=<optimized out>, argv=<optimized out>) at ./src/exercises/buffer-overflow-heap/buffer-overflow-heap.c:49
    (gdb) i r ca4
    ca4            0xd17d00000409b00400000000407c7020       0x407c7020 [rwRW,0x407c7000-0x407c7020]
    

    The capability in ca4 is, as expected, a reference to the first allocation (b1). The bounds on this capability must have been imposed by malloc.

  3. Example output:

    # /mnt/buffer-overflow-heap-baseline 0x1001
    b1=0x840ec000 b2=0x840ed400 diff=1400
    Overflowing by 1
    b2 begins: BBBB
    Overflowing by 401
    b2 begins: AABB
    
    # /mnt/buffer-overflow-heap-cheri 0x1001
    sz=1001, CRRL(sz)=1008
    b1=0x407c7000 [rwRW,0x407c7000-0x407c8008] b2=0x407c8400 [rwRW,0x407c8400-0x407c9408] diff=1400
    Overflowing by 1
    Overflowing by 401
    In-address space security exception
    

    Using addresses from the CHERI run, we might draw something that highlighted these key addresses:

    Address Contents
    0x000...00
    0x407c7000 Start of b1
    0x407c8001 Last byte of b1 allocation ("end" #1)
    0x407c8002 Start of CHERI representation padding
    0x407c8007 Last byte of CHERI representation padding ("end" #2)
    0x407c8008 Start of allocator size-class padding
    0x407c83FF Last byte of allocator size-class padding ("end" #3)
    0x407c8400 b2
    0xFFF...FF
  4. The first overflow, by 1 byte, is within bounds due to architectural precision and so, as far as the CPU is concerned, is not an overflow despite writing outside the logical bounds of the b1 allocation.

  5. In order to set bounds large enough to encapsulate large objects, CHERI's compressed capability representation may be able to express only larger bounds than the requested size. (More generally, the base and limits of a capability have increased alignment requirements as they are moved further apart, that is, as the capability length increases. For the capabilities in this example, the bases have strong alignments, of at least 10 bits, due to the allocator's use of size classes.)

    If bounds were simply widened with no additional consideration, then pointers to different objects might come to authorize access to (parts of) each other's memory! In order to ensure that capabilities to distinct C objects do not alias like this, various system components must take CHERI capability compression into consideration:

    • The compiler, for on-stack allocations and address-taken subobjects.
    • The linker, for objects in static storage
    • The heap allocator(s), for objects in dynamic storage

    These concerns do not usually reach "application level" C.

Adapt a C Program to CHERI C

This excercise presents an example C program that includes capability-related issues that might appear as bugs in software initially developed for non-CHERI architectures. The example C program is cat(1) from CheriBSD (and hence FreeBSD) modified to introduce the issues that we want to investigate.

  1. Read Sections 4.2, 4.2.1, 4.2.3 from the CHERI C/C++ Programming Guide. In Section 4.2.1, read only information on the following C-language types: long, uintptr_t and char *,... (pointer types).

  2. Compile cat/cat.c and cat/methods.c for the baseline architecture with the cat-baseline make target and for the CHERI-aware architecture with the cat-cheri target. The compiler should print some warnings when compiling cat-cheri. Save the output to examine the warnings later.

  3. Run both versions to print contents of an arbitrary file (e.g., /etc/hostid), once without any additional flags and once with the -n flag.

  4. Run the CHERI version, again without any additional flags, under gdb and examine the crash in more detail. Set appropriate breakpoints before your program is started.

  5. Get back to the compiler warnings and try to solve a bug that triggered the crash.

  6. Run the CHERI version, again with the -n flag, under gdb and examine the crash in more detail.

  7. Get back to the compiler warnings and try to solve a bug that triggered the crash.

  8. You just analysed two bugs in cat. How are they different and why they trigger crashes in different ways?

Source Files

cat/cat.c

/*-
 * SPDX-License-Identifier: BSD-3-Clause
 *
 * Copyright (c) 1989, 1993
 *	The Regents of the University of California.  All rights reserved.
 *
 * This code is derived from software contributed to Berkeley by
 * Kevin Fall.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 * 3. Neither the name of the University nor the names of its contributors
 *    may be used to endorse or promote products derived from this software
 *    without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 */

#if 0
#ifndef lint
static char const copyright[] =
"@(#) Copyright (c) 1989, 1993\n\
    The Regents of the University of California.  All rights reserved.\n";
#endif /* not lint */
#endif

#ifndef lint
#if 0
static char sccsid[] = "@(#)cat.c	8.2 (Berkeley) 4/27/95";
#endif
#endif /* not lint */
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");

#include <sys/param.h>
#include <sys/stat.h>
#ifndef NO_UDOM_SUPPORT
#include <sys/socket.h>
#include <sys/un.h>
#include <netdb.h>
#endif

#include <ctype.h>
#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <locale.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <wchar.h>
#include <wctype.h>

#include "cat.h"

int bflag, eflag, lflag, nflag, sflag, tflag, vflag;
int rval;
const char *filename;

static void usage(void) __dead2;
static void scanfiles(char *argv[], int verbose);

int
main(int argc, char *argv[])
{
    int ch;
    struct flock stdout_lock;

    setlocale(LC_CTYPE, "");

    while ((ch = getopt(argc, argv, SUPPORTED_FLAGS)) != -1)
        switch (ch) {
        case 'b':
            bflag = nflag = 1;	/* -b implies -n */
            break;
        case 'e':
            eflag = vflag = 1;	/* -e implies -v */
            break;
        case 'l':
            lflag = 1;
            break;
        case 'n':
            nflag = 1;
            break;
        case 's':
            sflag = 1;
            break;
        case 't':
            tflag = vflag = 1;	/* -t implies -v */
            break;
        case 'u':
            setbuf(stdout, NULL);
            break;
        case 'v':
            vflag = 1;
            break;
        default:
            usage();
        }
    argv += optind;

    if (lflag) {
        stdout_lock.l_len = 0;
        stdout_lock.l_start = 0;
        stdout_lock.l_type = F_WRLCK;
        stdout_lock.l_whence = SEEK_SET;
        if (fcntl(STDOUT_FILENO, F_SETLKW, &stdout_lock) == -1)
            err(EXIT_FAILURE, "stdout");
    }

    if (bflag || eflag || nflag || sflag || tflag || vflag)
        scanfiles(argv, 1);
    else
        scanfiles(argv, 0);
    if (fclose(stdout))
        err(1, "stdout");
    exit(rval);
    /* NOTREACHED */
}

static void
usage(void)
{

    fprintf(stderr, "usage: cat [-" SUPPORTED_FLAGS "] [file ...]\n");
    exit(1);
    /* NOTREACHED */
}

static void
scanfiles(char *argv[], int verbose)
{
    int fd, i;
    char *path;
    FILE *fp;

    i = 0;
    fd = -1;
    while ((path = argv[i]) != NULL || i == 0) {
        if (path == NULL || strcmp(path, "-") == 0) {
            filename = "stdin";
            fd = STDIN_FILENO;
        } else {
            filename = path;
            fd = open(path, O_RDONLY);
        }
        if (fd < 0) {
            warn("%s", path);
            rval = 1;
        } else if (verbose) {
            if (fd == STDIN_FILENO)
                do_cat((long)stdin, verbose);
            else {
                fp = fdopen(fd, "r");
                do_cat((long)fp, verbose);
                fclose(fp);
            }
        } else {
            do_cat(fd, verbose);
            if (fd != STDIN_FILENO)
                close(fd);
        }
        if (path == NULL)
            break;
        ++i;
    }
}

cat/cat.h

/*-
 * SPDX-License-Identifier: BSD-3-Clause
 *
 * Copyright (c) 1989, 1993
 *	The Regents of the University of California.  All rights reserved.
 *
 * This code is derived from software contributed to Berkeley by
 * Kevin Fall.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 * 3. Neither the name of the University nor the names of its contributors
 *    may be used to endorse or promote products derived from this software
 *    without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 */

#ifndef	_CAT_H_

/*
 * Memory strategy threshold, in pages: if physmem is larger than this,
 * use a large buffer.
 */
#define	PHYSPAGES_THRESHOLD (32 * 1024)

/* Maximum buffer size in bytes - do not allow it to grow larger than this. */
#define	BUFSIZE_MAX (2 * 1024 * 1024)

/*
 * Small (default) buffer size in bytes. It's inefficient for this to be
 * smaller than MAXPHYS.
 */
#define	BUFSIZE_SMALL (MAXPHYS)

#define SUPPORTED_FLAGS "belnstuv"

void do_cat(long file, int verbose);

#endif /* !_CAT_H_ */

cat/methods.c

/*-
 * SPDX-License-Identifier: BSD-3-Clause
 *
 * Copyright (c) 1989, 1993
 *	The Regents of the University of California.  All rights reserved.
 *
 * This code is derived from software contributed to Berkeley by
 * Kevin Fall.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.
 * 3. Neither the name of the University nor the names of its contributors
 *    may be used to endorse or promote products derived from this software
 *    without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 */

#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");

#include <sys/param.h>
#include <sys/stat.h>
#ifndef NO_UDOM_SUPPORT
#include <sys/socket.h>
#include <sys/un.h>
#include <netdb.h>
#endif

#include <ctype.h>
#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <locale.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <wchar.h>
#include <wctype.h>

#include "cat.h"

typedef uintptr_t ptroff_t;

extern int bflag, eflag, lflag, nflag, sflag, tflag, vflag;
extern int rval;
extern const char *filename;

static ssize_t
write_off(int fildes, const void *buf, ptroff_t off, size_t nbyte)
{

    return (write(fildes, (const void *)(off + (uintptr_t)buf), nbyte));
}

static void
verbose_cat(long file)
{
    FILE *fp;
    int ch, gobble, line, prev;
    wint_t wch;

    fp = (FILE *)file;

    /* Reset EOF condition on stdin. */
    if (fp == stdin && feof(stdin))
        clearerr(stdin);

    line = gobble = 0;
    for (prev = '\n'; (ch = getc(fp)) != EOF; prev = ch) {
        if (prev == '\n') {
            if (sflag) {
                if (ch == '\n') {
                    if (gobble)
                        continue;
                    gobble = 1;
                } else
                    gobble = 0;
            }
            if (nflag) {
                if (!bflag || ch != '\n') {
                    (void)fprintf(stdout, "%6d\t", ++line);
                    if (ferror(stdout))
                        break;
                } else if (eflag) {
                    (void)fprintf(stdout, "%6s\t", "");
                    if (ferror(stdout))
                        break;
                }
            }
        }
        if (ch == '\n') {
            if (eflag && putchar('$') == EOF)
                break;
        } else if (ch == '\t') {
            if (tflag) {
                if (putchar('^') == EOF || putchar('I') == EOF)
                    break;
                continue;
            }
        } else if (vflag) {
            (void)ungetc(ch, fp);
            /*
             * Our getwc(3) doesn't change file position
             * on error.
             */
            if ((wch = getwc(fp)) == WEOF) {
                if (ferror(fp) && errno == EILSEQ) {
                    clearerr(fp);
                    /* Resync attempt. */
#ifdef __FreeBSD__
                    memset(&fp->_mbstate, 0, sizeof(mbstate_t));
#endif
                    if ((ch = getc(fp)) == EOF)
                        break;
                    wch = ch;
                    goto ilseq;
                } else
                    break;
            }
            if (!iswascii(wch) && !iswprint(wch)) {
ilseq:
                if (putchar('M') == EOF || putchar('-') == EOF)
                    break;
                wch = toascii(wch);
            }
            if (iswcntrl(wch)) {
                ch = toascii(wch);
                ch = (ch == '\177') ? '?' : (ch | 0100);
                if (putchar('^') == EOF || putchar(ch) == EOF)
                    break;
                continue;
            }
            if (putwchar(wch) == WEOF)
                break;
            ch = -1;
            continue;
        }
        if (putchar(ch) == EOF)
            break;
    }
    if (ferror(fp)) {
        warn("%s", filename);
        rval = 1;
        clearerr(fp);
    }
    if (ferror(stdout))
        err(1, "stdout");
}

static void
raw_cat(long file)
{
    long pagesize;
    int off, rfd, wfd;
    ssize_t nr, nw;
    static size_t bsize;
    static char *buf = NULL;
    struct stat sbuf;

    rfd = (int)file;

    wfd = fileno(stdout);
    if (buf == NULL) {
        if (fstat(wfd, &sbuf))
            err(1, "stdout");
        if (S_ISREG(sbuf.st_mode)) {
            /* If there's plenty of RAM, use a large copy buffer */
            if (sysconf(_SC_PHYS_PAGES) > PHYSPAGES_THRESHOLD)
                bsize = MIN(BUFSIZE_MAX, MAXPHYS * 8);
            else
                bsize = BUFSIZE_SMALL;
        } else {
            bsize = sbuf.st_blksize;
            pagesize = sysconf(_SC_PAGESIZE);
            if (pagesize > 0)
                bsize = MAX(bsize, (size_t)pagesize);
        }
        if ((buf = malloc(bsize)) == NULL)
            err(1, "malloc() failure of IO buffer");
    }
    while ((nr = read(rfd, buf, bsize)) > 0)
        for (off = 0; nr; nr -= nw, off += nw)
            if ((nw = write_off(wfd, buf, off, (size_t)nr)) < 0)
                err(1, "write(2) failed");
    if (nr < 0) {
        warn("%s", filename);
        rval = 1;
    }
}

void
do_cat(long file, int verbose)
{

    if (verbose) {
        verbose_cat(file);
    } else {
        raw_cat(file);
    }
}

Courseware

This exercise has presentation materials available.

Answers - Adapt a C Program to CHERI C

  1. Example output:
    # make cat-baseline
    clang -o cat-baseline cat.c methods.c -g -O2 -target riscv64-unknown-freebsd -mno-relax -Wall -Wcheri -march=rv64gc -mabi=lp64d
    # make cat-cheri
    clang -o cat-cheri cat.c methods.c -g -O2 -target riscv64-unknown-freebsd -mno-relax -Wall -Wcheri -march=rv64gcxcheri -mabi=l64pc128d
    methods.c:70:43: warning: binary expression on capability types 'ptroff_t' (aka 'unsigned __intcap') and 'uintptr_t' (aka 'unsigned __intcap'); it is not clear which should be used as the source of provenance; currently provenance is inherited from the left-hand side [-Wcheri-provenance]
         return (write(fildes, (const void *)(off + (uintptr_t)buf), nbyte));
                                              ~~~ ^ ~~~~~~~~~~~~~~
    methods.c:80:7: warning: cast from provenance-free integer type to pointer type will give pointer that can not be dereferenced [-Wcheri-capability-misuse]
         fp = (FILE *)file;
              ^
    2 warnings generated.
    
  1. Example output:
    # ./cat-baseline /etc/hostid
    bb5fbb47-10ab-11ec-a609-f5a47707c223
    # ./cat-baseline -n /etc/hostid
         1  bb5fbb47-10ab-11ec-a609-f5a47707c223
    # ./cat-cheri /etc/hostid
    cat-cheri: write(2) failed: Bad address
    # ./cat-cheri -n /etc/hostid
    In-address space security exception (core dumped)
    #
    
  1. When run without gdb, cat-cheri prints:
    # ./cat-cheri /etc/hostid
    cat-cheri: write(2) failed: Bad address
    
    Looking at the source code, we can see there is only one call to write(2):
    # grep -R write src/exercises/adapt-c/cat
    src/exercises/adapt-c/cat/methods.c:write_off(int fildes, const void *buf, ptroff_t off, size_t nbyte)
    src/exercises/adapt-c/cat/methods.c:    return (write(fildes, (const void *)(off + (uintptr_t)buf), nbyte));
    src/exercises/adapt-c/cat/methods.c:                    if ((nw = write_off(wfd, buf, off, (size_t)nr)) < 0)
    src/exercises/adapt-c/cat/methods.c:                            err(1, "write(2) failed");
    
    The call is in the write_off() function and the message with the error is printed in case the call fails. Let's see what are arguments and result values for the system call by setting a breakpoint for write() in gdb:
    # gdb ./cat-cheri
    (...)
    (gdb) b write
    Function "write" not defined.
    Make breakpoint pending on future shared library load? (y or [n]) y
    Breakpoint 1 (write) pending.
    (gdb) r /etc/hostid
    Starting program: /root/cat-cheri /etc/hostid
    
    Breakpoint 1, write (fd=<optimized out>, buf=<optimized out>, nbytes=<optimized out>) at /usr/home/john/work/cheri/git/cheribsd/lib/libc/sys/write.c:48
    48                  __libc_interposing[INTERPOS_write])(fd, buf, nbytes));
    
    Even though the debugger believes that the function arguments are optimized out, from the CHERI-RISC-V calling conventions we know that the arguments are in the ca0, ca1, and ca2 registers:
    (gdb) info registers ca0 ca1 ca2
    ca0            0x1      0x1
    ca1            0x40802000       0x40802000
    ca2            0x25     0x25
    (gdb) disassemble 
    Dump of assembler code for function write:
    => 0x000000004027fa98 <+0>:     auipcc  ca3,0xb7
       0x000000004027fa9c <+4>:     clc     ca3,-424(ca3)
       0x000000004027faa0 <+8>:     clc     ca5,496(ca3)
       0x000000004027faa4 <+12>:    cjr     ca5
    End of assembler dump.
    
    We can see that write() was called to write to stdout (ca0) 37 bytes (ca2) from a buffer with an untagged capability (ca1). The write() libc function does not include a trapping instruction but it jumps with cjr. Let's see where it jumps to:
    (gdb) ni 4
    _write () at _write.S:4
    4       _write.S: No such file or directory.
    (gdb) disassemble 
    Dump of assembler code for function _write:
    => 0x0000000040282f40 <+0>:     li      t0,4
       0x0000000040282f42 <+2>:     ecall
       0x0000000040282f46 <+6>:     bnez    t0,0x40282f4e <_write+14>
       0x0000000040282f4a <+10>:    cret
       0x0000000040282f4e <+14>:    auipcc  ct1,0xffffd
       0x0000000040282f52 <+18>:    cincoffset      ct1,ct1,-1166
       0x0000000040282f56 <+22>:    cjr     ct1
    End of assembler dump.
    
    write() jumped to _write(), a system call wrapper written in assembly, that uses the ecall instruction to make a system call. Let's see what is its result:
    (gdb) ni 2
    4       in _write.S
    (gdb) info registers ca0 ct0
    ca0            0xe      0xe
    ct0            0x1      0x1
    (gdb) 
    
    The write() system call failed as the kernel set ct0 to 0x1 and returned errno 0xe in ca0. Looking at errno(2) and write(2), we can conclude that we passed an incorrect address to the buffer. It is likely here because the capability is just the address, without a tag.
  1. When compiling cat-cheri, the compiler printed:

    ./src/exercises/adapt-c/cat/methods.c:70:43: warning: binary expression on capability types 'ptroff_t' (aka 'unsigned __intcap') and 'uintptr_t' (aka 'unsigned __intcap'); it is not clear which should be used as the source of provenance; currently provenance is inherited from the left-hand side [-Wcheri-provenance]
            return (write(fildes, (const void *)(off + (uintptr_t)buf), nbyte));
                                                 ~~~ ^ ~~~~~~~~~~~~~~
    

    As the CHERI C/C++ Programming Guide says in Section 4.2.3:

    In the CHERI memory protection model, capabilities are derived from a single other capability.

    In our case, off + (uintptr_t)buf resulted in an untagged capability because off holds an integer value in an untagged capability and, as the compiler warns, it is used to create a resulting capability. In order to create a capability using the correct source capability, we can tell the compiler that off does not hold a valid address with a cast:

    diff --git a/src/exercises/adapt-c/cat/methods.c b/src/exercises/adapt-c/cat/methods.c
    index bb78a75..6520735 100644
    --- a/src/exercises/adapt-c/cat/methods.c
    +++ b/src/exercises/adapt-c/cat/methods.c
    @@ -67,7 +67,7 @@ static ssize_t
     write_off(int fildes, const void *buf, ptroff_t off, size_t nbyte)
     {
     
    -       return (write(fildes, (const void *)(off + (uintptr_t)buf), nbyte));
    +       return (write(fildes, (const void *)((size_t)off + (uintptr_t)buf), nbyte));
     }
     
     static void
    
  2. Example output:

    # gdb -ex run --args ./cat-cheri -n /etc/hostid
    (...)
    Starting program: /root/cat-cheri -n /etc/hostid
    
    Program received signal SIGPROT, CHERI protection violation.
    Capability tag fault caused by register cs4.
    verbose_cat (file=<optimized out>) at cat/methods.c:87
    87              for (prev = '\n'; (ch = getc(fp)) != EOF; prev = ch) {
    
    Thread 1 (LWP 100061 of process 2694):
    #0  verbose_cat (file=<optimized out>) at cat/methods.c:87
    #1  do_cat (file=<optimized out>, verbose=<optimized out>) at cat/methods.c:214
    #2  0x0000000000102dca in scanfiles (argv=0x3fbfdff7c0 [rwRW,0x3fbfdff7a0-0x3fbfdff7e0], verbose=<optimized out>) at cat/cat.c:172
    #3  0x0000000000102c52 in main (argc=3, argv=0x0) at cat/cat.c:128
    

    gdb says that cs4 triggered a CHERI exception:

    (gdb) info register cs4
    cs4            0x403545d0       0x403545d0
    

    cs4 holds an untagged capability and the program tries to load a word using cs4 which violates CHERI restrictions:

    (gdb) disassemble $pcc,+4
    Dump of assembler code from 0x102f7e to 0x102f82:
    => 0x0000000000102f7e <do_cat+294>:     clw     a0,16(cs4)
    End of assembler dump.
    

    Looking at the values of local variables, we can see that cs4 holds the value of the fp variable:

    (gdb) info locals
    fp = 0x403545d0
    gobble = 0
    line = 0
    prev = 10
    ch = <optimized out>
    wch = <optimized out>
    

    It means that for some reason fp holds an invalid (NULL-derived) capability.

  1. When compiling cat-cheri, the compiler printed:

    ./src/exercises/adapt-c/cat/methods.c:80:7: warning: cast from provenance-free integer type to pointer type will give pointer that can not be dereferenced [-Wcheri-capability-misuse]
            fp = (FILE *)file;
                 ^
    

    As the CHERI C/C++ Programming Guide says in Section 4.2:

    (...) only pointers implemented using valid capabilities can be dereferenced.

    and in Section 4.2.1:

    int,int32_t,long,int64_t,... These pure integer types should be used to hold integer values that will never be cast to a pointer type without first combining them with another pointer value – e.g., by using them as an array offset.

    In our case, long is cast to a pointer type which results in a NULL-derived capability without a tag, with an address set to an integer value, and which cannot be dereferenced. We can fix this bug by using a data type that can hold both integer values and pointers - uintptr_t:

    diff --git a/src/exercises/adapt-c/cat/cat.c b/src/exercises/adapt-c/cat/cat.c
    index 344e505..54cc864 100644
    --- a/src/exercises/adapt-c/cat/cat.c
    +++ b/src/exercises/adapt-c/cat/cat.c
    @@ -166,10 +166,10 @@ scanfiles(char *argv[], int verbose)
                            rval = 1;
                    } else if (verbose) {
                            if (fd == STDIN_FILENO)
    -                               do_cat((long)stdin, verbose);
    +                               do_cat((uintptr_t)stdin, verbose);
                            else {
                                    fp = fdopen(fd, "r");
    -                               do_cat((long)fp, verbose);
    +                               do_cat((uintptr_t)fp, verbose);
                                    fclose(fp);
                            }
                    } else {
    diff --git a/src/exercises/adapt-c/cat/cat.h b/src/exercises/adapt-c/cat/cat.h
    index c88f930..047c0b7 100644
    --- a/src/exercises/adapt-c/cat/cat.h
    +++ b/src/exercises/adapt-c/cat/cat.h
    @@ -51,6 +51,6 @@
     
     #define SUPPORTED_FLAGS "belnstuv"
     
    -void do_cat(long file, int verbose);
    +void do_cat(uintptr_t file, int verbose);
     
     #endif /* !_CAT_H_ */
    diff --git a/src/exercises/adapt-c/cat/methods.c b/src/exercises/adapt-c/cat/methods.c
    index bb78a75..afe29d3 100644
    --- a/src/exercises/adapt-c/cat/methods.c
    +++ b/src/exercises/adapt-c/cat/methods.c
    @@ -71,7 +71,7 @@ write_off(int fildes, const void *buf, ptroff_t off, size_t nbyte)
     }
     
     static void
    -verbose_cat(long file)
    +verbose_cat(uintptr_t file)
     {
            FILE *fp;
            int ch, gobble, line, prev;
    @@ -166,7 +166,7 @@ ilseq:
     }
     
     static void
    -raw_cat(long file)
    +raw_cat(uintptr_t file)
     {
            long pagesize;
            int off, rfd, wfd;
    @@ -207,7 +207,7 @@ raw_cat(long file)
     }
     
     void
    -do_cat(long file, int verbose)
    +do_cat(uintptr_t file, int verbose)
     {
     
            if (verbose) {
    
  1. The first bug resulted in a system call error because there was no capability operation on an invalid capability. The operating system could not copy memory from the user address space because it checked if a user process passed an invalid capability and returned an error.

    The second bug resulted in a CHERI exception because an invalid capability was used to load a word from memory.

For more information on the C/C++ programming languages, CHERI compiler warnings and errors, we recommend to read the CHERI C/C++ Programming Guide.

Extending heap allocators for CHERI

CHERI's architectural protection is driven by software -- the compiler, linker, OS kernel, run-time linker, run-time libraries, and so on all manage capabilities as part of their program execution. Heap allocators, which are integrally tied into our notions of spatial and temporal safety, are typically extended to use CHERI in five ways:

  1. To implement spatial safety, bounds and permissions are set on returned pointers. (In this exercise.)

  2. To prevent bounds overlap on larger allocations from arising due to imprecise bounds caused by capability compression, large allocations are aligned and padded more strongly. (Not in this exercise.)

  3. If the allocator's free() implementation relies on reaching allocator metadata via its pointer argument (e.g., by looking immediately before or after to reach free-list pointers), then the implementation must be changed as access will otherwise be prevented by CHERI bounds and monotonicity. (In this exercise.)

  4. To implement temporal safety, allocated memory is registered with a temporal-safety run-time library when allocated, to implement kernel-assisted revocation. On free, the memory is is held in quarantine until revocation has been performed. (Not in this exercise.)

  5. To handle a further set of classes of misuse and pointer corruption, it is also important to perform validation of arguments to free(), such as by checking that the pointer is to the first byte of a valid allocation. (Not in this exercise.)

This exercise asks you to extend a simplified memory allocator with CHERI focusing only on (1) and (3) above. It supports only small fixed-size allocations that will not require further alignment or padding, and we will not consider temporal safety in this exercise.

The complete exercise is embodied in cheri-allocator.c, including the simplified allocator and also a main() routine that initializes and uses the allocator. main() allocates memory, and then overflows the allocation to corrupt internal allocator metadata, leading to a crash. Heap metadata corruption is a powerful exploitation tool; CHERI assists with mitigating it through pointer integrity features, but it is preferable to deterministically close vulnerabilities (e.g., via spatial safety).

  1. Compile cheri-allocator.c with the CHERI-enabled allocator-cheri make target. Run the binary, which will crash.

  2. Use GDB to demonstrate to yourself that the overflow has corrupted allocator metadata, leading to an eventual crash during a later call to alloc_allocate().

  3. Modify the allocator to use the cheri_bounds_set() API to set suitable bounds on the pointer returned by alloc_allocate(). Recompile cheri-allocator.c with a CHERI-enabled target.

  4. Use GDB to demonstrate to yourself that the overflow operation now causes an immediate crash as a result of attempting to store out of bounds, rather than triggering a later crash due to heap metadata corruption.

  5. Remove the overflow (performed with memset()) from the program. Recompile cheri-allocator.c with the CHERI-enabled make target.

  6. Use GDB to explore why the program now crashes in alloc_free(): How did adding bounds during allocation break later freeing of that memory?

  7. Correct the bug through the use of the cheri_address_get() and cheri_address_set() APIs, which allow transferring an address from one capability (with one set of bounds) to another (with a different set of bounds). What capability should we be using to provide the new bounds? Recompile cheri-allocator.c with the CHERI-enabled make target.

  8. Demonstrate that the program now runs successfully to completion.

The resulting allocator is now substantially safer with respect to spatial safety, preventing underflows and overflows from corrupting allocator metadata or the contents of other allocations. However, to continue hardening the allocator against various attacks, further work would be required, including better validating the argument of the free() function. This would ideally test that the pointer being freed points to memory managed by the allocator, that the pointer is in bounds, and that it points to the start of a current allocation. Further temporal safety also requires quarantining freed memory until all pointers to it have been revoked.

Source Files

cheri-allocator.c

/*
 * SPDX-License-Identifier: BSD-2-Clause-DARPA-SSITH-ECATS-HR0011-18-C-0016
 * Copyright (c) 2022 Robert N. M. Watson
 */

#include <sys/cdefs.h>

#include <assert.h>
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <sysexits.h>

#ifdef __CHERI_PURE_CAPABILITY__
#include <cheriintrin.h>
#endif

/*
 * Implement a very simple allocator for a fixed-size data type, with inline
 * metadata.  Calls to alloc_allocate() return a pointer to a fixed-size byte
 * array.  Calls to alloc_free() return it to the allocator for reuse.
 *
 * The implementation is simplistic, and is designed to support an exercise
 * relating to: (a) bounds setting; and (b) monotonicty and rederivation.
 * Each allocation is described by 'struct allocation', which consists of
 * free-list pointers and an array of bytes that make up the allocation
 * itself.  Those allocations are stored as a sequential array in a global
 * variable initialised by BSS:
 *
 *  /--------- index 0 ----------\ /--------- index 1 ----------\ /--...
 *
 * +--------+-----------------...-+--------+-----------------...-+---...
 * | a_next | a_bytes[ALLOC_SIZE] | a_next | a_bytes[ALLOC_SIZE] |
 * +--------+-----------------...-+--------+-----------------...-+---...
 *
 *                                ^                              ^
 *      \_________________________/    \_________________________/
 *        If unallocated, pointer        If unallocated, pointer
 *        to next free allocation.       to next free allocation.
 *
 * Allocation storage is sized below the threshold requiring extra alignment
 * or padding to account for capability bounds compression.
 */
#define	ALLOC_SIZE		128		/* Allocation data size. */
struct alloc_storage {
    struct alloc_storage	*a_next;		/* Free list. */
    uint8_t			 a_bytes[ALLOC_SIZE];	/* Allocated memory. */
};

#define	ALLOC_MAX	16			/* Availaable allocations. */
struct alloc_storage alloc_array[ALLOC_MAX];	/* Underlying storage. */
struct alloc_storage *alloc_nextfree;		/* Next available memory. */

/*
 * Initialise the free list, pointing alloc_nextfree at the array, and then
 * chaining array entries into the list.
 */
static void
alloc_init(void)
{
    int i;

    alloc_nextfree = &alloc_array[0];
    for (i = 0; i < ALLOC_MAX - 1; i++)
        alloc_array[i].a_next = &alloc_array[i + 1];
    alloc_array[ALLOC_MAX - 1].a_next = NULL;
    assert(alloc_array[ALLOC_MAX - 1].a_next == NULL);
}

/*
 * Allocate memory, pulling it off the free list and updating pointers as
 * needed.
 */
static void *
alloc_allocate(void)
{
    struct alloc_storage *alloc;

    if (alloc_nextfree == NULL)
        return (NULL);
    alloc = alloc_nextfree;
    alloc_nextfree = alloc->a_next;
    alloc->a_next = NULL;

    /* Return pointer to allocated memory. */
    return (alloc->a_bytes);
};

/*
 * Free memory, inserting it back into the free list.  Note use of
 * __containerof() to convert pointer to a_bytes back into the container
 * struct pointer.
 */
static void
alloc_free(void *ptr)
{
    struct alloc_storage *alloc;

    /* Convert pointer to allocated memory into pointer to metadata. */
    alloc = __containerof(ptr, struct alloc_storage, a_bytes);
    alloc->a_next = alloc_nextfree;
    alloc_nextfree = alloc;
}

int
main(void)
{
    void *ptr1, *ptr2, *ptr3;

    /* Initialise allocator. */
    alloc_init();
    printf("Allocator initialised\n");

    /*
     * Allocate some memory.
     */
    printf("Allocating memory\n");
    ptr1 = alloc_allocate();
    printf("Allocation returned %p\n", ptr1);

    /*
     * Run off the end of the memory allocation, corrupting the next
     * allocation's metadata.  Free when done.
     */
    printf("Preparing to overflow %p\n", ptr1);
    memset(ptr1 + ALLOC_SIZE, 'A', sizeof(void *));
    printf("Overflowed allocation %p\n", ptr1);

    printf("Freeing allocation %p\n", ptr1);
    alloc_free(ptr1);
    printf("Allocation %p freed\n", ptr1);

    /*
     * Perform three sequential allocations to cause the allocator to
     * dereference the corrupted pointer, performing a store.
     */
    printf("Allocating memory\n");
    ptr1 = alloc_allocate();
    printf("Allocation returned %p\n", ptr1);

    printf("Allocating memory\n");
    ptr2 = alloc_allocate();
    printf("Allocation returned %p\n", ptr2);

    printf("Allocating memory\n");
    ptr3 = alloc_allocate();
    printf("Allocation returned %p\n", ptr3);

    /*
     * Clear up the mess.
     */
    printf("Freeing allocation %p\n", ptr3);
    alloc_free(ptr3);
    printf("Allocation %p freed\n", ptr3);

    printf("Freeing allocation %p\n", ptr2);
    alloc_free(ptr2);
    printf("Allocation %p freed\n", ptr2);

    printf("Freeing allocation %p\n", ptr1);
    alloc_free(ptr1);
    printf("Allocation %p freed\n", ptr1);

    exit(EX_OK);
}

Answers

Introducing heap-allocator bounds

  1. GDB will show a CHERI tag violation resulting from memset() overwriting the a_next field in the second allocation entry, which is tripped over by a later call to alloc_allocate():
Starting program: /root/cheri-allocator
Allocator initialised
Allocating memory
Allocation returned 0x104550
Preparing to overflow 0x104550
Overflowed allocation 0x104550
Freeing allocation 0x104550
Allocation 0x104550 freed
Allocating memory
Allocation returned 0x104550
Allocating memory
Allocation returned 0x1045e0
Allocating memory

Program received signal SIGPROT, CHERI protection violation.
Capability tag fault caused by register ca0.
alloc_allocate () at cheri-allocator.c:83
83              alloc_nextfree = alloc->a_next;
(gdb) p alloc
$1 = (struct alloc_storage *) 0x4141414141414141 [,0x4141402800000000-0x414142a000000000] (invalid,sealed)
  1. When compiling for CHERI C, use cheri_bounds_set() to set bounds on the returned pointer:
        /* Return pointer to allocated memory. */
#ifdef __CHERI_PURE_CAPABILITY__
        return (cheri_bounds_set(alloc->a_bytes, ALLOC_SIZE));
#else
        return (alloc->a_bytes);
#endif
  1. With this change, the memset() call in main() triggers a bounds violation exception on overflow:
Starting program: /root/cheri-allocator
Allocator initialised
Allocating memory
Allocation returned 0x104550
Preparing to overflow 0x104550

Program received signal SIGPROT, CHERI protection violation.
Capability bounds fault caused by register ca3.
memset (dst0=0x1045d0 <alloc_array+144> [rwRW,0x104550-0x1045d0], c0=65, length=15) at /usr/home/john/work/cheri/git/cheribsd/lib/libc/string/memset.c:94
94                              *dst++ = VAL;

Reaching allocator metadata

  1. Following this change, alloc_free() crashes with a bounds violation, due to reaching outside the bounds of the passed memory allocation:
Starting program: /root/cheri-allocator
Allocator initialised
Allocating memory
Allocation returned 0x104420
Freeing allocation 0x104420

Program received signal SIGPROT, CHERI protection violation.
Capability bounds fault caused by register cfp.
alloc_free (ptr=<optimized out>) at cheri-allocator.c:106
106             alloc->a_next = alloc_nextfree;
(gdb) bt
#0  alloc_free (ptr=<optimized out>) at cheri-allocator.c:106
#1  main () at cheri-allocator.c:137
(gdb) p alloc
$1 = (struct alloc_storage *) 0x104410 <alloc_array> [rwRW,0x104420-0x1044a0]
  1. We need to create a new capability, derived from alloc_array but with the address generated from pointer to the memory being freed. One way to do this is using the cheri_address_get() and cheri_address_set(), reading the address from one capability and setting it on the other:
#ifdef __CHERI_PURE_CAPABILITY__
        /*
         * Generate a new pointer to the allocation that is derived from the
         * one passed by the consumer.
         */
        ptr = cheri_address_set(alloc_array, cheri_address_get(ptr));
#endif

Note that this is not a complete solution to providing spatial safety here: software could still accidentally pass an out-of-bounds pointer.

CheriABI Showcase

This exercise demonstrates several aspects of CheriABI, which defines how CheriBSD processes are constructed, how function calls are made, how a process interacts with the kernel through system calls, and other such foundational details.

The Kernel Voluntarily Honors Capability Bounds

kern-read-over.c demonstrates a (potential) loss of spatial safety when pointers are passed from userspace to the kernel. The kernel, by convention, has full access to all of userspace memory. Even when CheriBSD is running CheriABI programs, this is true: the kernel holds a capability with full RWX access to all userspace addresses. Therefore, the kernel can act as a confused deputy, accessing memory with its legitimate authority but without intent.

  1. Compile kern-read-over.c for both the baseline architecture with the kern-read-over-baseline make target and the CHERI-enabled architecture with the kern-read-over-cheri target.

  2. Run these programs and observe their outputs.

  3. Focusing on the read() system call, what happens in the two versions of the program. When, in particular, does it look like the CHERI version notices something is amiss?

  4. If you have done the inter-stack-object buffer overflow exercise, contrast the behaviors of the two CHERI-enabled programs.

The Process Memory Map

In most UNIX programs, the rights to manipulate the virtual memory map are ambient: any piece of code can change the virtual memory permissions associated with a page, munmap pages, or even request a replacement mapping ("fixed mmap") almost anywhere in the address space. This risks allowing evasion of CHERI capabilities' protection properties, as CHERI capabilities are interpreted in combination with the virtual memory map.

Therefore, the CheriBSD kernel avails itself of a software permission bit in CHERI capabilities. Such permissions are not architecturally interpreted but are still subject to architectural protection (and so, for example, a zero permission bit may not be set to one without simultaneously clearing the capability tag). In particular, CheriBSD defines CHERI_PERM_SW_VMEM, sets this permission bit when returning pointers to new allocations of address space, and requires that capabilities passed to address-space-manipulating functions bear this permission. Userspace components are free to clear this permission when delegating access to address space.

  1. Compile perm-vmem.c for both the baseline architecture with the perm-vmem-baseline make target and for the CHERI-enabled architecture with the perm-vmem-cheri target.

  2. Run these programs and observe their outputs. The printf format strings for capabilities, %p and %#p, elide some usually-excessive details, and CHERI_PERM_SW_VMEM is generally regarded as one such. gdb's pretty-printing, similarly. However, we can programmatically extract the permissions field and display it.

  3. Modify perm-vmem.c to verify that madvise(MADV_FREE) and mmap(MAP_FIXED) also are permitted for the capability returned directly from mmap but are not permitted for the heap-derived pointer.

(Extra Credit!) Initial Process Construction

We have largely focused on program behavior after it has been loaded and is running. Let us look in a little more detail at some aspects of the initial construction. While modern ELF loading is well beyond the scope of this document, and is perhaps best summarized as "here be dragons", we can nevertheless take a quick glance at some interesting features of CheriABI startup.

  1. Compile print-more.c for both the baseline (print-more-baseline) and the CHERI-enabled (print-more-cheri) architectures.

  2. Run both these programs and observe their outputs. As might be predicted, the CHERI version reports a wide variety of capabilities to different parts of the address space. Run both programs several times; what do you observe?

    Let us examine several interesting aspects of the reported capabilities.

  3. Launch gdb ./print-more-cheri and have it start the program and stop before running any instructions, with starti. Where do we find ourselves?

  4. Use info inferiors to obtain the child process identifier (PID) and !procstat vm NNN (replacing NNN with the child PID) to show the initial address space arranged by the kernel.

    Which of these initial mappings are targeted by the values reported for &rodata_const, &relro_ptr, &rw_ptr and printf in step 2? What are the permissions for these mappings?

  5. Just because the page mappings exist, however, CHERI programs need to have capabilities installed to access them. Here at the very beginning of a process's life, we are in a good position to see the root capabilities that the kernel makes available. Use info registers to see the initial contents of the register file.

  6. Let's begin our tour with csp, the capability stack pointer register.

    First, what may strike you as surprising (and why) about the stack pointer being replaced by a capability?

    Second, compare the address space map obtained above with the current csp value; what has the kernel arranged to "back" the region of address space within stack bounds?

    If you are familiar with Stack Clash Vulnerabilities, explain how the two aspects above work in tandem to mitigate this class of vulnerability.

    Third, contrast the relative order of &argv[0] and &stack_local as reported on the two different architectures in step 2 above.

  7. Having access to the stack is all well and good, but surely there is more to a process than that. At the beginning of a CheriABI process's life, the capability in ca0 (the first "argument register") points to the "auxiliary vector", an array of Elf_Auxinfo structures constructed by the kernel.

    gdb can ask the kernel for, and display, the information in the auxiliary vector with info auxv. However, the pretty-printer is not capability aware, so let's also directly spelunk the structure. Use some gdb scripting to print out the auxiliary vector in its entirety:

    set $i = 0
    while(((Elf_Auxinfo *)$ca0)[$i].a_type != 0)
    p ((Elf_Auxinfo *)$ca0)[$i]
    set $i = $i + 1
    end
    

    Use the more human-friendly info auxv to interpret the a_type values.

    In addition to the AT_ARGV value we have already (indirectly) seen above, there are many other capabilities to nearby parts of the address space, including the initial environment vector (AT_ENVV) and the executable path (AT_EXECPATH).

    More usefully, however,

    • AT_PHDR supplies a read/write capability to the loaded executable.

    • AT_ENTRY supplies a read/execute capability to the loaded executable, pointed at its entrypoint.

    • AT_BASE supplies a full read/write/execute capability to the program's "interpreter" (dynamic loader). The elevated permissions here allow the loader to (relatively) easily relocate itself early in execution.

    From which of these capabilities are the displayed values of &rodata_const, &relro_ptr, and &rw_ptr from step 2 sourced? What permissions have been shed in the derivation? How do these permissions differ from those of the underlying page mappings?

  8. The displayed value for printf is tagged as being a (sentry). Modify the program to attempt to display the result of computing either

    • *(char *)(printf) or
    • (void*)((uintptr_t)printf + 1).

    Compile and run this modified version (or both). What happens?

    Sentry (short for "Sealed Entry") capabilities are a special form of capabilities: they are immutable and inert, conveying to the bearer no authority to the target, until they become the program counter, at which point they are unsealed into being an ordinary capability. Thus, we can neither read through nor mutate our handle to printf, yet we can jump to it.

    If you are familiar with Return Oriented Programming and Jump Oriented Programming, you may wish to consider the cumulative challenge added by CHERI's architectural provenance requirement combined with pervasive use of sentry capabilities for dynamically resolved symbols.

Source

kern-read-over.c

/*
 * SPDX-License-Identifier: BSD-2-Clause-DARPA-SSITH-ECATS-HR0011-18-C-0016
 * Copyright (c) 2020 SRI International
 * Copyright (c) 2022 Microsoft Corporation
 */
#include <assert.h>
#include <err.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/wait.h>

#define bsz 16

int
main(void)
{
    int fds[2];
    pid_t pid;

    if (pipe(fds) == -1)
        err(1, "pipe");
    if ((pid = fork()) == -1)
        err(1, "fork");
    if (pid == 0) {
        char out[2*bsz];
        for (size_t i = 0; i < sizeof(out); i++) {
            out[i] = 0x10 + i;
        }

        if (write(fds[0], out, sizeof(out)) != sizeof(out)) {
            err(1, "write");
        }
        printf("Write OK\n");
    } else {
        int res;
        char upper[bsz] = { 0 };
        char lower[bsz] = { 0 };

        waitpid(pid, NULL, 0);

        printf("lower=%p upper=%p\n", lower, upper);
        assert((ptraddr_t)upper == (ptraddr_t)&lower[sizeof(lower)]);

        res = read(fds[1], lower, sizeof(lower) + sizeof(upper));
        assert(res != 0);
        if (res > 0) {
            printf("Read 0x%x OK; lower[0]=0x%x upper[0]=0x%x\n",
                res, lower[0], upper[0]);
        } else if (res < 0) {
            printf("Bad read (%s); lower[0]=0x%x upper[0]=0x%x\n",
                strerror(errno), lower[0], upper[0]);
        }
    }

    return 0;
}

perm-vmem.c

/*
 * SPDX-License-Identifier: BSD-2-Clause
 * Copyright (c) 2022 Microsoft Corporation
 */
#include <assert.h>
#include <errno.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>

#ifdef __CHERI_PURE_CAPABILITY__
#include <cheri/cherireg.h>
#define PRINTF_PTR "#p"
#else
#define PRINTF_PTR "p"
#endif

int
main(void)
{
    char *m, *p;
    int res;

    /* Get a page from the kernel and give it back */
    p = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE | MAP_ANON,
        -1, 0);
    assert(p != MAP_FAILED);
    printf("Directly mapped page at p=%" PRINTF_PTR "\n", p);
#ifdef __CHERI_PURE_CAPABILITY__
    printf(" p.perms=0x%lx\n", __builtin_cheri_perms_get(p));
#endif
    /*res = madvise(p, 4096, MADV_FREE);
    assert(res == 0);

    p = mmap(p, 4096, PROT_READ|PROT_WRITE, MAP_FIXED | MAP_PRIVATE | MAP_ANON,
        -1, 0);
    assert(p != MAP_FAILED);*/

    res = munmap(p, 4096);
    assert(res == 0);

    /* Get a pointer to a whole page of the heap*/
    m = malloc(8192);
    p = __builtin_align_up(m, 4096);
    printf("Punching hole in the heap at p=%" PRINTF_PTR "\n", p);
#ifdef __CHERI_PURE_CAPABILITY__
    printf(" p.perms=0x%lx\n", __builtin_cheri_perms_get(p));
#endif

    /*char *q = mmap(p, 4096, PROT_READ|PROT_WRITE, MAP_FIXED | MAP_PRIVATE | MAP_ANON,
        -1, 0);
    assert(q != MAP_FAILED);

    if (madvise(p, 4096, MADV_FREE) != 0) {
        printf("madvise failed: %s\n", strerror(errno));
    }*/

    if (munmap(p, 4096) != 0) {
        printf("munmap failed: %s\n", strerror(errno));
    }

    printf("Done\n");
}

print-more.c

/*
 * SPDX-License-Identifier: BSD-2-Clause
 * Copyright (c) 2022 Microsoft Corporation
 */
#include <assert.h>
#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>

#ifdef __CHERI_PURE_CAPABILITY__
#define PRINTF_PTR "#p"
#else
#define PRINTF_PTR "p"
#endif

static const int rodata_const = 42;
static int (*const relro_ptr)(const char *, ...) = printf;
static int (*rw_ptr)(const char *, ...) = printf;

int
main(int argc, char **argv)
{
    int stack_local;

    printf("&argv[0]=%" PRINTF_PTR "\n", &argv[0]);
    printf(" argv[0]=%" PRINTF_PTR "\n",  argv[0]);
    printf("&stack_local=%" PRINTF_PTR "\n", &stack_local);

    printf("&rodata_const=%" PRINTF_PTR "\n", &rodata_const);

    printf("&relro_ptr=%" PRINTF_PTR "\n", &relro_ptr);
    printf("&rw_ptr=%" PRINTF_PTR "\n", &rw_ptr);

    printf("printf=%" PRINTF_PTR "\n", printf);
}

Courseware

This exercise has presentation materials available.

Answers

The Kernel as a Potentially Confused Deputy

  1. Example output from a baseline architecture:

    Write OK
    lower=0x80922400 upper=0x80922410
    Read 0x20 OK; lower[0]=0x10 upper[0]=0x20
    

    And from a CHERI-enabled architecture:

    Write OK
    lower=0x3fffdfff28 upper=0x3fffdfff38
    Bad read (Bad address); lower[0]=0x10 upper[0]=0x0
    
  2. On the baseline architecture, the kernel dutifully writes 0x20 bytes to the target address, regardless of the C object model. On the CHERI architecture, the kernel acts intentionally with the capability provided by userspace and so encounters a trap when copying bytes out to userspace. Because the kernel updates lower[0], we can conjecture that the kernel is not performing explicit bounds checks but is rather operating under a trust-but-validate model, handling the architectural trap when it uses copyout() to copy bytes from its internal pipe buffer to the indicated userspace buffer.

  3. Not all capability faults are fatal! While the inter-stack-object overflow exercise let the program die of the SIGPROT thrown its way, here, the kernel maps the architectural trap to a failure return rather than a fatal signal.

The Process Memory Map

  1. Example output from a baseline architecture:

    Directly mapped page at p=0x84dc0000
    Punching hole in the heap at p=0x83b48000
    Done
    

    And from a CHERI-enabled architecture:

    Directly mapped page at p=0x40139000 [rwRW,0x40139000-0x4013a000]
     p.perms=0x7817d
    Punching hole in the heap at p=0x407d1000 [rwRW,0x407d1000-0x407d3000]
     p.perms=0x6817d
    munmap failed: Memory protection violation
    Done
    
  2. This amounts to adding calls to madvise(p, 4096, MADV_FREE) and mmap(p, 4096, PROT_READ|PROT_WRITE, MAP_FIXED|MAP_PRIVATE|MAP_ANON, -1, 0) in the right places and verifying that the operations return appropriately. Additionally, you may wish to check errno in failure cases and the contents of memory after mmap to ensure that it has, indeed, behaved as expected in both cases.

(Extra Credit!) Initial Process Construction

  1. Example output from a baseline architecture:

    &argv[0]=0x806e0d08
     argv[0]=0x806e0fd0
    &stack_local=0x806e0c94
    &rodata_const=0x105c0
    &relro_ptr=0x105c8
    &rw_ptr=0x13e18
    printf=0x11c60
    

    And from a CHERI-enabled architecture:

    &argv[0]=0x3fbfdff8d0 [rwRW,0x3fbfdff8d0-0x3fbfdff8f0]
     argv[0]=0x3fbfdffe30 [rwRW,0x3fbfdffe30-0x3fbfdffe4a]
    &stack_local=0x3fffdfff6c [rwRW,0x3fffdfff6c-0x3fffdfff70]
    &rodata_const=0x1005c8 [rR,0x1005c8-0x1005cc]
    &relro_ptr=0x102f00 [rR,0x102f00-0x102f10]
    &rw_ptr=0x104070 [rwRW,0x104070-0x104080]
    printf=0x402608d8 [rxR,0x4013a000-0x40782000] (sentry)
    

    Running the baseline version multiple times should produce different output thanks to Address Space Layout Randomization (ASLR), a popular probabilistic countermeasure against pointer forgery. Because CHERI offers deterministic protection against pointer forgery by its very nature, ASLR for CheriABI programs is turned off.

  2. Example session:

    (gdb) starti
    Starting program: /buildroot/print-more-cheri 
    
    Program stopped.
    rtld_start () at /cheri/source/mainline/cheribsd/libexec/rtld-elf/riscv/rtld_start.S:62
    62      /cheri/source/mainline/cheribsd/libexec/rtld-elf/riscv/rtld_start.S: No such file or directory.
    

    We find ourselves nowhere within print-more-cheri but, rather, at the very beginning of the dynamic loader (rtld).

  3. Example session for CHERI:

    (gdb) info inferiors
      Num  Description       Executable        
    * 1    process 1013      /buildroot/print-more-cheri 
    (gdb) !procstat vm 1013
      PID              START                END PRT    RES PRES REF SHD FLAG  TP PATH
     1013           0x100000           0x101000 r--R-    1    3   3   0 CN--- vn /buildroot/print-more-cheri
     1013           0x101000           0x102000 r-xR-    1    3   3   0 CN--- vn /buildroot/print-more-cheri
     1013           0x102000           0x104000 rw-RW    2    3   3   0 CN--- vn /buildroot/print-more-cheri
     1013           0x104000           0x105000 rw-RW    1    1   1   0 ----- df 
     1013         0x40104000         0x4010f000 r--R-   11  344  32   0 CN--- vn /libexec/ld-elf.so.1
     1013         0x4010f000         0x4012a000 r-xR-   27    0   1   0 C---- vn /libexec/ld-elf.so.1
     1013         0x4012a000         0x4012b000 rw-RW    1  344  32   0 CN--- vn /libexec/ld-elf.so.1
     1013         0x4012b000         0x4012d000 rw-RW    2  344  32   0 CN--- vn /libexec/ld-elf.so.1
     1013         0x4012d000         0x4012f000 rw-RW    1    1   1   0 ----- df 
     1013       0x3fbfd80000       0x3fbfe00000 rw-RW    1    1   1   0 ----- df 
     1013       0x3fbfe00000       0x3fffde0000 -----    0    0   0   0 G---- gd 
     1013       0x3fffde0000       0x3fffe00000 rw-RW    0    0   0   0 ---D- -- 
     1013       0x3ffffff000       0x4000000000 r-x--    1    1  13   0 ----- ph 
    
    pointer mapping permissions
    &argv[0] 0x3fbfd80000 rw-RW
    argv " "
    stack_local 0x3fffde0000 rw-RW
    rodata_const 0x100000 r--R-
    relro_ptr 0x102000 rw-RW
    rw_ptr 0x104000 rw-RW
    printf not initially mapped n/a
  4. Abridged output:

    cra            0xd117200009e18201000000004010f040       0x4010f040 <rtld_start> [rxR,0x40104000-0x4012f000] (sentry)
    csp            0xd17d000003ff2ffe0000003fffe00000       0x3fffe00000 [rwRW,0x3fbfe00000-0x3fffe00000]
    ca0            0xd17d00000785b9b40000003fbfdff9b0       0x3fbfdff9b0 [rwRW,0x3fbfdff9b0-0x3fbfdffe10]
    pcc            0x4010f040       0x4010f040 <rtld_start>
    ddc            0x0      0x0
    
  5. In baseline programs, the stack is bounded only by operating system measures -- the kernel will refuse to grow what it considers to be "the stack" beyond some limit. However, architecturally, there is no a priori limit to stack growth. In CHERI, by contrast, the stack is accessed via a capability, which must be constructed up front.

    For the CHERI program, the kernel has backed the entirety of stack memory with page mappings:

     1013       0x3fbfe00000       0x3fffde0000 -----    0    0   0   0 G---- gd 
     1013       0x3fffde0000       0x3fffe00000 rw-RW    0    0   0   0 ---D- -- 
    

    The latter of these is marked as "growing down" (the D in the penultimate field) while the former is considered a "guard" mapping (the G flag and gd type), serving to prevent any other claimant to the address space.

    The initial bounds on the stack capability prevent half of Stack Clash: the stack capability cannot authorize access to a heap region, even if, say, indexing an on-stack array goes very far out of bounds. The primordial guard entry serves to prevent the second half: the heap cannot grow into the stack, because the kernel will not use that address space to satisfy mmap requests (and, moreover, no capability held by userspace, including the one in csp, bears CHERI_PERM_SW_VMEM, so the stack or its guard cannot be torn down).

    Traditionally, argv and its contents (as well as the environment vector and indeed the entire auxiliary vector) is placed above the initial stack pointer, so &argv[0] is above &stack_local. However, here we can see that CheriBSD chooses to locate all this initial data below the stack reservation, meaning that &stack_local is further up the address space than &argv[0]. This allows the kernel to ensure that parts of this initial state are immutable or that there exists exactly one capability to parts of the structure (allowing for easier reasoning about capability flow in userspace); these would not be true if this initial data were also reachable through the stack capability.

  6. Example session:

    (gdb) info auxv
    3    AT_PHDR              Program headers for program    0x100040
    4    AT_PHENT             Size of program header entry   56
    5    AT_PHNUM             Number of program headers      11
    6    AT_PAGESZ            System page size               4096
    8    AT_FLAGS             Flags                          0x0
    9    AT_ENTRY             Entry point of program         0x101a30
    7    AT_BASE              Base address of interpreter    0x40104000
    24   AT_EHDRFLAGS         ELF header e_flags             0x30005
    15   AT_EXECPATH          Executable path                0x3fbfdfffa0 "/mnt/tmp/print-more-cheri"
    18   AT_OSRELDATE         OSRELDATE                      1400051
    16   AT_CANARY            Canary for SSP                 0x3fbfdfff60
    17   AT_CANARYLEN         Length of the SSP canary       64
    19   AT_NCPUS             Number of CPUs                 1
    20   AT_PAGESIZES         Pagesizes                      0x3fbfdfff40
    21   AT_PAGESIZESLEN      Number of pagesizes            24
    22   AT_TIMEKEEP          Pointer to timehands           0x3ffffff020
    23   AT_STACKPROT         Initial stack protection       0x3
    25   AT_HWCAP             Machine-dependent CPU capability hints 0x112d
    27   AT_BSDFLAGS          ELF BSD flags                  0x0
    28   AT_ARGC              Argument count                 1
    29   AT_ARGV              Argument vector                0x3fbfdff880
    30   AT_ENVC              Environment count              16
    31   AT_ENVV              Environment vector             0x3fbfdff8a0
    32   AT_PS_STRINGS        Process strings                0x3fbfdfffc0
    0    AT_NULL              End of vector                  0x0
    (gdb)    set $i = 0
    (gdb)    while(((Elf_Auxinfo *)$ca0)[$i].a_type != 0)
     >   p ((Elf_Auxinfo *)$ca0)[$i]
     >   set $i = $i + 1
     >   end
    $1 = {a_type = 3, a_un = {a_val = 1048640, a_ptr = 0x100040 [rwRW,0x100000-0x104260], a_fcn = 0x100040 [rwRW,0x100000-0x104260]}}
    $2 = {a_type = 4, a_un = {a_val = 56, a_ptr = 0x38, a_fcn = 0x38}}
    $3 = {a_type = 5, a_un = {a_val = 11, a_ptr = 0xb, a_fcn = 0xb}}
    $4 = {a_type = 6, a_un = {a_val = 4096, a_ptr = 0x1000, a_fcn = 0x1000}}
    $5 = {a_type = 8, a_un = {a_val = 0, a_ptr = 0x0, a_fcn = 0x0}}
    $6 = {a_type = 9, a_un = {a_val = 1055280, a_ptr = 0x101a30 <_start> [rxR,0x100000-0x104260], a_fcn = 0x101a30 <_start> [rxR,0x100000-0x104260]}}
    $7 = {a_type = 7, a_un = {a_val = 1074806784, a_ptr = 0x40104000 [rwxRW,0x40104000-0x4012f000], a_fcn = 0x40104000 [rwxRW,0x40104000-0x4012f000]}}
    $8 = {a_type = 24, a_un = {a_val = 196613, a_ptr = 0x30005, a_fcn = 0x30005}}
    $9 = {a_type = 15, a_un = {a_val = 273802067872, a_ptr = 0x3fbfdfffa0 [rwRW,0x3fbfdfffa0-0x3fbfdfffba], a_fcn = 0x3fbfdfffa0 [rwRW,0x3fbfdfffa0-0x3fbfdfffba]}}
    $10 = {a_type = 18, a_un = {a_val = 1400051, a_ptr = 0x155cf3, a_fcn = 0x155cf3}}
    $11 = {a_type = 16, a_un = {a_val = 273802067808, a_ptr = 0x3fbfdfff60 [rwRW,0x3fbfdfff60-0x3fbfdfffa0], a_fcn = 0x3fbfdfff60 [rwRW,0x3fbfdfff60-0x3fbfdfffa0]}}
    $12 = {a_type = 17, a_un = {a_val = 64, a_ptr = 0x40, a_fcn = 0x40}}
    $13 = {a_type = 19, a_un = {a_val = 1, a_ptr = 0x1, a_fcn = 0x1}}
    $14 = {a_type = 20, a_un = {a_val = 273802067776, a_ptr = 0x3fbfdfff40 [rwRW,0x3fbfdfff40-0x3fbfdfff58], a_fcn = 0x3fbfdfff40 [rwRW,0x3fbfdfff40-0x3fbfdfff58]}}
    $15 = {a_type = 21, a_un = {a_val = 24, a_ptr = 0x18, a_fcn = 0x18}}
    $16 = {a_type = 22, a_un = {a_val = 274877902880, a_ptr = 0x3ffffff020 [rwRW,0x3ffffff020-0x3ffffff190], a_fcn = 0x3ffffff020 [rwRW,0x3ffffff020-0x3ffffff190]}}
    $17 = {a_type = 23, a_un = {a_val = 3, a_ptr = 0x3, a_fcn = 0x3}}
    $18 = {a_type = 25, a_un = {a_val = 4397, a_ptr = 0x112d, a_fcn = 0x112d}}
    $19 = {a_type = 27, a_un = {a_val = 0, a_ptr = 0x0, a_fcn = 0x0}}
    $20 = {a_type = 28, a_un = {a_val = 1, a_ptr = 0x1, a_fcn = 0x1}}
    $21 = {a_type = 29, a_un = {a_val = 273802066048, a_ptr = 0x3fbfdff880 [rwRW,0x3fbfdff880-0x3fbfdff8a0], a_fcn = 0x3fbfdff880 [rwRW,0x3fbfdff880-0x3fbfdff8a0]}}
    $22 = {a_type = 30, a_un = {a_val = 16, a_ptr = 0x10, a_fcn = 0x10}}
    $23 = {a_type = 31, a_un = {a_val = 273802066080, a_ptr = 0x3fbfdff8a0 [rwRW,0x3fbfdff8a0-0x3fbfdff9b0], a_fcn = 0x3fbfdff8a0 [rwRW,0x3fbfdff8a0-0x3fbfdff9b0]}}
    $24 = {a_type = 32, a_un = {a_val = 273802067904, a_ptr = 0x3fbfdfffc0 [rwRW,0x3fbfdfffc0-0x3fbfe00000], a_fcn = 0x3fbfdfffc0 [rwRW,0x3fbfdfffc0-0x3fbfe00000]}}
    

    rodata_const and relro_ptr could each be derived from either AT_PHDR (shedding write permission) or AT_BASE (shedding execute); despite that the pages are mapped read-write, the capability permissions will enforce that they cannot be used to modify the values here. If these are the only (non-TCB) capabilities to those locations, then the values must, indeed, be constant (outside bugs in the TCB).

    rw_ptr must, on the other hand, have come from AT_PHDR.

  7. In either case, the program will trap and abort (In-address space security exception).

    Sentries are believed to complicate would-be ROP or JOP attacks without excessively complicating the architecture or system software. Without, it would be possible to locate and invoke gadgets within resolvable functions in any loaded object, as the function must have execute permission to its entire body, and so the capability used to reference the function would need to (transitively) confer such rights. Sentries let us refer to some code without the rights to jump to any part of it except the intended entry point. At the time of this writing, CHERI-RISC-V always, and Morello optionally, automatically constructs sentry capabilities when executing linked control transfers.

    At present, the default linkage model of CHERI means that PCC has bounds of an entire loaded .text segment (the executable or one of its loaded libraries), so ensuring the use of sentries when crossing segments restricts the ability to source gadgets to any that may exist within the segment vulnerable to ROP or JOP injection.

    The curious reader should seek out additional information about CHERI's "object type" mechanism and sealed capabilities, of which sentries are just one example.

Appendix

This book and related source code are released under the following license:

SPDX-License-Identifier: BSD-2-Clause-DARPA-SSITH-ECATS-HR0011-18-C-0016

Copyright (c) 2020 Jessica Clarke
Copyright (c) 2020, 2022 Robert N. M. Watson
Copyright (c) 2020 SRI International
Copyright (c) 2022 Microsoft Corporation

This software was developed by SRI International and the University of
Cambridge Computer Laboratory (Department of Computer Science and
Technology) under DARPA contract HR0011-18-C-0016 ("ECATS"), as part of the
DARPA SSITH research programme.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.