Memory Safe Context Switching

Memory Safe Context Switching

Support for ucontext APIs is new since release 0.680. If you want to play with setcontext, getcontext, makecontext, and swapcontext then you have to build from source.

This document describes how Fil-C supports longjmp

, setjmp

, setcontext

, getcontext

, makecontext

, and swapcontext

in a totally memory-safe way. In particular, no misuse of those APIs in Fil-C can lead to stack corruption or any other violation of Fil-C's capability model.

These APIs are widely used:

longjmp

andsetjmp

are used in C programs to implement exception handling. It's especially common to use them to implement exceptions "thrown" from signal handlers.getcontext

,setcontext

,makecontext

, andswapcontext

(aka theucontext

APIs) are used to implement coroutines and fibers. For example, Boost usesucontext

as part of its fiber implementation.

The ucontext

APIs are less commonly used than longjmp

/setjmp

and some OSes (like Darwin) have deprecated them. However, they remain well supported in glibc.

Implementing these APIs in a way that preserves memory safety is hard since their misuse can result in restoring a dangling stack. For example, you could either setjmp

or getcontext

within some function, and then do any of the following things:

Return from that function. At this point, the context that was saved will attempt to restore a stack frame that no longer exists.

Exit from the thread. At this point, the context that was saved will attempt to restore execution on a stack that has been freed.

Even more friendly APIs like makecontext

and swapcontext

can be straightforwardly misused:

You can use

makecontext

to create a context that points to some stack, then free that stack, and then eitherswapcontext

orsetcontext

to that context. In Yolo-C, this will result in running on a dangling stack. Fil-C makes this not an error.You can call

swapcontext

with the second argument being the context that is currently executing. This might happen if you confuse the first and second arguments. In Yolo-C, in the best case, this will behave like alongjmp

; in the worst case, it will result in executing on a dangling stack. In Fil-C, this is a safety error that panics your program.

In Yolo-C, execution on a dangling stack results in the most confusing kinds of crashes, since the debugger won't even be able to print a stack trace! Worse, if the program has subtle bugs in its handling of contexts, then an attacker could exploit those bugs to cause the program to do whatever the attacker likes. In Fil-C, execution on a dangling stack is not possible: all such cases are either panics at the point where you misused longjmp

or one of the ucontext

APIs, or they are reliably legal execution because of how Fil-C manages stacks.

Fil-C implements setjmp

/longjmp

and the ucontext

APIs quite differently.

Making setjmp

/longjmp

Memory Safe

There is an impressive amount of depth to the depravity of setjmp

. Before going into the details of how Fil-C implements setjmp

/longjmp

, we need to discuss exactly what makes this function so amazingly evil.

setjmp

saves the context as it was at the moment when it was called so that when longjmp

is called later, setjmp

will return a second time. It is the fact that it returns twice that makes it so vile, and so we need to understand the implications precisely.

An Example

Consider this simple program:

#include <setjmp.h>
#include <stdio.h>
int main(int argc, char** argv)
{
volatile int x = 42;
jmp_buf jb;
if (setjmp(jb)) {
printf("x = %d\n", x);
return 0;
}
x = 666;
longjmp(jb, 1);
printf("Should not get here.\n");
return 1;
}

This program prints:

x = 666

And then exits. The flow is:

• On the first call to setjmp

, it returns 0 and saves its caller's context injb

. - Then we set x

to 666 andlongjmp

tojb

with the value 1. setjmp

returns 1, so weprintf

and exit.

Note that we have to mark x

as volatile

for the program to reliably print 666. Otherwise, the compiler is allowed to optimize the access to x

and have it return 42 instead. This might happen in the following ways:

• The compiler could constant fold x

to 42. This will happen in the example if we removevolatile

and use any optimization level above-O0

. Thenx = 42

gets printed. - Say that constant folding doesn't happen, maybe because we insert a asm("" : "+r"(x))

right after the definition ofx

. In that case, the compiler could register-allocatex

in a callee-save register, in which case the register ends up saved bysetjmp

. This also leads tox = 42

being printed. - Say that we experience register pressure for some reason, and x

doesn't make it into a callee-save register, but instead gets spilled. At any optimization level above-O0

, the compiler will splitx

into two variables: one forx = 42

and one forx = 666

, and the printf will reference the first one (sincex = 42

dominates theprintf

). Those two variables willalmost alwaysget separate spill slots. Hence, when we come out of thesetjmp

the second time, readingx

will still give 42.

Three things to reflect upon:

• To get the property that x

's value is observed to be 666 in the printf, we need to make sure that the compiler treatsx

as a stack allocation rather than a variable. Usingvolatile

achieves this. Also, passing a pointer tox

to anywhere is likely to accomplish this. - Spill slots are not the same as stack allocations. If a variable is stack-allocated, then it will get one stack allocation. If a variable is spilled, it may get multiple spills (often, a separate spill per assignment).

• The compiler is allowed to analyze the lifetime of spill slots and stack allocations. It's allowed to reuse spill slots. How does the compiler know that the x = 42

spill slot should stay alive until thelongjmp

happens? How come it won't get reused, resulting inx

having either 666 or any random garbage when we fall out of thesetjmp

a second time?

Here's a more diabolical version of the example that triggers spilling of x

to two different spill slots (one for 42 and one for 666) in gcc, clang, and filcc.

#include <setjmp.h>
#include <stdio.h>
int main(int argc, char** argv)
{
int x = 42;
asm volatile("" : "+r"(x));
jmp_buf jb;
int a = 1, b = 2, c = 3, d = 4, e = 5, f = 6, g = 7, h = 9, i = 10;
/* Force some spilling */
asm volatile("" : "+r"(a), "+r"(b), "+r"(d), "+r"(e), "+r"(f), "+r"(g), "+r"(h), "+r"(i));
if (setjmp(jb)) {
asm volatile("" : "+r"(a), "+r"(b), "+r"(d), "+r"(e), "+r"(f), "+r"(g), "+r"(h), "+r"(i));
printf("x = %d\n", x);
return 0;
}
x = 666;
void (*jump)(jmp_buf, int) = longjmp;
asm volatile("" : "+r"(x));
asm volatile("" : "+r"(jump), "+r"(a), "+r"(b), "+r"(d), "+r"(e), "+r"(f), "+r"(g), "+r"(h), "+r"(i));
jump(jb, 1);
asm volatile("" : "+r"(jump), "+r"(a), "+r"(b), "+r"(d), "+r"(e), "+r"(f), "+r"(g), "+r"(h), "+r"(i));
asm volatile("" : "+r"(x));
printf("Should not get here.\n");
return 1;
}

This program will print x = 42

even though x

is not constant folded or register-allocated.

Note that all of the examples so far work in Fil-C. Even the inline assembly that we're using to obfuscate variable values works in Fil-C, and has the desired effect.

What Is Even Happening

Let's take a look at how simple setjmp

is by looking at the musl implementation on x86_64:

__setjmp:
_setjmp:
setjmp:
mov %rbx,(%rdi) /* rdi is jmp_buf, move registers onto it */
mov %rbp,8(%rdi)
mov %r12,16(%rdi)
mov %r13,24(%rdi)
mov %r14,32(%rdi)
mov %r15,40(%rdi)
lea 8(%rsp),%rdx /* this is our rsp WITHOUT current ret addr */
mov %rdx,48(%rdi)
mov (%rsp),%rdx /* save return addr ptr for new rip */
mov %rdx,56(%rdi)
xor %eax,%eax /* always return 0 */
ret

This is only saving the callee-save registers, plus the stack pointer and instruction