Undefined behaviour under Fil-C, and nanofil's choices

Fil-C does not give C's undefined behaviour a meaning in general. It guarantees memory safety whatever the program does (GIMSO, gimso_semantics.md in the reference source), and it takes away the licence LLVM's optimizer would otherwise read into UB. This page states what that amounts to for each common kind of UB, with the source that causes it and what probes/ub.c observed, then where nanofil agrees and differs. Fil-C's own documentation covers pointers and capabilities but says nothing about integer UB; everything here comes from the pass source and probes.

Observed at staging af813053ecb5 (lib/filc-staging.json), reading nix build .#lib.x86_64-linux.filc-staging-ref-src, x86-64, October 2026.

Where UB is removed: KillUB, before any optimization

clang/lib/CodeGen/BackendUtil.cpp registers one pipeline-start callback, so this order holds at every -O level:

clang Sema + CodeGen (unchanged clang 20: constant folding, ICE rules)
        │
        ▼
KillUB ──▶ (O1+) SROA, EarlyCSE, inliner with InstCombine/GVN/SCCP/CVP…
        │
        ▼
CoroSplit ──▶ FilPizlonator (checks, flight pointers, frames)
        │
        ▼
the ordinary LLVM -O pipeline ──▶ DeleteRedundantPollchecks

llvm/lib/Transforms/Instrumentation/KillUB.cpp (107 lines; its own comment: "This is just the beginning! There are probably more UB killings that have to happen.") does four things to every function:

Since deluge (upstream 2de132744209) it also defines integer division: it selects a dividend of 0 and a divisor of 1 for a zero divisor, and a divisor of 1 for sdiv MIN, -1, so quotient and remainder are 0, or MIN and 0. It masks shift counts by the width and clears ctlz and cttz's zero-is-poison flag. It does not change the other instruction semantics that are UB in LLVM by themselves: fptosi out of range (poison), unreachable, undef from an uninitialized scalar. A reached unreachable that survives to pizlonation becomes a Fil-C panic, "llvm unreachable instruction" (FilPizlonator.cpp, the UnreachableInst case), as does llvm.trap. Optimizations that run before pizlonation at -O1 and up may still exploit these.

After pizlonation LLVM may again infer nsw/nuw from facts it proves itself (the probe's loop counter gets add nuw nsw). Those are derived, not read from the source's UB.

Memory safety does not depend on any of this: the pizlonator checks whatever IR it receives. The probe unreach-idx shows the distinction. At -O2, if (i >= 4) __builtin_unreachable(); return g[i]; with i = 10 loses its unreachable panic, because the optimizer exploited it, but the bounds check still panics.

What each kind of UB does

-O0 and -O2 are filcc; progtest.sh and the corpus compare against filcc -O1, which behaved as -O2 on every probe below.

C11 references are to the paragraph making the behaviour undefined. Memory-safety violations (bounds, null, use after free, rights) are always panics; that is the GIMSO guarantee, not an optimization choice.

Constant expressions are clang's

Fil-C leaves clang's Sema unchanged. An integer constant expression is judged and folded exactly as by upstream clang 20: _Static_assert, case labels and array sizes reject a signed overflow or an oversized shift (6.6p4), while a static initializer folds it with a warning (the default, non-pedantic mode). Signed overflow then wraps, and an oversized shift count is clamped to width − 1 (1 << 32 folds to INT_MIN), which differs from the masked runtime result above.

arith.md measures nanofil's const_eval against this. nanofil refuses an oversized shift or an undefined quotient in every constant context, and wraps signed overflow without a diagnostic (its fold_extension class). The wrap agrees with what Fil-C computes, both when folding a static initializer and at run time. Only pedantic clang rejects it.

nanofil's position

nanofil's IR has no poison, undef or UB flags. Integer operations wrap at their class width, shift counts are masked as on x86, and division is total, as Fil-C's is since deluge: x / 0 and x % 0 are 0, and MIN / -1 wraps to MIN with remainder 0. Only an invalid float to integer conversion is undefined in the interpreter. Memory is bytes, with no type-based aliasing. Allocations start zeroed. This is Fil-C's behaviour at -O0, and it is what KillUB makes of the source's UB. Where Fil-C's later optimizations still exploit LLVM's own UB (uninitialized scalars, unreachable), nanofil stays deterministic.

So programs that read an uninitialized scalar or reach __builtin_unreachable can differ from the -O1 reference in progtest.sh and the corpus. That is a difference in the reference, not a nanofil bug. Such a test is not a valid comparison.

Differences worth closing or keeping in mind:

• A constant LONG_MIN / -1L. clang folds a division of constants while it emits IR, before KillUB, and the overflowing one becomes poison: filcc prints 0 for long b = LONG_MIN / -1L; in a function body, while the same division of run-time values gives LONG_MIN. nanofil gives LONG_MIN for both.

• A reached unreachable panics as clang -O0's does. filc_amd64.pl's jump(unreachable, ...) calls filc_error with clang's message and the frame's origin (it was ud2, SIGILL, before October 2026). In Fil-C mode the difftest compares it as a trap (unreachable_arm in tests/filc/basic.ir). The plain SysV backend still emits ud2.

• Constant contexts: const_eval wraps signed overflow where C11 6.6p4 requires a diagnostic. This matches clang's default folding of static initializers; making it ub would refuse initializers clang accepts with a warning.

• A pointer rebuilt from integers. An integer cast to a pointer has the capability of the one pointer it was computed from, and none if it came from no pointer or from two. Fil-C infers this (inferPointerAsIntLaunderingForFunction) on LLVM's SSA after clang's optimizer, so filcc's outcome depends on the level: at -O0 an integer kept in a local comes back from a load and has none, and at -O1 a select clang makes an llvm.umin call has none either. nanofil applies the same rules to the SSA form of the source program at every level (launder.pl), so a local variable, a phi or a loop keeps the source, and a min or max keeps the selected value's. What it does not follow is a call: libgcrypt's sexp_null_cond masks (uintptr_t)w in inline functions and casts back, and filcc -O1 keeps w's capability only because inlining puts the round trip in one function; filcc -O0 and nanofil trap in gcry_pk_decrypt. The libgcrypt port selects the pointer as a pointer under __FILC__ (ports/patch/libgcrypt-1.12.2.patch). libsodium's sodium_free finds its guard pages with (unsigned char *)((uintptr_t)p & ~page_mask); its port steps back from p by the page offset instead (ports/patch/libsodium-unprotected-ptr.patch). tests/c/prog/uintptr-masked.c, launder-ssa.c shows the trap against a -O0 reference.

• Not probed: infinite loops without side effects (KillUB removes mustprogress, so C11 6.8.5p6 is not exploited), signed overflow in ++ on a pointer-sized index, ctlz/cttz of zero from __builtin_clz(0), and atomics.

Reproduce

nix build -o /tmp/filcc .#filcc-staging
for O in O0 O2; do /tmp/filcc/bin/clang -$O -w nanofil/probes/ub.c -o /tmp/ub-$O; done
for t in inc neg shl shl-check div0 div-check min-1 tbaa uninit fp \
         unreach unreach-idx wrap-idx null; do
  for O in O0 O2; do /tmp/ub-$O $t; echo "$t $O exit $?"; done
done
# nanofil: (cd nanofil/src && ../bin/nanofil cc ../probes/ub.c </dev/null) >ub.s,
# then as and link with filcc as progtest.sh does.