The Gallina backend: a nanofil IR compiler extracted from Rocq
November 2026. A campaign plan, written before any of it is built. It extends the formal campaign, which today scopes "implementation correctness" (native lowering) out of proof. This campaign moves the lowering itself into Rocq, so that claim comes within reach later.
The ambition
nanofil's middle and backend, from the IR to machine code, become Gallina functions in the Rocq development. Extraction turns them into a standalone nanofil IR compiler: first OCaml, then C++ through Bloomberg's Crane, then that C++ built by Fil-C as library code a program can call at run time.
Four things are wanted from it:
- Speed. The Prolog compiler spends about 650G user
instructions on
sqlite3.cwithout-Oand about 2000G with it (perf.md, item 10). A compiler of this shape in a compiled functional language should need a small fraction of that. The speed is wanted for JIT use and for very large files, not for everyday C builds; the scope note below says why. - A JIT component. A program that generates code at
run time, such as pcre2, can hand the compiler nanofil IR instead of
writing machine code itself. The regex experiment already does this with
the Prolog compiler,
as,ldanddlopen; what it lacks is a compiler fast enough to wait for. Built by Fil-C, the compiler is also a memory-safe library such a program can link. - Less mirroring.
Syntax.v,Cfg.vandValidate.vtranscribeir.plandcfg.plby hand, and gates check that the two agree. A backend written against the model's own definitions has nothing to mirror, and each stage that moves lets its Prolog twin retire. - More proofs. A lowering that is a Gallina function can be the subject of a theorem. So can a frontend that targets the IR: a MetaRocq extractor from typed λ□ to nanofil IR could be reasoned about through the IR semantics the model already has.
Scope: the C frontend stays in Prolog. That is not because it is cheap. Measured at M1, the frontend is about 40% of the plain SQLite compile, and 325 of the 347 s the 281-module test corpus takes, because headers dominate an ordinary translation unit. So a perfect backend barely speeds up everyday C compiles.
The campaign's speed goal serves two other uses:
- JIT use, such as pcre2 patterns compiled from IR, where there is no C to parse and the backend is the whole compile;
- very large files, where the backend is more than half of it.
The scope is mbrock's, and so is the reason: "i excluded the c frontend from the campaign scope mostly because it's big and messy and i am kinda mostly driven to do all this from thinking about jit etc". The frontend is something to reconsider later, outside this campaign.
What exists
The Prolog pipeline after the frontend
Lines are wc -l of nanofil/src today. guide.md walks one function through all of it.
- Middle end,
-Oonly:inline.pl707,promote.pl901,copy.pl329,simplify.pl794,local_storage.pl134, withescape.pl283,alias.pl167 andopt.pl154 to drive them.cfg.pl599 gives them the CFG, dominators, loops and liveness. - The checked stage:
pizlonate.pl267 states one check before each access, aflightfor each pointer computation and a poll on each backward edge.ir.pl2859 holds the grammar andvalid_module/1.check_reuse.pl1196 and its companions (half_check.pl,schedule.pl,check_share.pl) are the optional--checksevidence passes. - The Fil-C backend:
filc_abi.pl510 plans each function's frame, roots, entry and exit actions and call layouts.frame_slots.pl261 places slots.filc_amd64.pl3718 emits instructions, access checks, function objects, data and TLS.amd64.pl850 holds the scalar selection (compute_op//5) thatfilc_amd64.plreuses.asm.pl230 is the line terms and the AT&T printer. - Registers:
amd64_regs.pl1119 allocates, andamd64_regs_check.pl1395 withamd64_contract.pl230 checks each allocation by translation validation.
Without -O the path is the reference
path: pizlonate, validate,
filc_abi:lower_function/5, then filc_amd64
with --registers=stack, where every temporary lives in a
frame slot, and asm:print_asm/1. Its other defaults are
--checks=reference, --branches=reference and
--memcpy=call. About 5,800 lines of Prolog are on it, most
of them in filc_amd64.pl.
The Rocq model
In nanofil/rocq/theories, on Rocq 9.1 with stdpp and
Iris:
Term.vis ground Prolog terms.Syntax.vis the checked IR's core subset as inductives (op,inst,jump,block,func, the module's items).Maps.vis name maps andCfg.vtranscribescfg_idoms/2anddom_intervals/3.Semantics.vis the small-step meaning (step,step_rel) overValues.v,Memory.v,Ops.vandIeee.v, with the runtime as the module type inRuntime.v.Machine.vis the machine the proofs run on.Validate.visvalid, the validator, andSoundness.vprovesvalid_no_uncheckedabout it.Transport.vdecodes a printed module intoSyntax.v(decode_module); a function outside the subset stays asIOutsidewith its term.Extract.valready extractsdecode_module,judgeandjudge_validto OCaml, with zarith integers and native strings.extraction/main.mlreads Prolog canonical terms by hand, andmakelinksnanofil-model.rocq/gallina.plprints a checked module as a Gallinatermfor theNAMECode.vfixtures.
The syntax covers the checked stage only. It has no unchecked
load or store, and no vectors,
cas and atomics, setjmp or
asm_fixed. Of Lua 5.4.7, every function but one decodes
into it.
The first version
The first version is the reference path and nothing else:
- the asm line terms and their printer;
- the frame and ABI plan;
- stack-homed emission for the model's core subset;
- pizlonation, with the model's
validbetween it and emission.
It reads IR and writes assembly text for GNU as, the same text the Prolog reference emission prints.
It leaves out:
- the whole middle end (inline, promote, copy, simplify, local storage, escape and alias analysis);
- the check-reuse passes, and emission of the reuse forms they make
(
renew,half_check,check_limitand the rest); - the
amd64_regsallocator, its two checkers and thetempsandpointersmodes, with--operands,--branches=selected,--memcpy=inlineand--share; - plain (non-Fil-C) mode, and every construct outside
Syntax.v's subset. A module with anIOutsidefunction is refused, and the Prolog compiler compiles it as before.
"The fast simple register mode", as I read it.
Prolog has three register modes today: stack,
temps and pointers. Only stack is
simple, and it is the reference every other mode is checked against, so
the first version ports stack. A register allocator that is
fast and simple by design (one pass, block-local or linear scan, no
translation checker because it is proved instead) is its own later
milestone, M7. It is not a port of amd64_regs.pl: that
allocator and its checkers are half of -O's compile
time.
Reusing the model's definitions
The backend imports the model; it does not copy it.
- The input type is
Nanofil.Syntax.func. There is no second inductive for the checked IR.class,width,op,value,inst,jumpand the data items are the onesSemantics.vgives meaning to andValidate.vjudges. - Names are
Term.name, maps areMaps.v, block order and dominators areCfg.v, and decoding isTransport.decode_module. - The pipeline calls
Validate.validitself. Its theorem then speaks about exactly the module the backend lowers. - What is new belongs to the backend alone: asm lines and operands, frame plans, and the emission functions.
The new files live in their own project,
nanofil/rocq/backend/, with
-R ../theories Nanofil and -R . Backend, on
the pattern of rocq/spikes. Its Nix check compiles against
the model's compiled theories and is not part of the quick
nanofil-rocq gate, so that gate does not grow. It touches
no file of the program logic or the libc contracts.
Two places need more than the model has, and both go to the model's owner as requests instead of being worked around:
- The unchecked stage. Pizlonation's input has
load,storeand unproven calls, whichSyntax.instdoes not have. It gets no types of its own: pizlonation rewrites the module's term, as pizlonate.pl does, and the model's decoder reads the result. SoSyntax.instand every proof by cases over it are untouched, and nothing mirrors them. M4 says why. - Constructs outside the subset. Vectors, atomics,
setjmpand fixed-register asm reach the backend only whenSyntax.vand the semantics gain them. The backend never invents a private constructor for one.
Checking it against the Prolog compiler
The Prolog reference emission is the oracle, and the standard is byte identity.
- Per stage. Each ported stage gets a Prolog command that prints its output as canonical terms: the asm lines, the frame plan, the pizlonated module. The extracted stage must produce the same terms, or the same text.
- End to end. For every module of the corpus, the
assembly from the extracted compiler equals
nanofil compile's. The corpus is what exists:tests/filc/*.ir,tests/c,tests/c/prog, random modules fromtests/gen.plby seed, and the whole programs (hello, zlib, bzip2, Lua, SQLite). - By behaviour.
difftest.shandprogtest.shrun with the extracted backend swapped in, for the day the two are allowed to differ. - Coverage is reported, not assumed. Each run prints
how many functions were compiled, and how many modules were refused for
an
IOutsidefunction.
Byte identity holds until the Gallina backend is the reference. From then on a stage's Prolog twin is retired instead of kept in step, and that is where the mirroring goes away.
The quick subset of these tests runs in the backend's own Nix check. The large programs run in the package checks, which already exist and already run after a push.
Extraction
OCaml first. It is the extraction Rocq ships, the
model already uses it, and the driver, zarith and the Nix build are in
place. The compiler's entry points are Gallina functions from text to
text, so the OCaml driver only reads a file and writes one. Text is
built as lists of parts and joined once, as asm.pl does:
SQLite's assembly is about 100 MB.
Then Crane. Crane is a Rocq plugin
forked from the OCaml extraction. By its README it emits C++23 that
needs no runtime system and manages memory with shared_ptr,
is LGPL-2.1, installs against Rocq 9.0 and is under active development
with parts still experimental. Known risks, each settled in M6 before
anything depends on it:
- our pin is Rocq 9.1.1, so the plugin may need porting or a second Rocq in the build;
- arbitrary-precision
Zneeds a C++ home (a small bigint, or checked fixed-width integers where a bound is proved); - reference counting has a cost OCaml's collector does not, and deep
recursion over long lists needs care in C++ as it did in OCaml
(
Extract.vrecords one such overflow).
M6 met all three, and its record under Milestones says how each was settled.
Crane is an alpha, and we expect to patch it. Its site says so, and it will not take all of Gallina. Three rules follow.
- The backend stays in a plain subset: first-order inductives,
structural fixpoints,
list,option,Z, strings and records. No coinductives, no universe polymorphism, no type classes or functors on the extracted path, and no local fixpoint that is mutually recursive with its enclosing function (Crane's manual lists that shape among those its loop transform declines). The OCaml extraction does not care, and any second extractor will. - Crane's own switches are tried first. Its reference manual has type
mappings (
Crane Extract Inductive),Set Crane Arenafor region allocation of recursive fields,Set Crane NonAtomicRcfor a non-atomic count in place ofshared_ptr,Set Crane Reusefor in-place reuse andSet Crane Loopifyto turn recursion into loops. A compiler that runs once per module or per pattern and then frees everything is the case arenas are made for. - Past the switches, it is our fork. A change to what Crane emits is a patch in a set we own, applied by the Nix build to a pinned Crane revision, and offered upstream where it is general. The differential tests are what keep a patched extractor honest: the C++ build must print the same bytes as the OCaml build.
The fallback is Gallina to C. If Crane costs more to bend than it saves, CertiRocq (formerly CertiCoq) compiles Gallina to Clight, which any C compiler takes, and brings a small C runtime with its own collector; the Codegen plugin is a second, narrower route to C. Either gives C that filcc can build, so the library and JIT steps below do not depend on Crane. Neither is tried unless M6 fails, and whether CertiRocq's runtime (its tagged values and its collector) is at home under Fil-C is the first thing to find out then.
The Gallina is written with the second extraction in mind from the
start: no OCaml-only Extract Constant on a hot path without
a C++ twin, and no hand-written OCaml that the compiler's result depends
on. The canonical-term reader in main.ml was such code.
Since M4 the standalone compiler reads IR text with
IrText.v, and the OCaml reader serves only the gates that
take Prolog's terms. Three character primitives of IrText.v
have OCaml definitions. At M6 they needed no C++ twins of their own:
Crane's mappings of nat_of_ascii, ascii_of_nat
and string_of_list_ascii cover them.
The path to a Fil-C library and a JIT
The first JIT needs no encoder and no new way to load code. The regex
experiment (its
README, pcre2-jit.md)
already runs one. Behind pcre2_jit_compile, a patched
libpcre2 prints a pattern's bytecode as terms and runs a compiler,
rxc, on a cache miss. The experiment's regex compiler,
itself Prolog, turns the terms into unchecked nanofil IR, then
rxc runs nanofil -O compile, as
and ld -shared. The library loads the shared object with
dlopen and keeps it in a cache directory. The Fil-C runtime
the code calls is already in the loading process, so the trusted runtime
does not change.
What that route lacks is a fast compiler. A miss costs 0.15 to 0.8 s
per pattern on SWI-Prolog, and nearly all of it is
nanofil -O compile: as and the link together
take under a tenth of a second even for a module of 36 functions. On
Fil-C Trealla, which is what a libpcre2 with its compiler inside would
run, the median is 2.4 s. sljit takes 0.1 ms. So the steps are:
- The extracted compiler in
rxc.nanofil-backend compiletakes the place ofnanofil -O compile. The regex compiler emits unchecked IR, so this needs pizlonation (M4) and nothing after it. The regex compiler itself stays Prolog for now (0.07 s for 36 patterns). - Code worth running. The first version's code is
what Prolog makes without
-O. M5 measured it on the experiment's nine benchmark patterns: 2.7 to 16 times slower than-Ocode, 6.2 times in the geometric mean, and still 1.5 to 3.5 times faster than PCRE2's interpreter. The regex compiler keeps a node's choice state inallocslots, and without promotion every use of one is a checked load or store. Promotion, copy propagation and the selection of branches bring the code to 2.1 times-O's, which is all that-Ogives with stack homes; registers are the other 2 times. Simplification, inlining, the check passes and the inlinememcpychange nothing for this code. M5b ported the first three, before Crane and before the allocator, andrxcnow runsnanofil-backend compile-promoted: the measured figure is 2.10. libnanofil, pizlonated. Crane's output is plain C++23, and Fil-C is a C++ compiler (clang 20 with libc++), so the extracted compiler is built with filcc and given a C interface: IR text or a term tree in, assembly text out. libpcre2 links it and calls it in the process; the hook still runsasandldand loads withdlopen. It is memory-safe against any input, by Fil-C, and checked by the same differential tests as the OCaml build. Crane's README asks for Clang 19 or later; whether Fil-C's libc++ has everything the output uses is found out in M8.
Two further steps are options for later, taken only if measurement asks for them:
- An encoder and a shared-object writer in Gallina,
in place of
asandld, checked against them byte for byte. The result is still loaded withdlopen, so this too leaves the runtime alone. It removes two process starts per miss and the need for binutils where the program runs. - Installing code in the process, without a file.
Fil-C's
mmaprefusesPROT_EXEC, so a program cannot run bytes it wrote. The pinned staging runtime has one JIT (filc_llvm_jit.c,filc_llvm_jit.h), which compiles LLVM IR built through a curated C API and places the code in its own heap; it loads LLVM into the process and is refused to setuid programs. A nanofil entry of that kind would be a change to the trusted runtime. It would get a design note and an Oracle review before any code, and it is mbrock's decision.
The dlopen route trusts the cache directory, as the
experiment's README says: what is in it is loaded as code. The validator
is the asset on the compiler's side. valid is already
Gallina, already extracted and already proved
(valid_no_unchecked), so the compiler checks a client's IR
after pizlonation and before emitting anything.
Performance targets
These are targets. The figures known before the campaign are perf.md's:
sqlite3.c, 3106 functions and 218k checked instructions: 651.6G user instructions without-O, about 2000G with it. In a loaded run emission was 33 s of the plain compile and 151 of 269 s at-O.- Across 909 translation units,
-Ocosts 2.17× plain.
Targets, each measured in perf stat user instructions on
the same input and the same stages:
- OCaml, M3: the ported stages take at most a
twentieth of the Prolog stages' instructions on
sqlite3.cand onlvm.c. - OCaml, after M5: at most a fiftieth, with checked IR to assembly for all of SQLite inside 10G instructions.
- Crane C++: within 2× of the OCaml build.
- Pizlonated: within 4× of the unsafe C++ build, the top of Fil-C's usual range.
- As pcre2's compiler: IR to assembly for one pattern
in at most 10 ms, against 0.15 to 0.6 s in Prolog, so that a miss is
bounded by the regex compiler,
asandldat about a tenth of a second. The code's speed has its own target once M5 has measured the passes: within 2× of-Ocode on the experiment's nine benchmark patterns.
A stage that misses its target gets a profile and a note of where the time goes before the next stage starts.
The baseline, measured at M1. CPU seconds of the plain compile in SWI-Prolog, stage by stage, on one core of an orb (a 2.6 GHz Xeon):
- SQLite 3.51.2's amalgamation with
nix/sqlite.nix's flags, preprocessed: 141,500 lines in, 5.52 million asm lines and 135 MB out. Frontend 35.9, pizlonate 6.3, validate 13.6, emit 20.6, print 12.8. The stages this campaign ports are 53.3 of 89.2 s. - The 281 modules of the test corpus together: frontend 325.5, pizlonate 2.3, validate 5.6, emit 9.9, print 3.2. Their headers dominate, so the corpus tests sameness and SQLite tests speed. With the frontend at 94% of the corpus's time, no backend makes those compiles much faster.
At M1 the extracted printer printed SQLite's lines in 5.0 s, which is 2.5 times Prolog's speed, within 230 MB. Two things follow:
- Printing is where Prolog is nearest to native code, because SWI writes its text in C. The Gallina printer makes a string per digit and a list cell per part. So this ratio says little about emission.
- A twentieth of 53.3 s is 2.7 s, and the printer alone takes 5.0. The M3 target cannot be met unless the printer gets faster too, so M3 profiles it first and M5 owns what remains.
The profile, at the start of M3.
perf record with DWARF call graphs, on SQLite, in CPU
seconds: an orb has no hardware counters, so instructions are not
counted here. The targets for the two parts are a twentieth of Prolog's
stages: 0.64 s for printing, and 1.0 s for the plans and the emission
together.
The printer went from 5.0 s to 1.1 s:
- To 2.1 s by reading 64 lines at a time and not 4096. The terms, lines and parts of a large chunk outlived OCaml's 2 MB minor heap and were copied to the major one. This was the driver's fault and not the printer's.
- To 1.8 s in
decimal. The extracted code builds a number or a list written inside a function again at each call, so every digit built the number 10 three times and the list of ten digits once. They are definitions of their own now. The bound on the digits was the count of the number's bits, taken a bit at a time; it is 18 for numbers below 10^18. - To 1.5 s with one division per digit, and with zarith's conversions
between
Zandnatwhere the stdlib's count by ones. - To 1.1 s by filling a buffer per chunk; each part was a write.
What remains is a list cell per part and a string per digit.
The plans went from 7.8 s to 2.6 s:
- To 3.7 s by taking OCaml's hash of a string for a name's bucket, as the model's extraction does. The Gallina hash is eight integer operations per character.
- To 2.6 s with a 32 MB minor heap in the gate.
Of what remains, liveness is a third, the first and last touch of
each temporary a quarter, the fusing of checks a fifth, and the lowering
of instructions a tenth. No one function stands out. About half of all
of it is finding and adding names in maps: a name is a term such as
tmp(v12), hashed and compared as one at every step. The
repair is to number a function's temporaries once and key the maps and
sets by number, which changes every pass of M2 and belongs to M5. The
gate's 1.4 GB for SQLite is nearly all the module read as terms, which a
reader of NFB words would not build.
Both parts still miss their targets, by less than twice for printing and by more than twice for the plans before any emission. M3's emission is written with this in mind, with no number or list written inside a loop.
The profile, at the end of M3. The same tool on
nanofil-backend module, which reads SQLite's module, lowers
it, emits it and prints it in 8.8 s. As shares of that time:
- Reading the module as terms is 28%, and nearly two thirds of that is the collector. A reader of NFB words builds the module with no terms between; it becomes the compiler's input once it is faster than reading text.
- Building the plans is 24%. Slots are 16%, nearly all of it the first and last touch of each temporary (15%) and the liveness under that (8%); fusing is 5%. This is the part that waits for numbered temporaries.
- Emitting the functions is 12%: 7% for the instructions' lines and 4% for a function's context, which builds maps keyed by a name's term. Data and thunks do not show.
- Printing is 12%.
- The collector is 27% of everything, the minor one 15% and the major one 12%.
Inside the instructions' lines no one form stands out. The leaves are string concatenation (17%), the hash and comparison of a name (about 40% with the maps' lookups), and list append (10%). So the emission's own cost is names again: a slot is found by the term of its temporary, and a label is built from strings at each use. Numbered temporaries remove the first, and a label kept as parts until it is printed removes the second.
The measurements of M5. On the standalone compiler,
which compiles SQLite's IR text in 41 s where
nanofil compile takes 70.
The validator is the cost. The model's valid is
18 to 20 s of the 41, where Prolog's own validator takes 14. It grows
with the square of a function's definitions: two thirds of it is in
three functions of 6,000 to 14,800 definitions.
single_definitionsis 7.0 s. For each definition it filters the list of all of them.- The reuse forms' stage is 5 to 6 s more, and the blocks' own rules
4.2 s. Both find a name's definition and its class with
lookup, a walk along a list of pairs. - Dominance, the classes and the data are under a second together.
The validator is the model's, and nine proof files of the program
logic read single_definitions, so the repair is not this
campaign's to make alone. Decision 6 below is about it.
The first repair. theories/ValidFast.v counts
every definition in one pass into a map, and its valid_fast
is valid with the single definitions taken from that count.
valid_fast_eq proves valid_fast md = valid md
for every module, so valid stays the definition the proofs
and the gates read, and the compiler runs valid_fast. On
SQLite, pizlonation and validation went from 24.8 s to 17.2 s and the
whole compile from 41 s to 33 s, with the same text printed. The two
lookups are next; the blocks' own rules are 4.0 s of what
is left and the reuse forms' stage about 5.
Numbered temporaries gain nothing. A pass that numbered a function's temporaries and gave the plans integer names was written and measured: the validator and the plans took the same time. What they pay for is the walk along a list, not the comparison of two names at each step. So the repair is maps in place of the lists, with whatever names, and the pass is not in the tree.
pcre2 through rxc. With
NANOFIL_BACKEND set, rxc and the experiment's
run.sh compile the IR with
nanofil-backend compile, and no Prolog runs after the regex
compiler. The experiment's checks pass: the module of 36 functions is
Prolog's plain compile byte for byte, bench agrees with
PCRE2 on every pattern, and hook compiles its 39 patterns
and agrees on all of them.
Compile time, on an orb:
- The module of 36: 0.39 s, against 1.4 s for
nanofil compileand 6.3 s fornanofil -O compile. - A cold cache of 39 patterns through the hook: 4.5 to 4.7 s, against 12.8 s. That is 120 ms a miss against 330.
- One pattern, for a small, a middling and a large one: the regex
compiler 44 to 58 ms, the backend 16 to 43 ms,
as7 to 12 ms andld17 to 19 ms.nanofil -O compiletakes 214 to 548 ms for the same three. - The large one's 43 ms are 2 reading, 24 pizlonating and validating, 10 lowering and emitting, 1 printing, and 6 outside those.
So the 10 ms target is missed by two to four times, and validation is again the largest part. A miss is now bounded as the target wanted, by the regex compiler in SWI-Prolog and three process starts, at about a tenth of a second.
With M5b's passes. rxc and run.sh
now run nanofil-backend compile-promoted. For the module of
36 it prints 45,702 lines, nanofil compile-promoted's byte
for byte, in 0.35 s, against 1.4 s for that command and 6.2 s for
nanofil -O compile; the first version's text is 99,256
lines, and -O's 28,580. Per call, on the nine benchmark
patterns and as a multiple of -O's time, the code takes
2.10 in the geometric mean (1.56 to 2.91), where the first version's
takes 6.0 (2.8 to 16): the figure the Prolog selection measured below.
That is 3.0 to 11.8 times faster than PCRE2's interpreter. A cold cache
of 39 patterns through the hook takes 4.1 s, 106 ms a miss, against 4.5
to 4.7 s with the first version. bench and
hook agree with PCRE2 on every pattern.
The code's speed, and which passes it needs.
nanofil compile on the module with one selection of
-O's parts at a time, then bench. The figure
is the geometric mean over the nine benchmark patterns of the time per
call, as a multiple of full -O's:
- the first version's code: 6.2 (2.7 to 16);
- with selected branches: 5.5;
- with registers and selected branches, and no middle end: 4.0;
- promotion and copy propagation: 2.7;
- those and simplification: 2.6;
- those and inlining: 2.5, with the same count of assembly lines;
- promotion, copy propagation and selected branches: 2.1 (1.5 to 2.9);
- all of
-Obut with stack homes: 2.2 (1.6 to 2.9); - promotion, copy propagation, simplification and registers: 1.6;
- those and selected branches: 1.03;
- those and the combined checks: 1.08, and the inline
memcpy1.00; - all of
-Owithout simplification: 0.97.
Four things follow:
- Promotion with copy propagation is worth 2.3 times, registers 1.7 to 2 times after it, and branch selection 1.1 to 1.6 times, the more the less other work is left.
- Promotion, copy propagation and branch selection are everything
-Ogives with stack homes. Simplification, inlining, the check passes and the inlinememcpydo nothing for regex code. - The experiment's README had stack homes at 1.1 to 1.7 times
registers. Measured again it is 1.6 to 2.9, so the target of code within
2 times of
-O's waits for M7's allocator on most patterns. - In Prolog the registers are also what compiling costs: the module takes 1.5 s with stack homes and 6.6 s with registers.
The C++ build, measured at M6. The target was within
2× of the OCaml build, and it is missed: Crane's C++ takes 5 to 7.5
times the OCaml build's time and up to 4 times its memory, and it is
slower than Prolog. Wall time and peak memory on an orb, IR text to
assembly, for the regex module rx.ir (36 functions) and for
Lua's lvm.c as IR; each build prints the same bytes:
compile,rx.ir: Prolog 1.27 s and 43 MB, OCaml 0.34 s and 57 MB, C++ 2.40 s and 76 MB.compile,lvm.ir: Prolog 2.87 s and 162 MB, OCaml 1.76 s and 153 MB, C++ 8.36 s and 577 MB.compile-allocated,rx.ir: OCaml 0.39 s and 58 MB, C++ 2.87 s and 76 MB.compile-allocated,lvm.ir: OCaml 1.67 s and 153 MB, C++ 9.53 s and 501 MB.compile-promotedon one function of 19 kB of IR: OCaml 37 ms, C++ 164 ms. An empty module takes 3 ms in both, so none of it is start-up.
Where the time goes, by perf record on the C++
build:
- About 15% is freeing. A
Termis a reference-counted tree, and dropping the last reference walks it. With~Termemptied by hand, as an experiment, the twocompileruns took 2.05 s and 7.22 s. The build keeps the destructor: without it nothing is freed, and the target is missed either way. - With the destructor out of the profile, about 15% is copying and
moving
Termvariants and 7% ismemmove, and some parameters still go by value. - The rest has no single hot spot.
Set Crane Arena was tried and is worse: slower, and ten
times the memory. So no switch closes the gap, and what would is more
patches to what Crane emits for values, matches and calls. That work is
proposed for M8, where the same C++ is built by filcc and its speed
first matters to a user; until then the OCaml build is the fast one.
Milestones
Each ends with its gate green and a short note of what it found.
M0. The plan and the project. This document.
rocq/backend/ with its _RocqProject, a build
script, extraction to a nanofil-backend binary and a Nix
check outside the quick gate. Gate: the empty project builds, extracts
and passes parse.sh.
Done. make -C rocq/backend builds
nanofil-backend in two seconds. It compiles only the model
files the backend imports, which is Term.v so far, so it
does not wait for the model's proofs. The flake checks are
nanofil-backend, which takes ten seconds once its inputs
are built, and nanofil-backend-full. They have their own
workflow, and the quick Rocq gate does not read the directory.
M1. Asm lines and the printer. Asm.v:
lines, directives, operands and expressions as inductives, a decoder
from terms, and the printer. A Prolog command prints a module's line
list as terms. Gate: for the corpus, the printed text equals
nanofil compile's, byte for byte; the per-stage baseline is
measured and recorded here.
Done. The command is
nanofil asm-terms FILE TEXT, which writes the text and
prints the lines it is the text of. Asm.v has the types and
the printer, AsmTerm.v the decoder, and
rocq/backend/tests/asm.sh is the gate. The text is the same
for all 280 Fil-C modules of tests/filc,
tests/c and tests/c/prog, with and without
-O, and for SQLite's 5.52 million lines. Seven files are
skipped: six that nanofil refuses without their test's flags, and one
plain module. Found:
- A
matchon string literals extracts to a decision tree per bit: the decoder was 11,701 lines of OCaml. Names are compared withString.eqbinstead, and it is 1,713. - A driver that holds a module's terms, lines and parts needs 4 GB for SQLite. It now works a chunk of lines at a time.
- The baseline is under Performance targets above.
M2. Frames and the ABI plan.
filc_abi.pl and frame_slots.pl: slots, roots,
entry and exit actions, call layouts, signatures. Gate: the plan of
every corpus function in the subset equals Prolog's, as terms.
Done. The command is nanofil abi-terms FILE,
which prints the checked module, the plan of each function and the plans
of its weak thunks. Plan.v has the plan's types, whose
instructions are Syntax.v's. Fuse.v joins a
check to its access, Liveness.v and
FrameSlots.v give temporaries their slots,
Abi.v builds the plan, and Order.v is Prolog's
standard order of terms, which the plans are sorted by.
IrTerm.v and PlanTerm.v write a plan as
Prolog's term, and rocq/backend/tests/abi.sh is the gate.
Only the reference path is compared, without -O.
On the 280 modules of the corpus, the plans of 1089 functions and all
280 lists of thunks are the same terms. The backend refuses no function
of the subset. The model keeps 576 functions outside it, and they are
passed over: 535 for touch, 12 for asm_fixed,
11 for fence, 9 for setjmp and 9 for
raw_cas. On SQLite all 3106 plans and the thunks are the
same, with nothing outside. Found:
filc_abi.pl'sadapter_return/2zeroed too few aux words when a tuple with a float came back. Its test for a float leftKbound to the first float's class, and the list of zeros then kept only the cells of that class. Fortuple([l, s])the adapter zeroedaux(1)and leftaux(0)as it was, which let a caller reading the result as a pointer use a stale capability. Prolog is fixed (21ca173), andAbi.v, which had followed it to give the same plan, now zeroes the aux of every cell that holds no pointer.touchis the pizlonatedaccess, and it keeps a third of the corpus's functions out of the gate. It is the first construct to ask the model's owner for.- Building SQLite's plans took 7.8 s in OCaml, more than the whole M3 target of 2.7 s. The profile under Performance targets brought it to 2.6 s and says where the rest goes.
M3. Stack-homed emission.
filc_amd64.pl's reference emission and
amd64.pl's scalar selection, in pieces: scalars, compares
and jumps first; then checks, flights, raw accesses and polls; then
calls, variadics, function objects, data, aliases and TLS. Gate:
whole-module byte identity on the corpus, hello, zlib and Lua, with the
first performance target.
So far: every list of every module, and whole modules. The
command is nanofil emit-terms FILE, which prints the
checked module and then the reference emission's lines as terms: a list
for each item, for each weak thunk and for the end of the file.
Layout.v has a function's context and the loads and stores
of homes. EmitOps.v is amd64.pl's scalar
selection, with SSE2 and x87. EmitInst.v emits one
instruction of a plan: lets, span tests, flights, fused and reused
accesses, pointer effects, named locals, calls and the ABI's actions.
EmitObject.v emits a function's getter, generic entry,
variadic adapter and object, and Emit.v the function.
EmitData.v emits a data object with its relocation table, a
thread-local object, an init entry, a weak declaration and an alias's
getter. EmitThunk.v emits a weak thunk, and
EmitModule.v puts the lists in the file's order.
rocq/backend/tests/emit.sh is the gate. It emits each
list again from the module and compares it, as text, with the
reference's. Then, without -O, it compares what
nanofil-backend module prints for the module with what
nanofil compile prints for the file. For a preprocessed
.i file the text is nanofil cc's, which is
what nanofil-cc runs on a package's translation units.
Emission is total: a form that is not ported gives a todo
line, and the gate counts such a list as partial. None is left. The
todos that remain in the code stand where Prolog itself fails or
throws.
- On the 280 modules of the corpus, all 1090 functions of the subset,
all 2394 lists of data and aliases and all 1630 weak thunks are the
same. The 576 functions outside the subset have no plan and are passed
over. The 237 modules with no such function are
nanofil compile's text byte for byte, and the other 43 have no whole text yet. - On SQLite all 3106 functions, 1843 data lists and 181 thunks are the
same, and the whole module is
nanofil compile's 135 MB of assembly byte for byte. - zlib, GNU hello and Lua are built through
nanofil-ccwith each preprocessed translation unit kept, and each unit's text is compared. All 17 units of zlib and all 81 of hello arenanofil cc's text byte for byte, and 33 of Lua's 34. The one left isldo.c, whoseluaD_rawrunprotectedcallssetjmp; its other lists are the same. The three have 1,481 functions of the subset, 1,327 data lists and 1,385 thunks between them. The full check builds and compares them; it takes six minutes on 16 cores. - In OCaml, SQLite's plans take 2.8 s and its functions' emission 1.5
s.
nanofil-backend modulelowers and emits the whole of it in 3.9 s and prints it in 1.75 s, within 1.7 GB, where Prolog's emission stage takes 20.6 s. The target for lowering and emission is 1.0 s, so they miss it by four times; the profiles under Performance targets say where the time goes. EmitModule.vgives the module as pieces: one per item, one for the weak thunks and one for the end of the file. The driver asks for a piece, prints it and drops it, so a module's lines are never all alive. Before that SQLite took 6.0 s to lower and emit and 2.3 s to print, most of the difference in the collector.- With
-Othe lines of 538 corpus functions are the same, with all the data and thunks, and the backend refuses 256 when it builds their plans. Every one of the 256 has a reuse form (renew,renew_hull,renew_if,half_checkor a limit check), whichFuse.vleaves to M9, so they are not M3's. rocq/backend/tests/edgehas two fixtures for the arms the corpus does not reach.calls.irhas calls whose arguments end at cells 16, 17, 32 and 33, a pointer cell at exactly cell 32, results in a buffer and float tuples.data.irhas a weak read-only object, a negative addend, alignments of 64 and 128, thread-local objects with pointers, and every kind of init entry and alias. A mutation of the pointer cell limit in a thunk survived the corpus and is caught bycalls.ir.
What it showed:
- Porting a predicate at each of its call sites hides a wrong port.
result_registers/4gives a tuple's third float word an integer register once%xmm0and%xmm1are taken. Three function-side ports of it gave that word nothing, and the corpus has no such call. The thunk port followed the clause order, and a fixture then showed the difference. It is now one definition inLayout.v, shared by the function, the call, the generic entry and the thunk. - Some arms cannot be reached through the gate, because the validator
rejects their input: an alias with
locallinkage, an alias of a weak declaration, and a negativexconstant. An alias of a thread-local object was among them until Prolog began to take one;tests/edge/data.irnow has one of each linkage, since C cannot write the local one. - A data object of 200,000 items overflows the 8 MB stack in the
driver, with the stub emission as with the real one. With an unlimited
stack the lines are Prolog's.
payload_itemsis not tail-recursive either, so M5 looks at both.
What remains of M3:
- the functions outside the subset, which wait on the model:
touchfirst, andsetjmpfor Lua'sldo.c. With them come the 43 modules of the corpus that have no whole text.touchhas since come: the model hasTouch, the backend fuses it with its check asfilc_amd64.pldoes and emits the check alone, and 26 more files of the C corpus print Prolog's text; - the performance target. The profile says it is not met by the emission alone: it needs M5's numbered temporaries and a faster reader.
M4. Pizlonation and a standalone compiler.
Pizlonate.v and a Gallina reader for IR text. The pipeline
is read, pizlonate, valid, lower, emit, print. Gate: the
pizlonated module equals Prolog's as a term, and
nanofil-backend compile FILE.ir equals
nanofil compile FILE.ir with no Prolog in the pipeline.
Done.
Pizlonation is done. Pizlonate.v's
pizlonate_module takes a module's term and gives the term
pizlonate.pl gives: every access with its check, the flight forms, a
poll on each edge to a block at or before its source, the
loc markers of an unchecked module dropped, and the context
items dropped. It gives nothing where Prolog fails or throws.
- No
Unchecked.v. The plan was an inductive for the unchecked instructions. Pizlonation does not need one: it looks at a few instruction shapes and passes every other term through, so on terms it is 700 lines, andTransport.decode_modulethen reads its result intoSyntax.v's types, keeping a function outside the subset asIOutside. An inductive would have been a second copy of the instruction grammar, with a decoder and a printer to keep in step. - The gate is
tests/pizlonate.sh:nanofil pizlonate-termsprints a module and then pizlonate.pl's result, and the backend must make the second from the first. The corpus, with and without-O, the package translation units and SQLite are 857 modules and 18,191 items, all the same. SQLite's 4,949 items take 0.9 s, after 1.7 s to read the term. - Mutations. 36 wrong ports were tried, and an
Exampleor the gate catches 35. Nine of those got past the corpus alone, and each showed something the corpus does not have: a module that is in filc mode for one reason only (an external call, a weak function, read-only data, an alias, a local object), a checked module with alocmarker, and a namechk0taken by a phi or a parameter. Those are now the fixtures oftests/pizlonate/. The one left, keeping the context items, cannot show in this gate, which prints modules without them; the standalone compiler's gate reads them.
The standalone compiler is done.
nanofil-backend compile FILE.ir reads the text with
IrText.v and compiles the module with
Compile.v: pizlonate, read into Syntax.v's
types, the model's valid, lower, emit, print. It exits 1
for a module it refuses and 3 for one it does not take: a plain module,
whose backend (amd64.pl) is not ported, or one with a function outside
the subset.
- The reader takes what ir_text.pl's printer writes
and a little more: comments, names, quoted atoms and strings with their
escapes, symbol atoms, decimal integers, compounds, proper lists, and
one operator, the infix
-of a phi's edges. It gives nothing for the rest of Prolog's syntax. The rule is that it never reads a text differently from SWI-Prolog; it may refuse what Prolog reads. Every loop is a tail call, so a module of 200,000 items needs no more stack than one item. - The reader's gate is
tests/read.sh, with a new commandnanofil read-eachthat prints the module Prolog reads from each file. The fixtures and 387 texts of the corpus all read the same. Of 1,088 adversarial texts (escapes, quotes, UTF-8, numbers, operators, comments, clause ends), 626 read the same, both refuse 227, and the reader alone refuses 235.tests/read/refused.txtnames those 235, and the gate fails if the set changes either way. The largest groups are uses of-that are not a phi's edge, number forms other than decimal, and bytes outside ASCII that are not inside quotes. - A text in pieces. The driver gives the reader the
file a megabyte at a time, and a piece is a list of characters only
while its clauses are read. For SQLite's 78 MB of text this is 1.3 GB at
the peak in place of 2.8 GB, at the same speed. That reading in pieces
equals reading the whole list is not proved: two
Examples cut a text at every size from 1 to 40, and the gate runs the adversarial texts again in pieces of one byte and of seven. - The compiler's gate is
tests/compile.sh: the text printed must benanofil compile's, byte for byte, and each must refuse what the other refuses. The IR fixtures give 32 the same, 4 not taken and 7 refused by both. The C corpus and the package translation units give 374 the same and 24 not taken (17 with a function outside the subset, 7 plain); the frontend skips 7. The files not taken must be exactly the 28 thattests/compile/none.txtnames, so the gate fails when the backend stops taking a file, and when it starts taking a listed one. - Three known gaps, all on the side of taking less or
checking less than Prolog, none of which changes a line both print:
- A function's name must be an atom; a data object's may be any term, as in Prolog.
- The
context/2items are dropped unchecked. Prolog checks provenance's records before it drops them. - Prolog also runs
valid_module/1on the module as written. Here only the pizlonated module is judged.
- Mutations and fuzzing. Of 25 wrong readers, the
gates catch 24; the last showed that a test in the symbol atoms was
redundant, and it is removed. Of 6 wrong versions of
Compile.v, anExampleor the gate catches 4. The other two cannot show: one skips the early exit for a function outside the subset, which the driver then finds as a missing plan, and one asks an atom of every five-argument item, and onlyfunchas five. Of 5,876 mutated IR texts, 1,127 compile to the same bytes, both refuse 4,582, and 167 that Prolog compiles are not taken, as plain or outside the subset. No text differs, and the backend compiles nothing that Prolog refuses. - SQLite.
compileon SQLite's IR text prints Prolog's assembly, byte for byte, in 41 s and 1.3 GB, against 70 s and 2.1 GB fornanofil compile. Reading is 9.6 s, pizlonation and validation 24.8 s, lowering and emission 4.9 s, printing 2.0 s. Pizlonation is under a second of that, so the model's validator is more than half of the whole compile. It was written to be proved about, not to be fast, and it is M5's first target.
The driver's input stays text. An NFB reader is proved
(Nfb.decode), but its checked decoding of SQLite's two
stages took 17 s, where the text reader takes 9.6 s for the first and
the driver's OCaml reader of Prolog's terms about a second. It becomes
the input when it is the faster one. The NFB work now aims at decoding
straight into M5's numbered form: Syntax.v's own types with
a temporary named by an integer term, symbols keeping their names, and a
function outside the subset kept as a term.
The full check (nanofil-backend-full, about an hour)
runs only when asked, by workflow_dispatch; the quick one
runs on pushes that touch the backend's paths or the seven Prolog files
it must match (ir_text.pl, pizlonate.pl,
promote.pl, copy.pl, filc_abi.pl,
filc_amd64.pl and asm.pl), and on no
others.
M5. Speed, and pcre2 through rxc.
Profile, fix data structures, record the numbers in perf.md's style.
rxc gets a switch that runs nanofil-backend in
place of the Prolog compile. Gate: the second performance target; the
experiment's checks pass with the extracted compiler; the miss time and
the code's speed are recorded, with a list of the -O passes
that code needs.
Measured, and half done. The numbers are under Performance targets.
- Done: the
rxcswitch, the experiment's checks with the extracted compiler, the miss time, the code's speed and the list of passes. - Not done: the second performance target. After reading, SQLite takes 32 s, and the target is a fiftieth of Prolog's 53, about a second. The model's validator is 18 to 20 s of it and quadratic, and its repair needs the model's owner. Reading is 10 s more, and waits for the NFB reader. The plans and the emission still find names in lists.
- Dropped: numbering the temporaries, which measured as no gain.
M5b. The middle end pcre2's code needs. Promotion
and what it stands on: the CFG and dominators of cfg.pl,
escape.pl, promote.pl and
copy.pl, about 2,100 lines of Prolog. They run before
pizlonation, on the unchecked module, and their result is in the first
version's forms, so nothing after them changes. With them comes the
selection of branches in the emission (compare_branch and
selected_jump in filc_amd64.pl, under 100 lines): a branch
reads the flags of the comparison before it, and an edge to the next
block falls through. simplify.pl, inlining and local
storage are left out: M5's list says regex code does not need the first
two, and local storage changes nothing in the regex module. Gate: byte
identity on the corpus with the Prolog compiler run with those parts
alone (opt.pl's passes([promote, copy]), which leaves local
storage off, with
--registers=stack --branches=selected --memcpy=call --checks=reference),
and the regex experiment's patterns at the 2.1 times of full
-O that this selection measures in Prolog. This is the
first point where the rxc route is worth switching on.
- Reached: copy propagation.
Copy.vmakes of a promoted module the termcopy.plmakes of it, onTermCfg.v's blocks, CFG and strict-SSA check, which reuse the model'sCfg.vdominators.tests/promote.shholds it tonanofil promote-terms, which prints the module, its promotion and the copy pass's result: the same on every file of the corpus nanofil accepts, where copy.pl changes 1,034 functions in 221 files. - Reached: escape analysis and promotion.
Escape.vfinds the allocs whose address does not escape and their accesses, andPromote.vmakes of a module the termpromote.plmakes of it: the slots of an alloc as temporaries, with pruned phis, punned reads as shifts,laccesses split into words, and float slots only where a store comes before every load. The same gate holds it to the second termnanofil promote-termsprints: the same on every file of the corpus nanofil accepts, where promote.pl changes 1,384 functions in 261 files, and on all 4,949 items of SQLite, which it promotes in 2.8 s. The fixtures oftests/promotereach each arm in the quick check; 13 mutants ofPromote.veach make one of them differ. - Reached: the selection of branches.
Emit.v'sfunction_code truegives a branch the flags of the comparison before it, where that comparison's result has no other use, and lets an edge to the next block fall through, ascompare_branchandselected_jumpdo. 40 mutants of it each make a file of the quick check differ, and the fixturetests/promote/selected.irreaches each of its cases. - Reached: the byte gate.
nanofil-backend compile-promotedruns promotion, copy propagation and the selected emission, andtests/compile.shwithCOMMAND=compile-promotedholds its text to that of the newnanofil compile-promoted, which ispasses([promote, copy])andemit(stack, selected, call)with the reference checks. The same on 376 C programs and translation units (2,527,096 lines) and 38 IR fixtures; the files not taken are those ofcompile/none.txt, and none differs. - Reached: the regex figure, 2.10 times full
-Oper call, and therxcswitch now runs this compiler. The numbers are under Performance targets.
M6. Crane. Build Crane in Nix against our Rocq, extract the same entry points to C++, compile with clang++ and run the same differential tests. Gate: byte identity again, and the C++ target. If Crane cannot extract the development, the note says what stopped it, and the order of remedies is: change the Gallina, use a Crane switch, patch Crane, fall back to CertiRocq.
Done, but for the speed target. Crane at commit
c2aadd9, with a patch of ours, extracts the reader of IR
text, compile, compile_promoted and M7's
compile_allocated to 36,106 lines of C++23. clang builds
them with a driver of 127 lines as nanofil-backend-cpp,
which prints what Prolog and the OCaml build print, byte for byte. It
takes 5 to 7.5 times the OCaml build's time, so the 2× target is missed;
the figures and the profile are under Performance targets. CertiRocq was not
needed.
- The gates.
tests/compile.shholdscompileandcompile-promotedto Prolog: the same bytes on 376 C programs and translation units and on 38 IR fixtures, with no file differing. That run was before M7's files joined the extraction; the check repeats it on the fixtures with them. Prolog has nocompile-allocated, socrane/tests/same.shholds that command to the OCaml build's exit status, output and message: the same on 252 translation units of the corpus as IR and on 39 fixtures, the other 18 and 13 not taken or refused by both for the same reason, and no file differing. - The mappings.
CraneMap.vgivespositive,NandZtonf::Zofnfrt.h, a 64-bit word that becomes a GMP integer when a result does not fit;nattouint64_t,asciitocharandstringtostd::string.CraneSelfTest.vholds every mapped function to whatvm_computefinds for it, on numbers at and past 64 bits, andselftest.cppruns that in the check. - The check.
nanofil-backend-cranein the flake (nix/backend-crane.nix) builds Crane (nix/crane.nix), extracts, compiles, and runs the self-test and the three gates on the IR fixtures andtests/regs, and reads the Crane files back withparse.sh. It needs no Fil-C toolchain. On 16 cores it takes nine minutes: two to extract, four for clang on the one translation unit, and two forparse.sh, which extracts once more. So it is a job of its own in the Nanofil Backend workflow and no part of the quick Rocq gate.
What stopped Crane, in the order of the remedies:
- Changed in the Gallina. Three names whose C++ did not
compile (
fixedandtextrenamed,namesmade a notation), two local functions made top-level, andOrder.v's four merges written as top-level fixpoints on a count of steps. Each merge was a fixpoint over one list holding a local one over the other, a shape Crane's loop transform declines, and the recursion overflowed the C++ stack inAbi.frame_of. Prolog's bytes still match after each change. - Switches.
Loopifyturns recursion into loops, andNonAtomicRcmakes the reference counts plain integers.NoLoopifyis set forterm_eqbandOrder.term_compare, for which Loopify's C++ names a function it never declares. Loopify still declines 12 functions, which stay recursive: the stdlib AVL map'sjoin,map2,combineandequal, the validator's three walks along a chain of definitions (bounded by fuel), the two walks of a dominator tree, andAbi.one_per_symbolover a module's thunks. None has overflowed on the corpus.Arenawas tried and rejected. - Patched in Crane (
nix/crane-nanofil.patch, 705 lines in 17 files):- the port to Rocq 9.1, which is API changes in six files;
- a string literal is one
std::stringwith its length and octal escapes, and a character literal is escaped the same way; - Loopify skipped a switch's default branch, so a
returnthere dropped the list built so far; - numerals were folded in
int64and wrapped silently from 2^63; rc.h's assignment released the old block before counting the new one, which freesp->tailinp = p->tail;- costly values are borrowed by
const &where they were copied, a closure may be called more than once, and a match under a let's right-hand side is translated.
Three of those (the default branch, the numerals and the assignment) made C++ that compiled and computed something else. That is the argument for keeping the gates on every change to the backend: Crane is an alpha, as its site says, and its output is trusted only as far as it is compared. The patch is ours to maintain; none of it has been offered upstream yet.
M7. A fast simple register allocator. Designed for a proof, not ported. Gate: difftests and package checks by behaviour, and a measured run-time gain over stack homes on the benchmark gate.
Done. Regs.v
(nanofil-backend compile-allocated) numbers each function's
temporaries into nat identifiers, recognizes adjacent
single-use flight + check accesses and integer
let + cmp + branch sequences so
ephemeral values stay in %rax without taking a register
home, builds closed sub-intervals per block from CFG liveness, and
colors integer (w, l) temporaries and pointer
(p) address words in descending 4^depth
natural-loop weight order over %r10, %r11,
%rbx and %r12–%r15, preferring a
colored phi partner's register first so loop induction updates coalesce
in place. Capability lowers stay in their lowers[] frame
roots, and any temporary live across a call, helper call
(AStorePointer, ADispatch, alloc,
getters, va_arg) or poll edge takes only
callee-saved %rbx or %r12–%r15,
saved after the stack check and restored before every
leave. Phi edges compile via a parallel-move scheduler
using %rax and cycle-breaker %rcx with no
shadow slots, and scalar operations select directly into assigned
registers with three-address lea, imm32
operands and narrow loads and stores.
- Validation and proofs. Every allocated function is
checked before emission by a linear-time Gallina validator
(
valid_alloc).RegsProof.vproves interval disjointness and call-point exclusion, forward simulation ofvalid_allocacross instructions, simultaneous definitions, caller-saved call and poll clobbers, callee-saved save/restore preservation and capability root invariance, soundness of the parallel-move ready-peeling and%rcxcycle-breaking steps, and modular exactness of 32/64-bitleaandimm32selection (-2^31included), with executable rejection examples. - Behavioural gate.
compileandcompile-promotedkeep their byte-identity gates unchanged.tests/regs.sh(run withFUGC_SCRIBBLE=1 FUGC_VERIFY=1in thenanofil-backendcheck, withtests/regs/cycles-and-spills.ir; three planted bugs in call-crossing, cycle breaking and non-commutativesubeach fail it) checks refusal parity and runtime agreement across the corpus: all 44difftestfiles, all 9run-csuites, all 228 in-subsetprogtestC programs, GNU hello (74 allocated units), zlib (all 17 units), Lua 5.4.7 (all 33 non-setjmpunits, official suite and GC stress) and SQLite 3.51.2 (all 3,106 functions, matchingfilccoutput and database byte for byte, plain and under FUGC verification). - Speed. On
rx.ir(36 regex functions),compile-allocatedemits 28,199 lines (against 45,702 forcompile-promotedand 28,580 fornanofil -O compile) in 0.30 s, and a cold cache of 39 patterns throughrxctakes 2.9 s (75 ms a miss, against 4.1 s withcompile-promotedand 10.2 s withnanofil -O). Over 11 interleaved trials on the nine benchmark patterns, the allocated code runs at 1.05× of full Prolog-Oin the geometric mean (0.78× to 1.17×, matching Prolog's 1.03× ablation for this pass set) and 0.76× of stack-homedcompile-promotedon this Xeon orb (0.61× to 0.89×; 6.5× to 22.5× faster than PCRE2's interpreter). On SQLite's amalgamation, lowering, allocation, validation and emission of all 3,106 functions take 4.6 s (against 1.9 s for stack homes), writing 2.19M lines in place of 2.58M.
M8. The Fil-C library. The Crane output built by
filcc as libnanofil and called in the process by the pcre2
hook, which still runs as and ld and loads
with dlopen. Gate: the pizlonated library passes the
differential tests and the experiment's checks.
M9. Hybrid -O. Emission of the reuse
forms, so Prolog's middle end and check passes can feed the extracted
backend. This is where users of -O first feel the
speed.
Later. The rest of the middle end and the check
passes in Gallina, each replacing a *_refines run-time
check with a theorem. An encoder and a shared-object writer, and
installing code in the process, if measurement asks for them.
Outside this campaign. The C frontend. It is where an everyday C compile spends its time, so it is the thing to reconsider once the backend is done, as its own campaign.
M1 to M3 can run in parallel once M1's Asm.v types are
fixed, because the plan and the emission pieces own different files.
The proofs it enables
None of these is promised by the milestones above. They are what becomes statable.
- Pizlonation is valid.
valid (pizlonate m) = Some truefor every well-formed unchecked module. Withvalid_no_uncheckedit says no pizlonated module ever runs unchecked, with no validator run in the trust path. - Structural facts about lowering. Frame slots do not
overlap. Every pointer live across a poll or a call is a root. A check's
emitted sequence branches to a trap exactly when the model's
Checkfails. - Implementation correctness. A semantics for the asm
subset the backend emits, and a simulation from
step_relto it, one instruction kind at a time, starting with scalar selection againstOps.v. This is the row formal.md leaves out of proof today. - Proved passes instead of checked ones. A register
allocator or a check-reuse pass with a refinement theorem needs no
translation checker at compile time, which is where half of
-Ogoes. - A λ□ extractor. MetaRocq erases Rocq terms to λ□,
with a proof for the untyped erasure and a typed variant that keeps
types. A compiler from λ□ to nanofil IR, written in Gallina, would have
a correctness statement in terms of
Semantics.v, and its output would need no collector of its own, because Fil-C's is underneath. At the end of that road the extracted backend compiles itself to nanofil IR, and neither OCaml nor C++ is between the Gallina and the machine code.
Decisions wanted
Each has a recommendation; work proceeds on it unless told otherwise.
- What the first version's register mode is.
Recommended: stack homes now, a new simple allocator at M7, never a port
of
amd64_regs.pl. - Where the compiler starts. Recommended: checked IR
first (M1 to M3), since that is what
Syntax.vdescribes, then pizlonation (M4). It turned out to need no types of its own. - Byte identity as the standard, until the Gallina backend is the reference. Recommended: yes; it is the cheapest strong test there is, and the snapshot gate already works this way.
- The first JIT is the regex experiment's route:
as,ld -sharedanddlopen, with the extracted compiler inrxcfrom M5. Settled by mbrock. An encoder and installing code in the process are later options; the second is a runtime change and would wait for its own review. - What pcre2 needs after M5. The first version's code
has none of
-O, and M5 measured what each part is worth on the nine benchmark patterns. Promotion with copy propagation is 2.3 times, registers 1.7 to 2 times, branch selection 1.1 to 1.6; the check passes, simplification and inlining are nothing. Recommended: port promotion, copy propagation and branch selection (M5b, done) before Crane (M6) and the allocator (M7), and leave the check passes andsimplify.plfor later. Two earlier versions of this decision were wrong: the first credited the check passes, and the second took registers to be worth 1.1 to 1.7 times and put simplification in M5b. The other way to the same speed for pcre2 alone is a regex compiler that emits SSA with noallocslots; the port is preferred because every other producer of IR needs promotion too. - How the validator gets fast.
validis half of a compile and quadratic in a function's size, and it is the model's definition, with proofs that read it. Recommended: this campaign writes, in a file of its own, maps in place of the lists behindsingle_definitionsand thelookupof a definition and of a class, each proved equal to the list it replaces; the model's owner then switchesvalid_functo them, sovalidstays the same function and the proofs rewrite once. The backend takes no unproved replacement at extraction. Provingvalid (pizlonate m) = Some true, which removes the run altogether for the first version, stays the later goal. Agreed, with the equality at the top: the campaign's file provesvalid_fast md = valid md, the compiler runsvalid_fast, andvalid_funcand its proofs do not change.single_definitionsis done; thelookups are not. - What M8 does about the C++ build's speed. M6's C++
prints the right bytes at 5 to 7.5 times the OCaml build's time, against
a target of 2. That matters only in M8: on a single function the C++
took 4.4 times the OCaml build's time, and Fil-C's usual 1.5 to 4 times
comes on top. By that arithmetic a pattern's 16 to 43 ms in the OCaml
build would be 0.1 to 0.8 s in a pizlonated
libnanofil, which is slower than starting the OCaml build as a process, asrxcdoes now. That is an estimate; nothing has been built by filcc yet. Recommended: keeprxcon the OCaml build; begin M8 with a bounded attempt on Crane's output (the destructor's walk, the copies ofTermvariants, the parameters still passed by value) with 2 times OCaml onrx.iras the mark; and if that stalls above 3 times, measure CertiRocq's C on the same input before building the library on either. The other choice is to buildlibnanofilas it is, to learn first what filcc makes of Crane's C++, and take the speed after.