Prolog hosts

Archived: a record of finished work, kept as written. nanofil's archive lists the others.

nanofil runs on SWI-Prolog (the default, through src/swi/) and supports Trealla for the IR backend. Full Trealla frontend/interpreter testing remains paused. This file records the compatibility work and measurements.

Trealla backend restored (November 2026)

The pinned native Trealla (nix/trealla.nix) runs optimize/3, pizlonate_module/2 and filc_module_emit/3. This section was written and measured on revision 12f4cbd7; the pin has since moved, see Trealla 98a821db. Run the CLI from nanofil/src, with stdin closed:

NANOFIL_HOST=trealla TPL=/path/to/tpl ../bin/nanofil -O compile input.ir </dev/null

tests/backend-hosts.sh compares optimized IR, checked IR and assembly in stack, temps and pointers modes on both hosts, plus plain and -O CLI compilation. Its eight ABI fixtures cover floats, phi shadows, local storage, variadics, setjmp, globals and copies. A forty-parameter function exercises liveness masks beyond 64 bits: reverting the bitset fix makes the comparison fail even when compilation succeeds. The small parity step runs in the default Nix check; the full Trealla suite is still opt-in and currently stops parsing a multiline test literal in tests/unit.pl ("unterminated quoted atom").

What worked, and what broke

The parent of the October 24 pause, 8a5f5fa8, still passes all 57 of its unit tests on the pinned Trealla when rerun. The pause, d31875b3, was a decision to stop spending time on Trealla's memory and runtime cost, not a known backend correctness failure. Later SWI-only development introduced several independent incompatibilities:

• Register allocation (bbc0f97f, October 27) classified assembly labels with atom_number/2. On Trealla, a symbolic label throws through number_codes/2 instead of failing. Test explicitly for decimal digits.
• Its wide liveness masks also exposed a pinned-host arithmetic defect: (1<<64) /\ \1 evaluates to zero. Use the equivalent nonnegative set difference xor(A, A /\ B) instead of A /\ \B in both allocator and checker. No validation is disabled or weakened.
• Phi-shadow reuse (04c7e7cb, October 28) used missing select/4; replace it with indexed removal/insertion. Promotion (2ce8b0f3) and later slot/register selection used missing partition/4; share a portable, stable implementation.
• K&R parsing (a2ee3145) used DCG negation, and C23 booleans (36039eb5) used an if-then without a final else inside a DCG. Equivalent grammar rewrites let the CLI load. Trealla supplies read_line_to_codes/2 itself, not through SWI's library(readutil).
• Wide inlining (c0f190b3, November 5) used SWI's generative arg(_, Term, Arg). Enumerate argument indices explicitly.
• Trealla considers [] an atom; serialize it before testing atom/1 so printed IR agrees. On the nine-regex module, the branch-use scan via recursive =../2 also produced malformed temporary-name lists and a duplicate-key exception. Indexed arg/3 traversal avoids that failure; its underlying host defect has not been reduced further.

Remaining cost

On the nine patterns (tag, email, word, alt, entity, shortcode, key, date, lit) generated by experiments/regex-nanofil/rx.pl, CLI -O compile produced byte-identical 8,233-line assembly on SWI and Trealla. The frontend-free parity driver also matched optimized IR, checked IR and all three emission modes (36,882 lines including the wide-mask fixture). A separate driver isolates the stages below. These are CPU seconds on an orb, SWI 9.2.9 versus pinned native Trealla, with optimize(o1(wide,full), ...), validation and filc_module_emit(..., emit(pointers,selected,inline), print_asm):

Emission, especially allocating registers, dominates absolute time. With stack emission the emit stage takes 0.112/0.637 CPU seconds and the whole process 0.83/4.24 wall seconds (SWI/Trealla); outputs still match between hosts. This is an existing cheaper compilation mode, not a substitute for testing the allocating modes. There is no measured speedup from this compatibility repair, and no host/runtime tuning is claimed. An embedded Trealla build and full frontend/interpreter coverage are not established by these native-host measurements.

Trealla 98a821db: speed, memory and where they go (November 2026)

The pin moved from 12f4cbd7 (September 5) to upstream HEAD of November 10, 98a821db, 630 commits later, with one new patch for a wrong-answer defect in it (patches/trealla-heap-page-mark.patch, described in docs/trealla.md). tests/backend-hosts.sh passes and takes 9.2 s instead of 12.7 s. This revision fixes both host defects worked around above: atom_number/2 fails on a symbolic label and (1<<64) /\ \1 is 2^64. The workarounds are still in the source.

-O compile of the 39 single-function modules that experiments/regex-nanofil/hook.c compiles, in one process without the C front end. The outputs are SWI's, byte for byte, in both register modes:

So the new revision takes about 0.7 of the time and a third of the memory, and is still 2 to 4 times slower than SWI with 15 times the memory. The heap patch accounts for 210 MB of that: unpatched 98a821db peaks at 844 MB.

The rest is from the largest of the 39 (3242 lines of assembly), which takes 2.8 s of CPU and 600 MB on Trealla against 0.64 s and 38 MB on SWI.

Memory

Trealla has no garbage collector. A frame, its variable slots and the heap cells built under it are released by backtracking, by a tail call that reuses the frame, or when a deterministic call exits and nothing older refers into it. Binding an older variable to a newer frame's variable or structure pins that frame, and that is what deterministic code that returns terms through output arguments does all the time. nanofil's passes are such code, so nearly everything they touch stays allocated until the pass's caller fails or the process ends.

High-water marks for the largest pattern, and what is live after each stage:

The high-water marks come to 520 MB of the 600 MB. Emission is 2.3 of the 2.8 s and four fifths of the memory, and within it the register allocator and its checker (placed_lines).

Running a stage inside findall/3, so that leaving it backtracks, gives the same assembly in the same time: the peak is 558 MB with the stages isolated on unpatched 98a821db, and 362 MB with the heap patch and the three steps of one function's emission isolated as well. What is left is inside one call of the register allocator. This was a scratch driver; nanofil's source is unchanged.

Time

• Trealla runs naive reverse at 3.8 to 6.5 million inferences a second and SWI at 31 to 37 million, so the engine is 6 to 8 times slower on plain list code. On nanofil the gap is smaller, 2 to 4 times.
• A sampling profile of the largest pattern puts 51% of the time under match_head (clause selection and head unification), 21% under unify_head, about 15% in deref and get_slot, and 9% in copying for findall/3.
• strcmp is 7 to 9%: must_be/2 compares type names as strings on each call (2.9%), call/N looks its module up by name (0.7%), and the rest is atom and operator lookups while loading.
• By call count the workload is association lists: about 1.5 million calls each of get_assoc/3 and its helper and 1.4 million of the six-argument one, from the register allocator and its checker.

Hardware counters are unavailable in an orb, so there are no cache or branch figures.

What would help

• In nanofil: isolate each function's emission, and each pass, so that leaving it backtracks. That is the 600 to 362 MB above at no cost in time, and it is what stack mode shows from the other side: without the register allocator the same pattern needs 229 MB.
• In nanofil: fewer association-list lookups in the register allocator and its checker should help both hosts. Not tried.
• In Trealla: frame pinning is the design, not a defect, so a collector or a finer-grained pin is upstream's to decide. The strcmp calls in must_be/2 and call/N are small, local changes.

Scryer Prolog: not worth supporting yet (October 2026)

Scryer 0.10.0 (nixpkgs scryer-prolog) ran a scratch copy of src/ on Lua 5.4.7's files. The output was identical to SWI's: cc-ir and -O cc of lvm.c (76,757 lines of assembly) and lapi.c, and the preprocessor's tokens of lvm.c matched filcc -E's. It was slower and larger everywhere:

Scryer's packed strings did not make the character-list preprocessor cheaper: text it builds (spliced lines, spellings) is ordinary list cells again. Scryer also compiles all of src/ on every run.

Running there took, besides one real fix (an operator atom as an operand must be bracketed, Op == (=), not Op == '='; now in c_lower.pl):

• no :- if/else/endif: choose the non-SWI branches by script;
• argv from os:argv/1; forall/2 from library(iso_ext);
• missing: msort/2, sort/4, atomic_list_concat/2,3, atom_number/2, is_list/1, include/3, exclude/3, foldl/6, last/2, max_list/2, min_list/2, numlist/3, subtract/3;
• char lists, not atoms, for format/2,3 and format_//2 format strings (and no format(atom(A), ...)), getenv/2, process_create/3, and file names (path_canonical/2, file_exists/1 stand in for absolute_file_name/2, exists_file/1);
• write_canonical/1 prints lists as '.'(H, T), so tokens output differs in form.

So "Trealla's dialect" also leans on Trealla's SWI-like extras. A src/scryer/ layer would be about 60 lines plus a format wrapper, but at 4x SWI's time it buys only a strictness check. Revisit when Scryer is faster or caches compiled code.

SWI-Prolog: what the preprocessor's time was (October 2026)

Preprocessing Lua 5.4.7's lvm.c (65,628 tokens) took 3.2 s, against 0.06 s for filcc -E. Measured on SWI 9.2.9:

• Reading headers was 63% of it, and lvm.c reads 124 files 272 times (bits/types.h 19 times): guarded headers are included again and again. read_source/2 now keeps each file's tokens for the run, 3.2 s to 2.3 s.
• punctuator/1 and digraph/1 were member/2 over a list, tried with every punctuation character; as facts they are indexed, 2.3 s to 2.1 s. All of Lua, preprocessing and lexing, went from 56.8 s to 35.6 s.
• What remains is flat: hide-set memberchk, macro lookups, and re-lexing each token (0.36 s).
• swipl -O gave nothing on tokens and 4% on -O cc. Turning GC off gave nothing and 4x the memory; the profiler's 12% in GC is not cost.
• Startup is 0.26 s, SWI compiling src/ on every run, about 9 s of Lua's 35. A saved state (swipl -o nanofil.state -c swi/compat.pl main.pl, run with swipl -x nanofil.state -- ARGS) starts in 0.05 s with identical output, interpreter commands included. bin/nanofil --save-state FILE builds one and NANOFIL_STATE=FILE uses it; the Nix check and nix/nanofil-cc.nix (hello, zlib, Lua) do. A state does not rerun directives, so swi/compat.pl sets the UTF-8 streams again on restore (without that, stderr printed \u00E9 for é under LANG=C).
• Not tried: skipping a re-included header whose guard macro is defined (clang's multiple-include optimization), and SWI strings instead of char lists, which would leave the shared dialect.