M2: naive x86-64 SysV backend

Archived: a record of finished work, kept as written. nanofil's archive lists the others.

src/amd64.pl turns the phase-one IR into GNU as text. It is deliberately naive, because it is the reference that later instruction selection and register allocation are checked against.

Both backends build a list of assembly terms rather than text, for example ins(mov, q, [reg(rsp), reg(rbp)]), label(L) and quad(S+8), with operands in AT&T order and concrete register names (reg(eax), reg(cl)). src/asm.pl defines the vocabulary and is the only place that knows AT&T syntax. Later passes and tests can inspect instructions as terms instead of matching text.

cd nanofil/src
../bin/nanofil compile ../tests/fixtures/memory.ir > memory.s

Scheme

• Frame. push %rbp; mov %rsp, %rbp; sub $N, %rsp. Every temporary has its own 8-byte slot below %rbp, and every phi destination has an extra shadow slot. N is rounded to 16.
• Instructions. Load operands into %rax, %rcx (and use %rdx for division), compute, store %rax to the destination slot. A w result is always zero-extended into %rax, so slots are uniformly 8 bytes and a w read is a 4-byte load of the slot's low half, which is exactly the IL's subtyping rule.
• Shifts use %cl; x86 masks the count to the operand width, as the IR specifies.
• Phis. On each edge, every source is copied to its shadow slot and then every shadow slot to its destination, so phis read their inputs simultaneously. jnz gets a .false edge label so each edge carries its own copies.
• alloc subtracts the size rounded up to 16 from %rsp, so calls stay 16-byte aligned without tracking.
• Calls pass up to six integer arguments in the SysV registers. More than six, in a definition or a call, is rejected with a diagnostic.
• unreachable is ud2.

The output is about four times larger than qbe's. That is the price of an obviously right reference.

Testing

difftest.sh now has three variants: original (qbe on the file), printed (qbe on nanofil's printed copy) and nanofil (this backend). Each must match the interpreter. VARIANTS=nanofil selects one.

• Fixtures, including calls.ir, which uses all six argument registers with mixed classes, reads an l result at w, and passes constants and a global address. Its expected values were checked by hand.
• Random corpus. Seeds 1 to 8 of gen.py, 8,098 compared runs: no mismatches.
• qbe-bugs. nanofil gets all three repros right, and qbe still gets them wrong. The Nix check runs them against nanofil alone.
• Mutation check (manual). Four planted backend bugs: signed less-than as unsigned, loadsh into l zero-extended, sequential phi copies, and rem returning the quotient. The random corpus caught all four; the fixtures caught three (they don't cover loadsh into l).

Driver memory

difftest used to evaluate every run in one forward maplist. Trealla kept every run's constraint store reachable, which peaked at about 10 GB on a 200-function file and got jobs killed. Driving the runs through findall over member reclaims each run's heap on backtracking. Peak memory is now 330 MB, and the run is a third faster.