Joseph Ferano
63d9b87b7a
The transfer channel was the only thing between 41 programs and the corpus. It is there now: a guard after every Flan call, a landing pad per restart-case, handler-bind and with-allocator, a transfer exit per function that runs its fdefers, and check_at and check_slice, which could not exist until the guard did. Measured by what the programs print and what they exit with, never by reading bytes. spike/x86/survey.sh builds every program in test/programs both ways and diffs stdout and the exit status; it did not exist, so it is here too, and it is the progress meter. before 41 MATCH 1 DIFFER 41 refused by name after 83 MATCH 0 DIFFER 0 refused by name The one DIFFER was bounds.flan, and it was the honest answer to "--x86 is silently a --no-bounds-checks build". It is not one any more: check_at and check_slice signal through the channel exactly as emit.ml's do, so a bounds violation signals, a restart-case catches it, and an unhandled one exits 134 on both backends. The transitional refusal that would have said so retired before it was written. check_no_transfer is not removed, it is narrowed to the one place the argument still holds: a global's initialiser runs from flan..init-globals, before main and before anything can handle anything, so a transfer out of it has nowhere to go. Four bugs, and three of them are the shape item 16 predicted -- code that reads correctly and answers wrong, found by output and not by objdump: - The body fell through into the transfer exit, so every fdefer ran twice on a normal return. emit.ml cannot have this bug: its ret terminates the block. - A Vec crossed to the runtime as the address of a *copy*, so pushes grew the copy and an in-bounds (at v 1) signalled against a length of zero. - ucomis sets CF, ZF and PF together for a NaN, so sete answered true for (= x x) and the prelude's NaN test never fired: (/ 0.0 0.0) formatted as -9223372036854775808. Flan's comparisons are LLVM's ordered ones, so < and <= swap and =, != take a setnp beside them. - A union read field 0 through the struct table and was refused by name rather than laid out as a tag and a payload. And one that could not have been found later: emit_globals_init stored a null *into* the channel slot rather than a cell address into it, which is a null pointer for every callee to write through. Harmless while nothing could transfer; a fault the first time a guard loaded through it.
106 lines
3.8 KiB
Bash
Executable File
106 lines
3.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Does the hand-written backend agree with LLVM?
|
|
#
|
|
# The only honest test of a hand-encoded backend is what the program prints and
|
|
# what it exits with -- DISCUSS.md item 15 and item 16 both say so, and both
|
|
# say it after a disassembly that read perfectly beside a wrong answer. So this
|
|
# builds every program in test/programs twice, runs both, and diffs stdout and
|
|
# the exit status. objdump is for after a program already has the wrong answer.
|
|
#
|
|
# Both sides get the same bounds-check setting (the default: on). A sweep that
|
|
# compared a checked build against an unchecked one would say nothing about
|
|
# bounds.flan, which is the one program the two backends disagreed about.
|
|
#
|
|
# Five outcomes, and the third is the progress meter:
|
|
#
|
|
# MATCH built both ways, same stdout, same exit status
|
|
# DIFFER built both ways, and disagreed
|
|
# REFUSED X86.Unsupported -- a node this backend does not lower (exit 3)
|
|
# NOX86 failed to build through --x86 for some other reason
|
|
# SKIP no main, does not compile at all, or does not terminate
|
|
#
|
|
# Usage: spike/x86/survey.sh [name-substring ...]
|
|
set -u
|
|
here=$(cd "$(dirname "$0")" && pwd)
|
|
root=$(cd "$here/../.." && pwd)
|
|
cd "$root" || exit 1
|
|
|
|
dune build --root . bin/main.exe 2>&1 | head -30
|
|
flan=$root/_build/default/bin/main.exe
|
|
test -x "$flan" || { echo "build failed"; exit 1; }
|
|
|
|
out=$(mktemp -d); trap 'rm -rf "$out"' EXIT
|
|
|
|
# The two that run until something stops them. Not a failure and not a match;
|
|
# they are excluded by name because a timeout cannot tell them apart from a
|
|
# backend that hung.
|
|
forever="dev-loop dev-watch"
|
|
|
|
TIMEOUT=${TIMEOUT:-20}
|
|
|
|
declare -a match=() differ=() refused=() nox86=() skip=()
|
|
|
|
for src in "$root"/test/programs/*.flan; do
|
|
name=$(basename "$src" .flan)
|
|
if [ $# -gt 0 ]; then
|
|
want=0
|
|
for pat in "$@"; do case "$name" in *"$pat"*) want=1;; esac; done
|
|
[ $want = 1 ] || continue
|
|
fi
|
|
case " $forever " in *" $name "*) skip+=("$name:runs-forever"); continue;; esac
|
|
|
|
# LLVM first. A program that does not compile at all, or has no main, is not
|
|
# this backend's business -- the frontend refused it either way.
|
|
if ! "$flan" build "$src" -o "$out/$name.llvm" >"$out/$name.llvm.err" 2>&1; then
|
|
if grep -q "in function \`_start\|undefined reference to \`main\|crt1.o" "$out/$name.llvm.err"; then
|
|
skip+=("$name:no-main")
|
|
else
|
|
skip+=("$name:does-not-compile")
|
|
fi
|
|
continue
|
|
fi
|
|
|
|
"$flan" build "$src" --x86 -o "$out/$name.x86" >"$out/$name.x86.err" 2>&1
|
|
rc=$?
|
|
if [ $rc = 3 ]; then
|
|
why=$(head -1 "$out/$name.x86.err" | sed 's/^x86: //')
|
|
refused+=("$name:$why")
|
|
continue
|
|
fi
|
|
if [ $rc != 0 ]; then
|
|
nox86+=("$name:$(head -1 "$out/$name.x86.err")")
|
|
continue
|
|
fi
|
|
|
|
( cd "$out" && timeout "$TIMEOUT" "$out/$name.llvm" >"$out/$name.llvm.out" 2>/dev/null )
|
|
a=$?
|
|
( cd "$out" && timeout "$TIMEOUT" "$out/$name.x86" >"$out/$name.x86.out" 2>/dev/null )
|
|
b=$?
|
|
if [ "$a" = "$b" ] && cmp -s "$out/$name.llvm.out" "$out/$name.x86.out"; then
|
|
match+=("$name")
|
|
else
|
|
differ+=("$name:llvm=$a/x86=$b")
|
|
if [ "${SURVEY_SHOW:-}" = 1 ]; then
|
|
echo "--- $name: llvm exit $a, x86 exit $b"
|
|
diff "$out/$name.llvm.out" "$out/$name.x86.out" | head -20
|
|
fi
|
|
fi
|
|
done
|
|
|
|
echo
|
|
echo "MATCH ${#match[@]}"
|
|
echo "DIFFER ${#differ[@]}"
|
|
[ "${#differ[@]}" = 0 ] || printf ' %s\n' "${differ[@]}"
|
|
echo "REFUSED ${#refused[@]}"
|
|
if [ "${#refused[@]}" != 0 ] && [ "${SURVEY_QUIET:-}" != 1 ]; then
|
|
printf '%s\n' "${refused[@]}" | sed 's/^[^:]*://' | sort | uniq -c | sort -rn \
|
|
| sed 's/^/ /'
|
|
fi
|
|
echo "NOX86 ${#nox86[@]}"
|
|
[ "${#nox86[@]}" = 0 ] || printf ' %s\n' "${nox86[@]}"
|
|
echo "SKIP ${#skip[@]}"
|
|
if [ "${#skip[@]}" != 0 ] && [ "${SURVEY_QUIET:-}" != 1 ]; then
|
|
printf '%s\n' "${skip[@]}" | sed 's/^[^:]*://' | sort | uniq -c \
|
|
| sed 's/^/ /'
|
|
fi
|