Joseph Ferano
e0e5c1e645
The transfer exit returned whatever the return temporary held where emit.ml returns zero. Meaningless to a caller -- its guard sees the channel set and never looks -- but main is a caller with no guard, and what it finds in rax is the process exit status. The survey compares stderr as well now, which is where every message the new machinery produces goes: the bounds and slice errors, the three restart refusals, the transfer failure. Each carries a location this backend emits by hand as a .rodata label and a length in a register, and an exit status of 134 with the wrong text beside it is exactly the failure that reads as a match. It also walks spike/x86's own probes. p6-transfer.flan is the two re-propagation branches the corpus does not reach. Every transfer in restarts.flan stops at a restart-case inside the handler-bind's extent, so the handler frames never come off on the transfer path; and in nested and shadowed the inner frame offers the name, so a restart-case the transfer is not aimed at never has to put the target back. allocators.flan already covers the third. 89 MATCH 0 DIFFER 0 refused, over test/programs and spike/x86, comparing stdout, stderr and the exit status. DISCUSS.md item 17 is the report.
121 lines
4.5 KiB
Bash
Executable File
121 lines
4.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Does the hand-written backend agree with LLVM?
|
|
#
|
|
# The only honest test of a hand-encoded backend is what the program prints and
|
|
# what it exits with -- DISCUSS.md item 15 and item 16 both say so, and both
|
|
# say it after a disassembly that read perfectly beside a wrong answer. So this
|
|
# builds every program in test/programs twice, runs both, and diffs stdout,
|
|
# stderr and the exit status. objdump is for after a program already has the
|
|
# wrong answer.
|
|
#
|
|
# stderr is not an afterthought: every message the condition machinery produces
|
|
# goes there -- the bounds and slice errors, the three restart refusals, the
|
|
# transfer failure -- and each carries a location string this backend emits by
|
|
# hand as a .rodata label and a length in a register. An exit status of 134
|
|
# with the wrong text beside it is exactly the failure that looks like a
|
|
# match.
|
|
#
|
|
# Both sides get the same bounds-check setting (the default: on). A sweep that
|
|
# compared a checked build against an unchecked one would say nothing about
|
|
# bounds.flan, which is the one program the two backends disagreed about.
|
|
#
|
|
# Five outcomes, and the third is the progress meter:
|
|
#
|
|
# MATCH built both ways, same stdout, same stderr, same exit status
|
|
# DIFFER built both ways, and disagreed
|
|
# REFUSED X86.Unsupported -- a node this backend does not lower (exit 3)
|
|
# NOX86 failed to build through --x86 for some other reason
|
|
# SKIP no main, does not compile at all, or does not terminate
|
|
#
|
|
# Over test/programs, and over spike/x86's own probes, which are here for the
|
|
# paths the corpus does not walk.
|
|
#
|
|
# Usage: spike/x86/survey.sh [name-substring ...]
|
|
set -u
|
|
here=$(cd "$(dirname "$0")" && pwd)
|
|
root=$(cd "$here/../.." && pwd)
|
|
cd "$root" || exit 1
|
|
|
|
dune build --root . bin/main.exe 2>&1 | head -30
|
|
flan=$root/_build/default/bin/main.exe
|
|
test -x "$flan" || { echo "build failed"; exit 1; }
|
|
|
|
out=$(mktemp -d); trap 'rm -rf "$out"' EXIT
|
|
|
|
# The two that run until something stops them. Not a failure and not a match;
|
|
# they are excluded by name because a timeout cannot tell them apart from a
|
|
# backend that hung.
|
|
forever="dev-loop dev-watch"
|
|
|
|
TIMEOUT=${TIMEOUT:-20}
|
|
|
|
declare -a match=() differ=() refused=() nox86=() skip=()
|
|
|
|
for src in "$root"/test/programs/*.flan "$root"/spike/x86/*.flan; do
|
|
name=$(basename "$src" .flan)
|
|
if [ $# -gt 0 ]; then
|
|
want=0
|
|
for pat in "$@"; do case "$name" in *"$pat"*) want=1;; esac; done
|
|
[ $want = 1 ] || continue
|
|
fi
|
|
case " $forever " in *" $name "*) skip+=("$name:runs-forever"); continue;; esac
|
|
|
|
# LLVM first. A program that does not compile at all, or has no main, is not
|
|
# this backend's business -- the frontend refused it either way.
|
|
if ! "$flan" build "$src" -o "$out/$name.llvm" >"$out/$name.llvm.err" 2>&1; then
|
|
if grep -q "in function \`_start\|undefined reference to \`main\|crt1.o" "$out/$name.llvm.err"; then
|
|
skip+=("$name:no-main")
|
|
else
|
|
skip+=("$name:does-not-compile")
|
|
fi
|
|
continue
|
|
fi
|
|
|
|
"$flan" build "$src" --x86 -o "$out/$name.x86" >"$out/$name.x86.err" 2>&1
|
|
rc=$?
|
|
if [ $rc = 3 ]; then
|
|
why=$(head -1 "$out/$name.x86.err" | sed 's/^x86: //')
|
|
refused+=("$name:$why")
|
|
continue
|
|
fi
|
|
if [ $rc != 0 ]; then
|
|
nox86+=("$name:$(head -1 "$out/$name.x86.err")")
|
|
continue
|
|
fi
|
|
|
|
( cd "$out" && timeout "$TIMEOUT" "$out/$name.llvm" \
|
|
>"$out/$name.llvm.out" 2>"$out/$name.llvm.diag" )
|
|
a=$?
|
|
( cd "$out" && timeout "$TIMEOUT" "$out/$name.x86" \
|
|
>"$out/$name.x86.out" 2>"$out/$name.x86.diag" )
|
|
b=$?
|
|
if [ "$a" = "$b" ] && cmp -s "$out/$name.llvm.out" "$out/$name.x86.out" \
|
|
&& cmp -s "$out/$name.llvm.diag" "$out/$name.x86.diag"; then
|
|
match+=("$name")
|
|
else
|
|
differ+=("$name:llvm=$a/x86=$b")
|
|
if [ "${SURVEY_SHOW:-}" = 1 ]; then
|
|
echo "--- $name: llvm exit $a, x86 exit $b"
|
|
diff "$out/$name.llvm.out" "$out/$name.x86.out" | head -20
|
|
diff "$out/$name.llvm.diag" "$out/$name.x86.diag" | head -20
|
|
fi
|
|
fi
|
|
done
|
|
|
|
echo
|
|
echo "MATCH ${#match[@]}"
|
|
echo "DIFFER ${#differ[@]}"
|
|
[ "${#differ[@]}" = 0 ] || printf ' %s\n' "${differ[@]}"
|
|
echo "REFUSED ${#refused[@]}"
|
|
if [ "${#refused[@]}" != 0 ] && [ "${SURVEY_QUIET:-}" != 1 ]; then
|
|
printf '%s\n' "${refused[@]}" | sed 's/^[^:]*://' | sort | uniq -c | sort -rn \
|
|
| sed 's/^/ /'
|
|
fi
|
|
echo "NOX86 ${#nox86[@]}"
|
|
[ "${#nox86[@]}" = 0 ] || printf ' %s\n' "${nox86[@]}"
|
|
echo "SKIP ${#skip[@]}"
|
|
if [ "${#skip[@]}" != 0 ] && [ "${SURVEY_QUIET:-}" != 1 ]; then
|
|
printf '%s\n' "${skip[@]}" | sed 's/^[^:]*://' | sort | uniq -c \
|
|
| sed 's/^/ /'
|
|
fi
|