Ground truth for jump table recovery, used to measure the accuracy of B2R2's middle-end on every push.
A jump table is how compilers implement a dense switch: an indirect branch
reads a target out of a table in the binary. Recovering those tables is what
decides whether a reconstructed CFG is complete, and getting it wrong is silent.
This repository pins down the right answer for real binaries per
architecture (x86, x64, 32-bit ARM, AArch64, MIPS, RISC-V, PowerPC and m68k)
so that a regression shows up as a failing check rather than as a quietly
worse CFG.
Ground truth lives in git; binaries do not. The JSON is small, reviewable, and
belongs in history. The binaries are large and change a few times a year, so
they ship as one tarball per configuration attached to a release. A
configuration is an architecture and a compiler, <arch>-<compiler> (e.g.,
x64-gcc), and both ground-truth/ and binaries/ are split by it.
SHA256SUMS one line per binary; the integrity anchor
exclusions.json cases left out of scoring, and why
ground-truth/<config>/<case>.json one file per binary
schema/jumptable.schema.json JSON Schema for the above
scripts/ generate shasums, pack tarballs, find redundant cases
binaries/<config>/ is where a local build lands; it is gitignored.
Every case is a stripped binary, built both as a PIE and as a non-PIE, named
<project>-<version>-<compiler>-<pie|nopie>-<opt>-<program>:
| projects | GNU coreutils 9.9 and GNU binutils 2.45 |
| compilers | gcc 16.2.0 and clang 23.0.0 |
| architectures | x86, x64, ARMv7 as armhf (hard float) and armel (soft float), aarch64, big-endian MIPS32r2 (o32) and MIPS64r2 (n64) as mips32 and mips64, RV32GC (ilp32d) and RV64GC (lp64d) as riscv32 and riscv64, big-endian 32-bit PowerPC (SysV, hard float) and 64-bit PowerPC (ELFv1) as ppc32 and ppc64, and the Motorola 68020 as m68k |
| binaries | 546 (x86), 548 (x64), 397 each (armhf, armel), 548 (aarch64), 409 (mips32), 542 (mips64), 546 each (riscv32, riscv64), 548 each (ppc32, ppc64) and 276 (m68k), of which 24, 24, 6, 24, 18, 18, 24, 24 and 12 are from binutils |
| tables | 10,458 (x86), 9,582 (x64), 5,279 each (armhf, armel), 5,581 (aarch64), 9,287 (mips32), 9,024 (mips64), 8,964 (riscv32), 8,273 (riscv64), 4,948 (ppc32), 4,528 (ppc64) and 4,736 (m68k) |
- We exclude 37 coreutils programs (e.g., ls, vdir, test, true, yes) whose jump tables are already covered by another included program, to avoid redundant evaluation. Two tables count as the same when their kind, entry size, entry count, and targets relative to the indirect branch all match;
scripts/find-redundant.pylists any case that adds no distinct table. - We keep three binutils programs:
as-new,objdumpandreadelf. The other seven link the same libbfd, so they add only about 10% of the distinct binutils tables but would take over 60% of the binutils scoring time.
x86 and x64 take both compilers at -O2 and -Os:
- We leave out
-O0and-O1because they mostly repeat-O2: gcc builds almost no jump table at-O0, and 85-89% of the tables at-O1have an-O2counterpart of the same kind, entry size, entry count, and target count in the same program. - We leave out
-O3and-Ozto bound the scoring time. Every jump table bug B2R2 has hit at those levels so far also showed up at-O2or-Os. - clang builds no
stdbuf, and neither does gcc for the non-PIEx86build.
armhf and armel take only the cells, a compiler at an optimization level,
that B2R2 already gets entirely right on both. The check passes only when
nothing is missed or invented, and one that fails from the start catches no
regression. A cell joins once B2R2 gets all of it right:
- gcc at
-O2and clang at-Os, coreutils only. gcc at-Osand clang at-O2still miss a table or two in five and six programs, and of the binutils programs only the clang PIEas-newpasses at-O2and-Os. - clang at
-O0, binutils included. Only a third of its coreutils tables have an-Oscounterpart of the same kind, entry size, entry count, and target count in the same program, and clang-O2, which would repeat two thirds of them, is not in yet. - clang builds no
stdbuf, and neither does gcc for the non-PIE build at-O2.
aarch64 takes both compilers at -O2 and -Os, as x86 and x64 do,
binutils included. As with ARM, a cell joins only once B2R2 gets all of it
right; these four already pass:
- We leave out
-O1and-O3because they mostly repeat-O2: 73-75% (gcc) and 88-95% (clang) of their tables have an-O2counterpart of the same kind, entry size, entry count, and target count in the same program. gcc builds no jump table at-O0, and its-Ozbinaries are byte for byte its-Osones. - clang at
-O0and-Ozis not in yet. None of its-O0tables and only 41% of its-Ozones have an-Oscounterpart, but B2R2 still invents targets in the non-PIEas-newandobjdumpat-O0, and misses or invents some inas-new,objdumpand the non-PIEexprat-Oz. - clang builds no
stdbuf.
mips32 and mips64 take both compilers at -O2 and -Os, as x86 and
x64 do, binutils included. As with ARM, a cell joins only once B2R2 gets all
of it right, which leaves two out for now:
- gcc at
-Osis not in yet formips32, as B2R2 still misses a table in the PIEsort, and gcc at-O2takes coreutils only formips64, as B2R2 still misses one inreadelf. - We leave out
-O1and-O3because they mostly repeat-O2: 83-90% (gcc) and 91-97% (clang) of their tables have an-O2counterpart of the same kind, entry size, entry count, and target count in the same program. gcc builds almost no jump table at-O0(none formips32, 14 formips64), and its-Ozbinaries are byte for byte its-Osones. - We leave out clang at
-O0and-Ozto bound the scoring time: 71-74% of the-O0tables and 98% of the-Ozones have an-O2or an-Oscounterpart, respectively. - clang builds no
stdbuf, and neither does gcc for the non-PIEmips32build.
riscv32 and riscv64 take both compilers at -O2 and -Os, as x86 and
x64 do, binutils included. B2R2 already gets every cell of both right, so, as
with Intel, the levels left out only bound the scoring time:
- We leave out
-O1and-O3because they mostly repeat-O2: 69-92% (gcc) and 95-97% (clang) of their tables have an-O2counterpart of the same kind, entry size, entry count, and target count in the same program. gcc builds almost no jump table at-O0(14 per architecture), and its-Ozbinaries are byte for byte its-Osones. - We leave out clang at
-O0and-Oz: 67-69% of the-O0tables and 99% of the-Ozones have an-O2or an-Oscounterpart, respectively. - clang builds no
stdbuf, and neither does gcc for the non-PIE build.
ppc32 and ppc64 take both compilers at -O2 and -Os, as x86 and x64
do, binutils included. B2R2 already gets every cell of both right, so, as with
Intel, the levels left out only bound the scoring time:
- We leave out
-O1and-O3because they mostly repeat-O2: 64-93% (gcc) and 96-100% (clang) of their tables have an-O2counterpart of the same kind, entry size, entry count, and target count in the same program. gcc builds no jump table at-O0, and its-Ozbinaries are byte for byte its-Osones. - We leave out clang at
-O0and-Oz: 55-59% of the-O0tables and all of the-Ozones have an-O2or an-Oscounterpart, respectively. - clang builds a jump table on PowerPC only for a switch of 64 cases or more, so its configurations hold 132 (
ppc32) and 140 (ppc64) tables, all but 8 of them in binutils. The 252 cases of each that hold none still check that no table is invented. - clang builds no
stdbuf.
m68k takes gcc only, at -O2 and -Os, binutils included. B2R2 already gets
every cell right, so, as with Intel, the levels left out only bound the
scoring time:
- We leave out
-O1and-O3because they mostly repeat-O2: 78-93% of their tables have an-O2counterpart of the same kind, entry size, entry count, and target count in the same program. gcc builds no jump table at-O0, and its-Ozbinaries are byte for byte its-Osones. - gcc builds
stdbuffor both the PIE and the non-PIE build.