[pgranger23]

Fourth step of following SPINE releases. The test suite no longer fails on SPINE changes that need no work in the reader. A change to what the CAFMakers rely on fails a ctest entry of its own, so that a later step can tell the two apart.

What changes

• spine_reader_contracts (new ctest entry, tests/test_contracts.cpp): what the CAFMakers rely on.

  • The table. One row per model field a CAFMaker reads: class, field, unit, default as JSON, readers.
    • There are 228 rows. They come from a scan of the three ports (the spine-reader pull requests of DUNE/ND_CAFMaker, justinjmueller/sbn_ml_cafmaker and DUNE/dune_ml_cafmaker), with SPINE 1.4.0's units and defaults.
    • From now on, a row changes only by hand.
  • The check. For each class, it walks golden_members.inc and compares each listed member's unit, and the member of a default object, with its row.
    • A row whose field or class the model no longer has fails ("RecoParticle.csda_ke: listed for ND SBN, not in the model").
  • Enums. Every member of ParticleShape, ParticlePID and NuCurrentType is checked by name. ND compares shape and current_type with named members; SBN stores the numbers of shape, pid, chi2_pid and current_type. Members SPINE adds are not a change.
  • spine_reader_tests now excludes contract*. ci/sl7-ups.sh still runs the whole binary.

• The golden test checks the model against the current fixtures' columns. column_problems() runs on every current_* case except current_v2_fixed_only, whose columns are a subset by design:

  • a full file has a column for every field;
  • a lite file has one for every field except the lite_skip ones, and none for those (both ways);
  • no column is outside the model;
  • element kinds (bool, integer, float, string) agree.
  • Width differences are printed but are not failures. Today there are four: RecoParticle.momentum, TruthParticle.ke, .mass and .reco_momentum, as documented in tools/fixtures/README.md. FlashHypothesis is printed as "not checked against a file".
  • The function has its own test, on a doctored copy of current_v1's reco_particles. The column is picked at run time, not by name.

• Change detectors removed from test_model.cpp:

  • the field counts, the LAST_FIELD table and the class list;
  • schema_version == "1.4.0";
  • the defaults, units, flags and references;
  • the enum member counts and values.

  What remains holds for any tag:

  • the structure check, now for every class;
  • "field i has the name of enumerator i", for every field through golden_members.inc;
  • the member and descriptor types, and the enumeration pointers;
  • the assign test, comparing with default objects;
  • the NULL_ rename.

  Also dropped:

  • The literal scheme tables. The new check only requires each scheme to select a generated table or none, because no CAFMaker reads interaction_scheme or calls the table functions (SBN stores interaction_mode and interaction_type as numbers).
  • "Every field has a doc". SPINE can add a field without a docstring; the local tag below did that and failed it.

• Other tests no longer depend on a literal model:

  • test_reader.cpp:
    • Meta checks mismatched, extra and its four fields instead of complete();
    • the summary line comes from the report;
    • the strict run_info case opens current_v1_lite;
    • the mismatch text is built from the enum values;
    • field counts use kNumFields;
    • defaults are compared with RecoParticle{}.
  • check_spine_dump.cmake checks only the meta [Meta]: prefix.
  • test_core_real.cpp requires no missing field only for real files written by the model's SPINE version. The design did not list this check, but it failed the same way.

• tests/member_compare.hpp (new) holds the helpers the tests share:

  • equal_member(), moved from the golden test;
  • nan_equal(), which compares with NaN equal to NaN;
  • member_json(), used in messages;
  • all_nan().

• tools/release/scan_consumer_fields.py (new, Python 3.9+, standard library only) lists the model fields one CAFMaker build reads.

  • How it finds them. It compiles the CAFMaker's sources with -fsyntax-only against a copy of include/ whose members and Field enumerators are marked [[deprecated("SPINEFIELD Class.field")]]. Each deprecation in the CAFMaker's own tree is one read.
  • Input. --build takes a Makefile build, read with make -n -B; --compile-commands takes a compile_commands.json.
  • Environment. --setup and --wrap run the compiles in the CAFMaker's environment, for example the SL7 container.
  • Flags. -Wno-error and the other flags go last, after the CAFMaker's own -Werror; the output and dependency flags are removed, so nothing is written to the build.
  • Contract rows. --contracts tests/test_contracts.cpp lists the fields read that have no row (exit 1).
  • It is run by hand and is documented in tools/release/EGM.md.

• Docs.

  • The model EGM.md files no longer give the version or field counts.
  • The README, root EGM.md and tools/EGM.md list the new entry and tools/release/, with the rule that a row is added when a CAFMaker starts reading a field.
  • tools/gen/README.md step 4 says what to do when spine_reader_contracts fails.

Checked

• Expected changes. Each case below is a copy of this branch with one change, a regenerated model and member list, and a full build and ctest. The last two use local SPINE tags in a scratch clone, and their current fixtures were rewritten with make_fixtures.py --tag.

  Change	spine_reader_tests	spine_reader_contracts	on main
  schema relabelled 1.4.1	pass	pass	fails
  RecoParticle.csda_ke default nan → 0.0	pass	fails: "the model has 0.0, the contract "nan""	passes (not caught)
  its unit MeV → GeV	pass	fails: "the model has 'GeV', the contract 'MeV'"
  ParticlePID.MUON 2 → 7	pass	fails (MUON)
  csda_ke removed, fixtures not rewritten	fails (column outside the model)	fails: "listed for ND SBN, not in the model"
  tag v1.4.99: adds RecoParticle.test_score	pass	pass	fails (counts, field index, summary, doc)
  tag v1.4.88: removes crt_scores (read by no CAFMaker)	pass	pass

  The other entries (generated_model, golden_members, spine_dump_output) pass in every case.

• The scanner, on the three ports in the SL7 container: ND 47 fields, SBN 219, FD 31.

  • The sets equal those of the scan the table was written from: 228 in all, 24 read by all three, each with a row.
  • The ND ML filler is unchanged since fbda59e, the commit of that scan.
  • With one row removed, --contracts lists it and exits 1. A compile_commands.json input gives the same 47 fields.
  • The build folders and the ports' trees are unchanged afterwards.

• Builds:

  • ctest under LCG_110: 6 entries, 5 run and pass; spine_reader_real_files is skipped without files.
  • With SPINE_READER_REAL_FILES set to a real PDSP file, all 6 pass.
  • -Wall -Wextra -Wpedantic: no warnings.
  • ci/sl7-ups.sh (gcc 12, C++17): 132 test cases pass.
  • ci/sanitize.sh (ASan and UBSan): all entries pass.

• Lint: flake8 --max-line-length=100 on scan_consumer_fields.py gives no warnings.