Skip to content

perf(proof): store witness commitments as indexed affine arrays - #164

Open
flyq wants to merge 8 commits into
liquan/perf/vector-msmfrom
liquan/perf/compact-witness-storage
Open

flyq wants to merge 8 commits into
liquan/perf/vector-msmfrom
liquan/perf/compact-witness-storage

Conversation

@flyq

@flyq flyq commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Ports PR #5 of suryuhh/salt-optimizations (commits 37d3c93, 6b4ff37, 7c435b7, 52a460c, d62dedb): how a proof holds its node commitments and how the verifier reads them.

  • A proof's commitments are two sorted vectors, node ids and 64-byte affine points, instead of a BTreeMap of 128-byte projective points.
  • Verifier queries name their commitment by index into that list. The MSM runs over the list itself instead of one copied point per query.
  • The transcript hashes bytes as they are appended instead of buffering them until the next challenge.

The wire format does not change.

Stacked on the vector MSM PR (liquan/perf/vector-msm).

Breaking API changes

  • SaltProof.parents_commitments is a PathCommitments, not a BTreeMap<NodeId, SerdeCommitment>. It keeps len, get, contains_key, keys, values, iter, insert and remove, but they take a NodeId by value.
  • SerdeCommitment wraps an AffineElement, not an Element.
  • ipa_multipoint::transcript::Transcript.state is gone.

New: PathCommitments, AffineElement, multi_scalar_mul_affine, IndexedQuery, MultiPointProof::check_indexed.

Changes on top of the external commits

  • The main-trie walk of sorted_parents_and_points is bounded. The verifier now builds its queries from this new function, which walked from a bucket root to the root with while node != 0. fix(salt): allow 2^40 bucket capacity and validate what proofs and witnesses decode #150 bounded the same walk in record_key_paths by the trie's fixed depth; this walk gets the same bound.
  • words_to_limbs lives in the shared IFMA module. The external code took it from the batched-commitment module, which lands after this PR in this series. Without the move the crate does not build on x86_64.
  • Tests for what the mutation gates found, described below.

The decode preallocation is capped at 65,536 entries by the external series' own last commit.

Cherry-pick conflicts were shallow: imports, multi_scalar_mul_par (already deleted by #152), and two type changes in create_sub_trie.

Mutation gates

The diff-scoped gates left 15 survivors.

Eleven were real gaps and are now killed by four tests:

  • PathCommitments::is_empty and contains_key.
  • Decoding a wire map that is not in key order or repeats an id. It must give what a BTreeMap of the same entries gives.
  • The error for a value that is not a map.
  • create_indexed_queries reading a leaf pair's value from the slot the pair names. If a proof declares one level for a bucket and a key lies beyond the first segment, the pair names another key's slot. The queries must read that key from kvs or refuse, as the old form did, and never use the value of the key that produced the pair.

Four are equivalent and suppressed with their reasons in mutants/suppressions.toml: the two position-reuse guards, the dedup of node ids that are already distinct, and a boundary already reviewed in create_leaf_node_queries (now test-only, so its entry moves to the new function).

Result on this head: cargo-mutants 65 of 65 viable mutants killed (3 suppressed, 34 unviable); spec-gate 2 of 2 killed (1 suppressed); suppression hygiene passes.

Testing

Passing on this head, on an Apple Silicon machine:

  • cargo fmt --all -- --check; cargo clippy --all-targets -- -D warnings, native and --target x86_64-unknown-linux-gnu
  • cargo test; with test-bucket-resize (CI's settings); the random stress test; --no-default-features with and without test-bucket-resize. These include the byte-level round trip of the committed proof fixtures (salt/tests/proof_fixture_codec.rs).
  • cargo check -p salt --target riscv64imac-unknown-none-elf --no-default-features
  • cargo check --all-targets --target i686-unknown-linux-gnu

Not run:

  • Any AVX-512 path (ifma_normalize.rs and the vector decode inside PathCommitments' deserializer). No such CPU or emulator on the machine used; compiled and linted for x86_64 only.
  • The i686 test job.
  • Benchmarks on this head.

Merging

Please keep this trailer in the squash message:

Co-authored-by: suryuhh <suryuhh@users.noreply.github.com>

🤖 Generated with Claude Code

suryuhh and others added 8 commits November 14, 2026 14:27
(cherry picked from commit 37d3c9362166c3671e2cb771fc67d0acef1166a0)
(cherry picked from commit 6b4ff3761575793e1fbddf037476a9b9c3b6f3b4)
(cherry picked from commit 7c435b76fbdd88478a9325b117c92563f106da2a)
(cherry picked from commit 52a460c12b03e6c352f7c9a4cb207e38fc244bbf)
(cherry picked from commit d62dedba5fa61b164ce17f0953d3a3d39f534894)
The verifier now builds its queries from sorted_parents_and_points, which
walked from a bucket root to the trie root with `while node != 0`. #150
bounded the same walk in record_key_paths by the trie's fixed depth so a
corrupt parent id cannot spin forever; give this walk the same bound.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
ifma_normalize took words_to_limbs from ifma_committer, a module of the
batched trie commitment change, which lands after this one here. Without
it the crate does not build on x86_64. Define the helper next to its
inverse, limbs_to_words, so both changes take it from one place.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ir names

The diff-scoped mutation gates left 15 survivors in the ported code.

Eleven were real gaps, now covered: PathCommitments' is_empty and
contains_key; its decode of a wire map that is not in key order or
repeats an id (it must give what a BTreeMap gives); the message for a
value that is not a map; and create_indexed_queries taking a leaf pair's
value from the slot the pair names. That last one matters when a proof
declares one level for a bucket and the key lies beyond the first
segment: the pair then names another key's slot, and the queries must
read that key from kvs or refuse, not use the value of the key that
produced the pair.

Four are equivalent and suppressed: the two position-reuse guards, the
dedup of node ids that are already distinct, and the boundary already
reviewed in create_leaf_node_queries, whose entry moves with the code now
that the old function is test-only.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.73504% with 102 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
banderwagon/src/ifma_normalize.rs 85.89% 44 Missing and 12 partials ⚠️
salt/src/proof/prover.rs 93.65% 21 Missing and 5 partials ⚠️
banderwagon/src/element.rs 93.41% 9 Missing and 8 partials ⚠️
ipa-multipoint/src/transcript.rs 84.61% 2 Missing ⚠️
ipa-multipoint/src/multiproof.rs 99.39% 1 Missing ⚠️
Files with missing lines Coverage Δ
banderwagon/src/ifma.rs 92.31% <100.00%> (+90.77%) ⬆️
banderwagon/src/lib.rs 70.00% <ø> (ø)
salt/src/proof/salt_witness.rs 98.43% <100.00%> (ø)
salt/src/proof/shape.rs 98.91% <100.00%> (+0.37%) ⬆️
salt/src/proof/subtrie.rs 97.70% <100.00%> (ø)
salt/src/proof/test_utils.rs 100.00% <100.00%> (ø)
ipa-multipoint/src/multiproof.rs 99.84% <99.39%> (-0.16%) ⬇️
ipa-multipoint/src/transcript.rs 95.50% <84.61%> (+3.37%) ⬆️
banderwagon/src/element.rs 92.63% <93.41%> (+12.90%) ⬆️
salt/src/proof/prover.rs 96.95% <93.65%> (-1.61%) ⬇️
... and 1 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Performance Benchmark Comparison

Compared 5 benchmark(s) against the latest main baseline.

Detailed Comparison
Benchmark Baseline Throughput (Kelem/s) New Throughput (Kelem/s) Change
update 10000 KVs/1 threads 83.52 86.48 +3.55%
update 10000 KVs/2 threads 160.32 164.45 +2.58%
update 10000 KVs/4 threads 299.94 303.74 +1.27%
update 10000 KVs/8 threads 490.69 498.42 +1.58%
update 10000 KVs/16 threads 617.42 672.47 +8.92%

@github-actions

Copy link
Copy Markdown

Mutation testing - PASS

Mutation score: 100.0% (2/2 viable mutants killed)

  • caught: 2
  • survived, real gaps: 0
  • timed out, inconclusive: 0
  • suppressed: 1
  • unviable: 0
  • timeout total: 0

No unsuppressed survivors or timeouts remain.

@github-actions

Copy link
Copy Markdown

Mutation testing - PASS

Mutation score: 100.0% (46/46 viable mutants killed)

  • caught: 46
  • survived, real gaps: 0
  • timed out, inconclusive: 0
  • suppressed: 1
  • unviable: 55
  • timeout total: 0

No unsuppressed survivors or timeouts remain.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants