Skip to content

DDIR: corgi pin to 708ca28 (range into gather, foldhash rows, faster byte sorts) - #1067

Open
frankmcsherry wants to merge 1 commit into
TimelyDataflow:master-nextfrom
frankmcsherry:ddir-corgi-pin-708ca28
Open

frankmcsherry wants to merge 1 commit into
TimelyDataflow:master-nextfrom
frankmcsherry:ddir-corgi-pin-708ca28

Conversation

@frankmcsherry

Copy link
Copy Markdown
Member

Moves DDIR's corgi pin from 2751e04 to 708ca28, bringing in the three corgi PRs merged today:

  • WIP#130: a range read only by a gather runs as one span copy, without writing out positions. ClickBench 0.898×.
  • WIP#131: byte and word rows hash with an in-tree copy of foldhash. The Hash op over 13 ClickBench queries goes from 179 to 41 ms, and DDIR's hash-prefixed keys change once.
  • WIP#132: byte lists sort seven bytes per pass, with cheaper order checks. Measured in DDIR at the old pin, LDBC SF1 was 4.7% faster.

One code change. The lambda printer has to spell corgi's new fused GatherRange op, which now appears in optimized graphs (for example in the top-20 reduces of IC2, IC5 and IC9). It prints the range and gather the fusion stands for, spelled as they were before the fusion: a tuple read only by the fused node keeps its fields' spellings for it. Printed plans are unchanged, and the plan-pairs corpus passes without updates.

Checks: cargo test -p differential-dataflow -p differential-dogs3 -p interactive -p ddir-server (484 passed), the plan-pairs corpus, and the 1,000-case release difftest. --check-lambdas round-trips a lambda containing the fused op.

🤖 Generated with Claude Code

https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o

…byte sorts)

Moves the corgi pin from 2751e04 to 708ca28, which brings three merged corgi
changes:

- WIP#130: a `range` read only by a `gather` runs as one span copy.
- WIP#131: byte and word rows hash with an in-tree copy of foldhash. DDIR's
  hash-prefixed keys change once.
- WIP#132: byte lists sort seven bytes a pass, with cheaper order checks.

The lambda printer meets corgi's fused `GatherRange` in optimized graphs. It
prints the `range` and `gather` the fusion stands for, spelled as before the
fusion, so printed plans are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant