Skip to content

feat: implement sorted bins QPR - #596

Closed
cheb0 wants to merge 1 commit into
0-single-source-no-tokens-cachefrom
0-sorted-QPR
Closed

cheb0 wants to merge 1 commit into
0-single-source-no-tokens-cachefrom
0-sorted-QPR

Conversation

@cheb0

@cheb0 cheb0 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Description

All QPRs bins for every agg are now sorted by (token asc, MID desc).

  • Faster merge for high-cardinality aggs. Merging works on top of either heap (k-way) or two pointers. It's roughly x2 speedup for _exists_:remote_addr | by remote_addr: 6 sec => 3 sec (4 search workers and 20 fractions)
  • Protobuf format (proxy-store) is not changed. A new store protocol version reports if QPR is sorted.
  • Async search QPR now has a new version - 3. If version >= 3, then no sorting is needed. It's possible to do external merge for QPRs.
  • every Aggregator implementation stills calls sort on bins. Since some aggregator now yield bins in same order as sorting works, I suppose this sort shall work faster than before.

@github-actions

Copy link
Copy Markdown
Contributor

❌ Branch Name Validation Failed
Please refer to CONTRIBUTING.md

@github-actions

Copy link
Copy Markdown
Contributor

❌ Branch Name Validation Failed
Please refer to CONTRIBUTING.md

@cheb0
cheb0 marked this pull request as draft October 30, 2026 08:30
@github-actions

Copy link
Copy Markdown
Contributor

❌ Branch Name Validation Failed
Please refer to CONTRIBUTING.md

@cheb0
cheb0 marked this pull request as ready for review November 2, 2026 05:15
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

❌ Branch Name Validation Failed
Please refer to CONTRIBUTING.md

4 similar comments
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

❌ Branch Name Validation Failed
Please refer to CONTRIBUTING.md

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

❌ Branch Name Validation Failed
Please refer to CONTRIBUTING.md

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

❌ Branch Name Validation Failed
Please refer to CONTRIBUTING.md

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

❌ Branch Name Validation Failed
Please refer to CONTRIBUTING.md

@eguguchkin eguguchkin added this to the v0.83.0 milestone Nov 2, 2026
@eguguchkin
eguguchkin requested review from eqimd and forshev November 2, 2026 10:29
@cheb0 cheb0 closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants