Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
993649b
Bound the invocation prefilter and align table IDX like stock
ktsaou Nov 1, 2026
3865030
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 1, 2026
a89ed6b
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 1, 2026
f6be9b7
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 1, 2026
27c3cf4
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 1, 2026
1c6105d
Drop the trailing space inside a code span in SOW-0168
ktsaou Nov 1, 2026
b258c60
Remove trailing spaces inside Error: code spans in SOW-0122 and SOW-0168
ktsaou Nov 1, 2026
cad658a
Record SOW-0122 reproduction, aggregate results and full validation; …
ktsaou Nov 1, 2026
17ddf44
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 1, 2026
2f11219
List SOW-0166, SOW-0167 and SOW-0168 in the backlog order
ktsaou Nov 1, 2026
667bedb
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 3, 2026
247c8aa
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 3, 2026
e86de84
Record the user decisions for SOW-0122 and SOW-0168
ktsaou Nov 3, 2026
bcb5acf
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 3, 2026
9fb8790
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 3, 2026
86ca4e3
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 3, 2026
4cdc466
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 3, 2026
1dfff84
Apply 2026-11-03 backlog decisions to the SOW ledger
ktsaou Nov 3, 2026
b19938a
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 3, 2026
334107a
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 3, 2026
2a02a0f
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 4, 2026
e413409
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 4, 2026
f9399a6
Restore SOW ledger entries lost in the stack merge
ktsaou Nov 4, 2026
bd5957e
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 8, 2026
b16d4db
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 8, 2026
cffcb3d
Merge branch 'sow-0160/xz-dictionary' into sow-0122/journalctl-perf
ktsaou Nov 8, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 17 additions & 6 deletions .agents/sow/SOW-status.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ order; the user may reorder it.
1. SOW-0161 - File indexer source posting membership (completed 2026-11-01).
Comment thread
ktsaou marked this conversation as resolved.
Resolved
2. SOW-0160 - Go XZ reader dictionary allocation (completed 2026-11-04: Go
writer declares 256 KiB).
3. SOW-0122 - journalctl performance follow-ups.
3. SOW-0122 - journalctl performance follow-ups (completed 2026-11-01; gaps
tracked by SOW-0166, SOW-0167, SOW-0168).
4. SOW-0097 - Go duplication debt.
5. SOW-0098 - Rust legacy core / `jf` duplication debt (feeds SOW-0162).
6. SOW-0153 - Consumer docs release clarity.
Expand All @@ -25,11 +26,15 @@ order; the user may reorder it.
adopts Go's zero floor after a Netdata caller check).
13. SOW-0066 - v1.0.0 API freeze and release (not scheduled; user decision
2026-11-03).
14. SOW-0170 - Rust rewrite of Netdata systemd-journal.plugin (tracked; not
14. SOW-0166 - Directory realtime seek cost.
15. SOW-0167 - Go reader eager entry-array load.
16. SOW-0168 - journalctl follow wait and error parity (decided 2026-11-03:
cheap polling first).
17. SOW-0170 - Rust rewrite of Netdata systemd-journal.plugin (tracked; not
now, by user decision 2026-11-03).
15. SOW-0171 - Continue sealing when reopening a sealed file (tracked; SOW-0156
18. SOW-0171 - Continue sealing when reopening a sealed file (tracked; SOW-0156
decision N1, 2026-11-03).
16. SOW-0172 - Rust Log fails late on a relative directory (tracked 2026-11-03
19. SOW-0172 - Rust Log fails late on a relative directory (tracked 2026-11-03
from SOW-0156; needs a user decision).

## Current
Expand All @@ -44,8 +49,12 @@ None.
line numbers need refreshing at activation.
- SOW-0098 - Rust Legacy Core Duplication Debt: real `jf` duplication; feeds
SOW-0162.
- SOW-0122 - Journalctl Performance Followups: unmeasured large-input and
`--follow` scalability work for the CLIs.
- SOW-0166 - Directory Realtime Seek Cost: directory `--since` 17x slower than
stock (SOW-0122 measurement).
- SOW-0167 - Go Reader Eager Entry Array Load: every Go open loads the full
entry array.
- SOW-0168 - Journalctl Follow Wait And Error Parity: decided 2026-11-03
(cheap polling first).
- SOW-0125 - Netdata Filter Operators Gap Analysis: feature analysis; needs
semantic decisions before implementation.
- SOW-0153 - Consumer Documentation Release Clarity: stale `go/v0.3.0` text and
Expand Down Expand Up @@ -80,6 +89,8 @@ None.
read and continue every file earlier releases wrote. Follow-ups: SOW-0171,
SOW-0172.

- SOW-0122 - Journalctl Performance Followups: completed 2026-11-01;
`--list-invocations` 8-120x faster and IDX alignment matches stock.
- SOW-0161 - File Indexer Source Posting Membership: completed 2026-11-01;
source histograms count the captured ENTRY population once per entry.
- SOW-0137 - Rust Writer Residual Array-Open Performance: completed 2026-11-01.
Expand Down
425 changes: 425 additions & 0 deletions .agents/sow/done/SOW-0122-20260621-journalctl-performance-followups.md

Large diffs are not rendered by default.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# SOW-0166 - Directory Realtime Seek Cost

## Status

Status: open

Sub-state: measured gap from SOW-0122; not activated.

## Requirements

### Purpose

Directory realtime seeks (`--since`, `SeekRealtimeUsec` on directory readers)
should cost about what stock does, independent of how many entries precede the
target.

### User Request

Follow-up of SOW-0122 measurement (performance contract in `AGENTS.md`).

### Assistant Understanding

Facts (SOW-0122, `.local/sow0122/report.md`, 300-file 1.5M-entry directory):

- `--since/--until` 60 s window mid-directory: Rust 3.54 s, Go 3.46 s, stock
v262 0.21 s. `--since` at the last minute: Rust 17.3 s, Go 5.4 s, stock
0.16 s (one-off run). Single-file seeks are at parity.

Inferences (unverified):

- Rust `rust/src/journal/src/directory.rs` `fill_candidate` (around lines
375-392) may step and discard older entries file by file instead of seeking
each file by realtime or skipping files whose tail is older than the target.
The Go directory path is not traced yet.

### Acceptance Criteria

- Trace both languages, fix the per-file positioning to use journal-native
realtime seeks (entry-array bisection) and header head/tail bounds.
- On the SOW-0122 300-file directory fixture and procedure (SOW-0122
Implications And Decisions: 3 runs, median wall time), the 60 s
`--since/--until` window and the last-minute `--since` each run in at most
2x stock's median, or under 0.5 s when stock is below 0.25 s, in both
languages (proposed gate, agent-chosen; confirm at activation). The query
matrix and directory matrix still pass.

## Analysis

Pending activation.

## Pre-Implementation Gate

Not started.

## Implications And Decisions

None yet.

## Plan

Pending activation.

## Execution Log

### 2026-11-01

- Created from SOW-0122 measurements.

## Validation

Pending.

## Outcome

Pending.

## Lessons Extracted

None yet.

## Followup

None yet.

## Regression Log

None.
Loading
Loading