Skip to content

Add source-backed coverage planning, witness shrinking and replay - #58

Open
davidkpiano wants to merge 3 commits into
codex/xstate-v6-parityfrom
codex/source-coverage-planning
Open

davidkpiano wants to merge 3 commits into
codex/xstate-v6-parityfrom
codex/source-coverage-planning

Conversation

@davidkpiano

@davidkpiano davidkpiano commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Adds a compiler/runtime-independent source coverage protocol to the standalone graph library. The denominator comes from an independently extracted manifest; zero-hit outcomes remain in reports. Search can combine explicit event cases, schema/PBT cases and async targeted or LLM suggestions, but only executed source observations count.

import {
  getSourceCoverageWitnesses,
  getSourceCoverageReplay
} from '@statelyai/graph';

const result = await getSourceCoverageWitnesses({
  manifest,
  createSession,
  events: cases,
  suggestEvents: async ({ state, prefix, uncovered }) => [],
  maxDepth: 32,
  maxExecutions: 100000
});
for (const witness of result.witnesses) {
  await getSourceCoverageReplay(manifest, witness, createSession);
}

The API includes an async witness generator, suite reduction, sequence/payload shrinking through verified replay, machine-scoped reporting, explicit search bounds and execution errors. Each candidate starts from a fresh session and replays its prefix; stopped generators close active sessions. Source hash/runtime mismatches and unknown outcome IDs fail. Known child-machine observations are filtered from parent-scoped evidence; per-machine witnesses can be combined for full-file reporting.

Complete means extraction is complete and every required source outcome has a witness. Search exhaustion does not prove unreachability. This criterion does not claim all condition combinations, context values, sequences, schedules or SUT properties. JSON witnesses are evidence to replay, not self-authenticating proof.

Validation: 114 files / 4,292 tests passed, including a shared real-v6 source conformance fixture and tests for missing denominators, suggestions with setup prefixes, streaming cleanup, shrinking, stale/unknown identities, machine scope and budgets. Both typechecks, conventions, generated schema, build, publint and packed consumer smoke passed. Verify components requiring tsx IPC used equivalent node --import tsx commands because that socket is blocked in this environment.

Stacked on #57.
AST/instrumentation: https://github.com/statelyai/viz/pull/934.
Execution sessions: statelyai/xstate#5826.

Coverage stack: graph parity, graph planning/replay, AST + SDK, XState sessions, CLI + Viz graph.

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9f47c356-dfa4-4010-ad25-ca1db7cc161c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@davidkpiano
davidkpiano marked this pull request as ready for review October 26, 2026 20:33

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 4 potential issues.

Devin Review

Comment thread src/source-coverage.ts Outdated
Comment thread src/source-coverage.ts Outdated
Comment thread src/source-coverage.ts Outdated
Comment thread src/source-coverage.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant