Skip to content

feat(FRBC): correct a usage forecast no plan can absorb, instead of clipping it - #286

Open
Flix6x wants to merge 7 commits into
dev/s2-frbc-completefrom
feat/s2-usage-correction
Open

Flix6x wants to merge 7 commits into
dev/s2-frbc-completefrom
feat/s2-usage-correction

Conversation

@Flix6x

@Flix6x Flix6x commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Stacked on #279 (base dev/s2-frbc-complete), as asked in the review of #271. Merge #279 first.

Why

S2 lets a resource manager expect any usage, negative included, and says nothing about what happens when the storage cannot take it. FlexMeasures plans a forecast usage as forced. So a gain at a full storage, or a draw at an empty one that the actuator cannot keep up with, leaves no feasible plan, and the storage gets no instruction. Until now the handler clipped negative usage to zero, which hid the first case and not the second.

What

With a usage_correction_sensor_id and FlexMeasures v1.1.0 or newer, FRBCSimple asks for a schedule with a correction of the usage forecast:

  • A second device on the storage's stock (same state-of-charge sensor), on a commodity of its own, so it never touches the site's electricity. No instruction is ever sent for it.

  • Per step, it may only move the expected usage towards zero: give back up to the expected usage, or take away up to the expected gain. It works the same on both sides, since S2's fill levels do not say which bound is an overflow.

  • It is priced as a last resort: at twice an upper bound on what one MWh of stock can be worth to the scheduler. The request carries a coherent set of prices to make that bound finite and numerically sound:

    • band breach prices at 10× the highest absolute market price over the window (soc_breach_price_factor);
    • the asset's own stored per-MW prices (peaks, commitments) added to the bound;
    • a breach of the site's capacity left out of the bound (see below), with the site's breach price kept from the asset, or else stated at FlexMeasures' default of 10000 per kW.

    Device capacities are kept hard, so the correction cannot move a usage past zero.

  • A negative expected usage is then recorded and planned as stated. The usage is sent inline, from the same forecast as the correction's limits and the translation back into instructions.

  • Where a correction cannot be planned (prices unreadable, a state-of-charge sensor with a resolution, a stored price that cannot be bounded, a stored flex context with commodities), the forecast is planned without its negative values, as before, with a warning saying why.

It is opt-in: FRBCSiteConfigurator(correct_usage=True), or S2_CORRECT_USAGE=1 for the example server. That is because it replaces FlexMeasures' default band breach price (1000 EUR/kWh) with a market-relative one, which is the deployment's call. Handlers without the new sensor behave exactly as before.

How the price was chosen

Probes against FlexMeasures v1.1.0 measured, per scenario, the correction price above which the scheduler stops using a correction it does not need. That threshold is what a MWh of stock is worth to the scheduler. Each matched one term of the bound:

What makes stock worth something Worth per MWh of stock
Buying or disposing of energy |price| / charging efficiency, divided again by the storage efficiency per step held
Flow commitment, peak, or any-breach price per MW price / step length
Site capacity breach P / step length + P (measured, but left out of the bound)
Fill-level band breach P / step length + P per step breached

With FlexMeasures' default breach prices, a dominant price was about 1e8 EUR/MWh, where HiGHS already returned over-corrected plans. With the coherent set, over a 24-hour window with prices from −120 to 450 EUR/MWh, the correction price is about 1.8e6 EUR/MWh:

  • It was used only where physics forced it.
  • It was never used instead of a band breach, or to profit from negative prices.
  • Solves took about 0.7 s and were identical on repeat.

Priced at the band's breach price instead, the correction replaced band breaches (9.24 vs 3.12 kWh) and corrected where nothing needed correcting.

Why the site's capacity breach is left out. Pricing the correction above a site breach needs the request to set a site breach price the solver can live with. In the co-simulation (below), a site breach price of 100× the market price kept the correction solvable but let the houses ignore their steering capacity, and 1000× pushed the correction price to 1.3e7 EUR/MWh, where HiGHS declared every house schedule infeasible (replays put the cliff between 0.7e7 and 1.3e7 for this house). So a correction can now be cheaper than a site breach, while it stays dearer than a band breach, which the plan still uses first.

FlexMeasures fills in its default site breach price only for a flex context without commodities; in a commodity entry, a site capacity without one stays hard. The request therefore states that default itself.

Evidence

  • Tests: 802 pass, 40 of them new; diff coverage 193 of 193 added statements.
  • Mutation gate: 35 of 35 mutants killed.
  • FlexMeasures v1.1.0 end to end: the exact request this handler builds, for a storage at 118 kWh under a 120 kWh hard top with a forced 14.4 kWh gain, is accepted and solved. The correction removes exactly 12.4 kWh, and the stock never exceeds 120. Without the correction device, the same request has no feasible plan.
  • Adversarial review: found that the stock read every forecast on the sensor while the limits used only the latest; that the asset's stored electricity flex context was dropped under a commodity list; that a stored relax-capacity-constraints would soften the limits; and that efficiencies above one lowered the bound. All are fixed in the second commit, which also makes the correction opt-in.

Co-simulation

One steered day, two houses, the community capacity policy, deterministic solver settings. Every run below had 0 degree-hours outside the comfort band.

Run Client Breach (kWh) Peak (kW) Notes
C1 #279, no correction 0.058 2.21 reference
D3 site breach 100× market 0.348 2.267 correction engaged, steering ineffective
D4 site breach 1000× market every house schedule infeasible (correction price 1.3e7), run aborted
D5 site breach left out, none stated site capacity hard in the commodity entry, schedules infeasible, stopped
D6 4b7b023 (this head) 0.052 2.21 no fallbacks, no infeasible jobs

D6 matches C1 on breach and peak. Cost differences between these runs are within what repeats of the same code showed on #279, so I draw no conclusion from them. One difference remains open: the community planner reports a planned residual breach of 2.419 kW at each step in D3 and D6, against at most 0.21 kW in C1, while the realized breach is the same.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW

Flix6x and others added 2 commits November 8, 2026 18:05
…lipping it

S2 lets a resource manager expect any usage, negative included, and says
nothing about what happens when the storage cannot take it. FlexMeasures
plans a forecast usage as forced, so a gain at a full storage, or a draw
at an empty one the actuator cannot keep up with, left no feasible plan.
The handler clipped negative usage to zero to avoid the first case.

With a usage_correction_sensor_id and FlexMeasures v1.1.0 or newer, the
handler now asks for a schedule with a correction instead: a second
device on the storage's stock, on a commodity of its own, which may only
move each step's expected usage towards zero. Its price is twice a bound
on what a unit of stock can be worth to the scheduler, computed from a
coherent set of prices the request carries: band breach prices of 10x,
and a site capacity breach price of 100x, the highest absolute market
price over the window. So it is used only where nothing physical keeps
the plan within the hard bounds. A negative usage is then recorded and
planned as stated. Where a correction cannot be priced or shared (no
prices, a state-of-charge sensor with a resolution), the forecast is
planned without its negative values, as before.

The site configurator and the example server give each site a "usage
correction" sensor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW
Signed-off-by: F.N. Claessen <claessen@seita.nl>
- The stock was planned against every forecast on the usage sensor, while
  the correction's limits and the translation back came from the latest
  one only, so an older forecast's gain had no correction. The stock now
  gets its usage inline, from the same series. Before any forecast has
  arrived since the handler started, the plan goes without usage, since
  the sensor may hold negative usage recorded as stated.
- A request with a commodity list made FlexMeasures ignore the site's
  stored electricity flex context: a contracted capacity or a peak price
  was lost. Those fields are carried into the electricity entry, and
  stored per-MW prices (peaks, commitments) go into the bound; one that
  is not a fixed price per MW leaves the plan without a correction.
- A stored relax-capacity-constraints would make the correction's limits
  soft; device capacities are now kept hard in the correction path.
- Efficiencies above one (a heat pump's COP, a storage that gains) no
  longer lower the bound.
- The correction is opt-in for sites: FRBCSiteConfigurator(correct_usage=)
  and the example server's S2_CORRECT_USAGE, since it also replaces
  FlexMeasures' default breach prices with market-relative ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x added this to the 0.10.0 milestone Nov 8, 2026
@Flix6x Flix6x added the S2 label Nov 8, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@coveralls

coveralls commented Nov 8, 2026 •

Copy link
Copy Markdown

Coverage Report for CI Build 42532459987

Warning

No base build found for commit 2e62cf2 on dev/s2-frbc-complete.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 96.168%

Details

  • Patch coverage: 197 of 197 lines across 3 files are fully covered (100%).

Uncovered Changes

No uncovered changes found.

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 3366
Covered Lines: 3237
Line Coverage: 96.17%
Coverage Strength: 4.43 hits per line

💛 - Coveralls

Flix6x and others added 4 commits November 8, 2026 19:08
…s with

The co-simulation's controller steers each home with peak prices it
stores on the site as sensor references. The bound accepted only fixed
prices per MW, so every plan fell back to planning without a correction,
in the very setting the correction is for. A stored price on a sensor is
now bounded by its highest absolute value over the window, read in the
market's currency per MW, as the market prices are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW
Signed-off-by: F.N. Claessen <claessen@seita.nl>
The first co-simulation run with the correction on had every request
refused with 422: the site's stored flex context holds aggregate-power,
which FlexMeasures accepts only at the top level, and the request copied
it into the electricity entry. Only the fields FlexMeasures' commodity
flex-context schema lists are copied now; the others stay where
FlexMeasures merges them in itself.

Checked against FlexMeasures v1.1.0's API: the request this handler
builds, against a stored context shaped like a co-simulation's (sensor
peak prices and capacities, inflexible devices, aggregate-power), is
accepted (202), and its job removes exactly the gain the hard top forces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Pricing the usage correction above a breach of the site's capacity forced
the request to set its own, lower site breach price: at FlexMeasures'
default, the correction price would exceed what HiGHS solves reliably.
In a steered co-simulation, a site breach price low enough to keep the
correction solvable made the houses ignore their steering capacity, and
one high enough to steer made every house's schedule infeasible.

The correction is now priced from the market and the fill-level band only,
and the site's breach prices are left to the asset or FlexMeasures. A
correction can then be cheaper than a site breach, but stays dearer than a
breach of the band.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ed request

FlexMeasures fills in its default site breach price (10000 per kW) only for
a flex context without commodities. In a commodity entry, a site capacity
without a breach price stays hard. A corrected request names its commodities,
so a steered site capacity became hard, and a house whose inflexible load
exceeded it had no feasible plan.

The electricity entry now states that default where the site's capacity may
be breached, unless the asset stores a breach price of its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpawsJs74cSwFLXkz7ZwW
Signed-off-by: F.N. Claessen <claessen@seita.nl>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants