Skip to content

[release/0.22] Avoid reparsing YAML documents in nested decoders - #1283

Merged
cpuguy83 merged 1 commit into
project-dalec:release/0.22from
cpuguy83:cpuguy83-backport-yaml-decoding
Oct 29, 2026
Merged

cpuguy83 merged 1 commit into
project-dalec:release/0.22from
cpuguy83:cpuguy83-backport-yaml-decoding

Conversation

@cpuguy83

@cpuguy83 cpuguy83 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Backports #1278 (1554e76) without release-specific adaptations or dependency changes.

Initializes each fresh nested decoder's anchors from the original AST instead of stringifying and reparsing the spec. Preserves alias-context isolation and original source positions, with regression tests and self-contained benchmarks.

Populate each nested decoder's anchor lookups from the original AST instead of serializing and reparsing the entire document. Preserve fresh decoder isolation and original source locations, with regression coverage for aliases, merges, decoder options, errors, and independent loads.

Self-contained synthetic specs, Go 1.26.8, GOMAXPROCS=2. Both versions use the same b.Loop benchmark harness and `-run '^$' -bench '^BenchmarkDecodeSpec$' -benchtime=1s -count=10 -benchmem`. Before uses load.go from c6da490; after uses this change.

```text
goos: linux
goarch: amd64
pkg: github.com/project-dalec/dalec
cpu: AMD Ryzen 7 9800X3D 8-Core Processor
                                                           │   before.txt   │              after.txt              │
                                                           │     sec/op     │   sec/op     vs base                │
DecodeSpec/simple/Unmarshal-2                                  163.38µ ± 2%   57.57µ ± 2%  -64.77% (p=0.000 n=10)
DecodeSpec/simple/LoadSpec-2                                   166.62µ ± 5%   59.95µ ± 1%  -64.02% (p=0.000 n=10)
DecodeSpec/simple/LoadSpecWithSourceMap-2                      164.70µ ± 1%   60.05µ ± 1%  -63.54% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/Unmarshal-2                     608.2µ ± 2%   115.5µ ± 2%  -81.02% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/LoadSpec-2                      613.5µ ± 2%   121.1µ ± 1%  -80.25% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/LoadSpecWithSourceMap-2         619.5µ ± 2%   121.0µ ± 2%  -80.46% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/Unmarshal-2                   9392.4µ ± 1%   716.9µ ± 1%  -92.37% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/LoadSpec-2                    9391.7µ ± 2%   746.0µ ± 2%  -92.06% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/LoadSpecWithSourceMap-2       9548.3µ ± 2%   745.9µ ± 2%  -92.19% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/Unmarshal-2                   62.259m ± 2%   2.485m ± 1%  -96.01% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/LoadSpec-2                    61.093m ± 1%   2.553m ± 2%  -95.82% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/LoadSpecWithSourceMap-2       61.351m ± 1%   2.544m ± 3%  -95.85% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/Unmarshal-2                 284.922m ± 2%   3.488m ± 2%  -98.78% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/LoadSpec-2                  273.754m ± 2%   3.463m ± 1%  -98.74% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/LoadSpecWithSourceMap-2     284.563m ± 2%   3.511m ± 2%  -98.77% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/Unmarshal-2               1719.320m ± 4%   7.599m ± 2%  -99.56% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/LoadSpec-2                1723.440m ± 2%   7.761m ± 3%  -99.55% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/LoadSpecWithSourceMap-2   1710.209m ± 2%   7.616m ± 3%  -99.55% (p=0.000 n=10)
geomean                                                         17.47m        839.6µ       -95.19%

                                                           │  before.txt  │                 after.txt                 │
                                                           │     B/s      │      B/s        vs base                   │
DecodeSpec/simple/Unmarshal-2                                1.149Mi ± 2%     3.262Mi ± 2%    +183.82% (p=0.000 n=10)
DecodeSpec/simple/LoadSpec-2                                 1.125Mi ± 4%     3.133Mi ± 1%    +178.39% (p=0.000 n=10)
DecodeSpec/simple/LoadSpecWithSourceMap-2                    1.140Mi ± 1%     3.128Mi ± 1%    +174.48% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/Unmarshal-2                  2.193Mi ± 2%    11.563Mi ± 2%    +427.17% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/LoadSpec-2                   2.174Mi ± 1%    11.024Mi ± 1%    +407.02% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/LoadSpecWithSourceMap-2      2.155Mi ± 2%    11.029Mi ± 2%    +411.73% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/Unmarshal-2                 6.695Mi ± 1%    87.681Mi ± 1%   +1209.69% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/LoadSpec-2                  6.690Mi ± 2%    84.267Mi ± 2%   +1159.59% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/LoadSpecWithSourceMap-2     6.580Mi ± 2%    84.267Mi ± 2%   +1180.58% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/Unmarshal-2                 48.83Ki ± 0%   1201.17Ki ± 2%   +2360.00% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/LoadSpec-2                  48.83Ki ± 0%   1166.99Ki ± 3%   +2290.00% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/LoadSpecWithSourceMap-2     48.83Ki ± 0%   1171.88Ki ± 3%   +2300.00% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/Unmarshal-2                234.4Ki ± 4%   18916.0Ki ± 2%   +7970.83% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/LoadSpec-2                 244.1Ki ± 4%   19052.7Ki ± 1%   +7704.00% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/LoadSpecWithSourceMap-2    234.4Ki ± 4%   18793.9Ki ± 2%   +7918.75% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/Unmarshal-2               302.7Ki ± 3%   67641.6Ki ± 2%  +22243.55% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/LoadSpec-2                297.9Ki ± 2%   66225.6Ki ± 3%  +22134.43% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/LoadSpecWithSourceMap-2   302.7Ki ± 0%   67495.1Ki ± 3%  +22195.16% (p=0.000 n=10)
geomean                                                      628.7Ki          12.72Mi        +1971.81%

                                                           │   before.txt   │               after.txt               │
                                                           │      B/op      │     B/op       vs base                │
DecodeSpec/simple/Unmarshal-2                                 227.81Ki ± 0%   77.88Ki ±  0%  -65.81% (p=0.000 n=10)
DecodeSpec/simple/LoadSpec-2                                  231.47Ki ± 0%   81.50Ki ±  0%  -64.79% (p=0.000 n=10)
DecodeSpec/simple/LoadSpecWithSourceMap-2                     231.58Ki ± 0%   81.61Ki ±  0%  -64.76% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/Unmarshal-2                    834.4Ki ± 0%   169.9Ki ±  0%  -79.64% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/LoadSpec-2                     841.9Ki ± 0%   176.6Ki ±  0%  -79.02% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/LoadSpecWithSourceMap-2        841.9Ki ± 0%   176.7Ki ±  0%  -79.02% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/Unmarshal-2                  37.291Mi ± 1%   1.184Mi ±  2%  -96.82% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/LoadSpec-2                   37.341Mi ± 1%   1.219Mi ±  1%  -96.74% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/LoadSpecWithSourceMap-2      37.412Mi ± 1%   1.223Mi ±  2%  -96.73% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/Unmarshal-2                  72.698Mi ± 0%   2.848Mi ±  0%  -96.08% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/LoadSpec-2                   72.763Mi ± 0%   2.916Mi ±  0%  -95.99% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/LoadSpecWithSourceMap-2      72.767Mi ± 0%   2.916Mi ±  0%  -95.99% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/Unmarshal-2                913.227Mi ± 1%   5.461Mi ±  1%  -99.40% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/LoadSpec-2                 921.721Mi ± 0%   5.380Mi ±  1%  -99.42% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/LoadSpecWithSourceMap-2    903.451Mi ± 1%   5.411Mi ±  2%  -99.40% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/Unmarshal-2               8907.17Mi ± 0%   15.62Mi ±  5%  -99.82% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/LoadSpec-2                8887.07Mi ± 0%   16.61Mi ±  8%  -99.81% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/LoadSpecWithSourceMap-2   8887.06Mi ± 0%   16.62Mi ± 12%  -99.81% (p=0.000 n=10)
geomean                                                        39.95Mi        1.266Mi        -96.83%

                                                           │  before.txt   │              after.txt              │
                                                           │   allocs/op   │  allocs/op   vs base                │
DecodeSpec/simple/Unmarshal-2                                  4.625k ± 0%   1.309k ± 0%  -71.70% (p=0.000 n=10)
DecodeSpec/simple/LoadSpec-2                                   4.670k ± 0%   1.354k ± 0%  -71.01% (p=0.000 n=10)
DecodeSpec/simple/LoadSpecWithSourceMap-2                      4.672k ± 0%   1.356k ± 0%  -70.98% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/Unmarshal-2                   15.072k ± 0%   2.465k ± 0%  -83.65% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/LoadSpec-2                    15.138k ± 0%   2.531k ± 0%  -83.28% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=1024/LoadSpecWithSourceMap-2       15.139k ± 0%   2.532k ± 0%  -83.28% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/Unmarshal-2                  24.744k ± 0%   2.477k ± 0%  -89.99% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/LoadSpec-2                   24.810k ± 0%   2.542k ± 0%  -89.75% (p=0.000 n=10)
DecodeSpec/nodes=1/padding=65536/LoadSpecWithSourceMap-2      24.814k ± 0%   2.545k ± 0%  -89.74% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/Unmarshal-2                 1679.17k ± 0%   39.56k ± 0%  -97.64% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/LoadSpec-2                  1679.92k ± 0%   40.31k ± 0%  -97.60% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=1024/LoadSpecWithSourceMap-2     1679.92k ± 0%   40.31k ± 0%  -97.60% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/Unmarshal-2                2511.81k ± 0%   39.59k ± 0%  -98.42% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/LoadSpec-2                 2512.75k ± 0%   40.34k ± 0%  -98.39% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=65536/LoadSpecWithSourceMap-2    2512.24k ± 0%   40.34k ± 0%  -98.39% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/Unmarshal-2               3104.65k ± 0%   39.57k ± 0%  -98.73% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/LoadSpec-2                3105.40k ± 0%   40.32k ± 0%  -98.70% (p=0.000 n=10)
DecodeSpec/nodes=32/padding=524288/LoadSpecWithSourceMap-2   3105.36k ± 0%   40.32k ± 0%  -98.70% (p=0.000 n=10)
geomean                                                        168.5k        9.039k       -94.63%
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 1554e76)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
@cpuguy83
cpuguy83 force-pushed the cpuguy83-backport-yaml-decoding branch from 64cbe52 to 41c8935 Compare October 29, 2026 17:33
@cpuguy83
cpuguy83 merged commit 8828872 into project-dalec:release/0.22 Oct 29, 2026
64 of 69 checks passed
@cpuguy83
cpuguy83 deleted the cpuguy83-backport-yaml-decoding branch October 29, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants