You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
group_vfold_cv() the same group ends up in both the analysis and the assessment set #618
When a stratum makes up less than pool of the rows, group_vfold_cv() assigns
rows of the same group to several folds, although the strata are constant within
groups. As a result, rows appear in more than one assessment set, and the same
group ends up in both the analysis and the assessment set of a resample.
library(rsample)
# 12 groups; the stratum is constant within each group.# Stratum "d" (groups g11 and g12) has 15 of 570 rows, less than pool = 0.1.sizes<- c(100, 90, 80, 70, 60, 50, 40, 30, 20, 15, 10, 5)
dat<-data.frame(
group= rep(paste0("g", seq_along(sizes)), sizes),
stratum= rep(c("a", "b", "c", "a", "b", "c", "a", "b", "c", "a", "d", "d"), sizes)
)
set.seed(2)
folds<- group_vfold_cv(dat, group=group, v=4, strata=stratum)
# Each row should be held out exactly once:held_out<- unlist(lapply(folds$splits, complement))
c(n_rows= nrow(dat), n_held_out= length(held_out))
#> n_rows n_held_out#> 570 585# Groups held out in more than one fold:fold_groups<- lapply(folds$splits, function(s) unique(assessment(s)$group))
names(which(table(unlist(fold_groups)) >1))
#> [1] "g11" "g12"# With pool = 0 the partition is correct:
set.seed(2)
folds_no_pool<- group_vfold_cv(dat, group=group, v=4, strata=stratum, pool=0)
length(unlist(lapply(folds_no_pool$splits, complement)))
#> [1] 570
Cause.check_grouped_strata() runs before pooling. make_strata() then sets
rows of rare strata to NA and assigns them to existing strata row by row
(out[is.na(x)] <- sample(levels(out), size = num_miss, replace = TRUE)), so
one group gets several strata. balance_groups_strata() assigns a fold to every
unique (group, stratum) pair, and the full_join(..., by = "..group") in make_groups() duplicates the rows of such groups.
Related.group_vfold_splits() calls as.character(strata) before make_strata(), so numeric strata are not binned into quartiles as documented
for strata; every value becomes its own stratum and is pooled in the same way.
We would start by reproducing the reported behavior on the current main branch, then prepare a focused fix that preserves group integrity when pooling rare strata. Regression tests would check that analysis and assessment groups remain disjoint and that each observation appears in exactly one assessment set per repeat.
Would this approach be welcome, and is anyone already working on it?
Activity
algoria-contributions commentedon Nov 10, 2026
Hi, we’d be happy to work on this at Algoria.
We would start by reproducing the reported behavior on the current main branch, then prepare a focused fix that preserves group integrity when pooling rare strata. Regression tests would check that analysis and assessment groups remain disjoint and that each observation appears in exactly one assessment set per repeat.
Would this approach be welcome, and is anyone already working on it?
Thanks!
Algoria