Apply V-Fold Cross-Validation to Nested Data

nest_cv creates v-fold cross-validation splits for each nested data frame of a nested data.table (e.g. the output of w2l_nest()) and returns one row per fold, with the outer (non-nested) columns broadcast to every fold. The cross-validation itself is performed by split_cv().

nest_cv(
  data,
  v = 10L,
  repeats = 1L,
  strata = NULL,
  breaks = 4L,
  pool = 0.1,
  seed = NULL,
  materialize = TRUE,
  out_type = "dt"
)

Arguments

data: A data.frame or data.table containing at least one nested data.frame/data.table column. A column named data is used if present, otherwise the first nested column.
v: Number of folds. A single integer >= 2. Default 10.
repeats: Number of repeats. A single integer >= 1. Default 1.
strata: NULL (default) or a single column name present in every dataset. Rows are shuffled within each stratum so that every fold receives a proportional share of each stratum.
breaks: Number of quantile bins used when strata is numeric with more than breaks distinct values. Default 4.
pool: Strata holding less than this proportion of the rows are merged with the next-smallest stratum. A number in [0, 0.5). Default 0.1.
seed: NULL (default) or a single number for reproducible folds. The caller's random-number stream is restored on exit.
materialize: Logical. If TRUE (default), train / validate list-columns holding the subsets are added. FALSE keeps only the row indices, which needs far less memory for large data.
out_type: Class of the train / validate subsets: "dt" (data.table, default) or "df" (plain data.frame, e.g. for 'ASReml-R' or other modelling functions that expect a data.frame). Factor levels are kept in full in every subset. Only used when materialize = TRUE; the returned fold table itself is always a data.table.

Value

A data.table with all non-nested columns of data, followed by the columns described in split_cv() (id, id2, train_idx, validate_idx and, if materialize = TRUE, train / validate, whose class follows out_type). Other nested columns are dropped (a message lists them).

Examples

# Example: Cross-validation for nested data.table demonstrations

# Setup test data
dt_nest <- w2l_nest(
  data = iris,                   # Input dataset
  cols = 1:2                     # Nest first 2 columns
)

# Example 1: Basic 2-fold cross-validation (reproducible)
nest_cv(
  data = dt_nest,                # Input nested data.table
  v = 2,                         # Number of folds (2-fold CV)
  seed = 123                     # Reproducible folds
)
#>            name     id                 train_idx              validate_idx
#>          <char> <char>                    <list>                    <list>
#> 1: Sepal.Length  Fold1       1,2,3,5,6,7,...[75]  4, 8, 9,11,12,15,...[75]
#> 2: Sepal.Length  Fold2  4, 8, 9,11,12,15,...[75]       1,2,3,5,6,7,...[75]
#> 3:  Sepal.Width  Fold1       1,2,3,4,6,9,...[75]  5, 7, 8,11,12,13,...[75]
#> 4:  Sepal.Width  Fold2  5, 7, 8,11,12,13,...[75]       1,2,3,4,6,9,...[75]
#>                 train           validate
#>                <list>             <list>
#> 1: <data.table[75x4]> <data.table[75x4]>
#> 2: <data.table[75x4]> <data.table[75x4]>
#> 3: <data.table[75x4]> <data.table[75x4]>
#> 4: <data.table[75x4]> <data.table[75x4]>

# Example 2: Repeated 2-fold CV, keeping only the split objects
nest_cv(
  data = dt_nest,                # Input nested data.table
  v = 2,                         # Number of folds (2-fold CV)
  repeats = 2,                   # Number of repetitions
  seed = 123,
  materialize = FALSE            # No train/validate copies (saves memory)
)
#>            name      id    id2                 train_idx
#>          <char>  <char> <char>                    <list>
#> 1: Sepal.Length Repeat1  Fold1       1,2,3,5,6,7,...[75]
#> 2: Sepal.Length Repeat1  Fold2  4, 8, 9,11,12,15,...[75]
#> 3: Sepal.Length Repeat2  Fold1       1,2,3,4,6,9,...[75]
#> 4: Sepal.Length Repeat2  Fold2  5, 7, 8,11,12,13,...[75]
#> 5:  Sepal.Width Repeat1  Fold1  2, 3, 7, 8, 9,11,...[75]
#> 6:  Sepal.Width Repeat1  Fold2  1, 4, 5, 6,10,14,...[75]
#> 7:  Sepal.Width Repeat2  Fold1  3, 9,10,14,15,16,...[75]
#> 8:  Sepal.Width Repeat2  Fold2       1,2,4,5,6,7,...[75]
#>                 validate_idx
#>                       <list>
#> 1:  4, 8, 9,11,12,15,...[75]
#> 2:       1,2,3,5,6,7,...[75]
#> 3:  5, 7, 8,11,12,13,...[75]
#> 4:       1,2,3,4,6,9,...[75]
#> 5:  1, 4, 5, 6,10,14,...[75]
#> 6:  2, 3, 7, 8, 9,11,...[75]
#> 7:       1,2,4,5,6,7,...[75]
#> 8:  3, 9,10,14,15,16,...[75]

# Example 3: data.frame subsets, ready for ASReml-R / lm() / glm()
cv_df <- nest_cv(dt_nest, v = 2, seed = 123, out_type = "df")
class(cv_df$train[[1]])        # "data.frame"
#> [1] "data.frame"

# Example 4: masking-style CV (keep all rows, hide validation phenotypes)
cv_idx <- nest_cv(dt_nest, v = 2, seed = 123, materialize = FALSE)
cv_idx[dt_nest, on = "name", full := i.data]   # attach the full nested table
#>            name     id                 train_idx              validate_idx
#>          <char> <char>                    <list>                    <list>
#> 1: Sepal.Length  Fold1       1,2,3,5,6,7,...[75]  4, 8, 9,11,12,15,...[75]
#> 2: Sepal.Length  Fold2  4, 8, 9,11,12,15,...[75]       1,2,3,5,6,7,...[75]
#> 3:  Sepal.Width  Fold1       1,2,3,4,6,9,...[75]  5, 7, 8,11,12,13,...[75]
#> 4:  Sepal.Width  Fold2  5, 7, 8,11,12,13,...[75]       1,2,3,4,6,9,...[75]
#>                   full
#>                 <list>
#> 1: <data.table[150x4]>
#> 2: <data.table[150x4]>
#> 3: <data.table[150x4]>
#> 4: <data.table[150x4]>
masked <- as.data.frame(cv_idx$full[[1]])      # a copy: dt_nest stays intact
masked$value[cv_idx$validate_idx[[1]]] <- NA
sum(is.na(masked$value))       # validation records to be predicted
#> [1] 75

Apply V-Fold Cross-Validation to Nested Data

Arguments

Value

See also

Examples