Reshape Wide Data to Long Format and Nest by Specified Columns

Source: R/w2l_nest.R

w2l_nest reshapes a wide-format data.frame or data.table into long format, then nests the result by name (the pivoted column identifier) and any optional grouping variables supplied via by. Each row of the returned table contains a nested data.table or data.frame in the data list-column.

w2l_nest(data, cols2l = NULL, by = NULL, nest_type = "dt")

Arguments

data

data.frame or data.table. Wide-format input dataset. Converted in-place to data.table via setDT() if necessary (no copy).

cols2l

numeric or character. Columns to pivot from wide to long, specified as integer indices or column names. Default NULL: when NULL, by must be provided and the function performs a pure nest operation (no melting).

by

numeric or character. Additional grouping variables for hierarchical nesting, specified as integer indices or column names. Default NULL.

nest_type

character. Class of each nested subset: "dt" for data.table (default) or "df" for data.frame.

Value

A data.table with one row per combination of name (and by levels, if provided). The data list-column holds the corresponding nested data.table or data.frame for each group. Grouping key columns are never duplicated inside the nested objects.

Details

Column resolution: both cols2l and by accept either integer column positions or character column names. Out-of-bounds indices and unknown names are caught early with informative error messages.

Overlap guard: if any column appears in both cols2l and by, the function stops with an error before attempting to melt, preventing silent structural corruption.

Factor-free melting: melt() is called with variable.factor = FALSE so the name column is always character, avoiding unexpected factor-level ordering in downstream grouping operations.

Memory efficiency:

  • setDT() converts data.frame inputs by reference — no full copy.

  • .SDcols restricts .SD to non-key columns, eliminating redundant storage of grouping keys inside each nested object.

  • For nest_type = "df", setattr(copy(.SD), "class", "data.frame") modifies the class attribute on a shallow copy rather than performing a deep column-by-column duplication as as.data.frame() would.

Note

  • Passing an empty table (0 rows) triggers a warning() and returns an empty nested data.table immediately.

  • cols2l and by must not overlap; overlapping columns will raise an error.

  • nest_type values other than "dt" or "df" raise an error.

See also

tidytable::nest_by() for a tidyverse-style equivalent.

Examples

# Example: Wide to long format nesting demonstrations

# Example 1: Basic nesting by group
w2l_nest(
  data = iris,                    # Input dataset
  by = "Species"                  # Group by Species column
)
#>       Species               data
#>        <fctr>             <list>
#> 1:     setosa <data.table[50x4]>
#> 2: versicolor <data.table[50x4]>
#> 3:  virginica <data.table[50x4]>

# Example 2: Nest specific columns with numeric indices
w2l_nest(
  data = iris,                    # Input dataset
  cols2l = 1:4,                   # Select first 4 columns to nest
  by = "Species"                  # Group by Species column
)
#>             name    Species               data
#>           <char>     <fctr>             <list>
#>  1: Sepal.Length     setosa <data.table[50x1]>
#>  2: Sepal.Length versicolor <data.table[50x1]>
#>  3: Sepal.Length  virginica <data.table[50x1]>
#>  4:  Sepal.Width     setosa <data.table[50x1]>
#>  5:  Sepal.Width versicolor <data.table[50x1]>
#>  6:  Sepal.Width  virginica <data.table[50x1]>
#>  7: Petal.Length     setosa <data.table[50x1]>
#>  8: Petal.Length versicolor <data.table[50x1]>
#>  9: Petal.Length  virginica <data.table[50x1]>
#> 10:  Petal.Width     setosa <data.table[50x1]>
#> 11:  Petal.Width versicolor <data.table[50x1]>
#> 12:  Petal.Width  virginica <data.table[50x1]>

# Example 3: Nest specific columns with column names
w2l_nest(
  data = iris,                    # Input dataset
  cols2l = c("Sepal.Length",      # Select columns by name
             "Sepal.Width", 
             "Petal.Length"),
  by = 5                          # Group by column index 5 (Species)
)
#>            name    Species               data
#>          <char>     <fctr>             <list>
#> 1: Sepal.Length     setosa <data.table[50x2]>
#> 2: Sepal.Length versicolor <data.table[50x2]>
#> 3: Sepal.Length  virginica <data.table[50x2]>
#> 4:  Sepal.Width     setosa <data.table[50x2]>
#> 5:  Sepal.Width versicolor <data.table[50x2]>
#> 6:  Sepal.Width  virginica <data.table[50x2]>
#> 7: Petal.Length     setosa <data.table[50x2]>
#> 8: Petal.Length versicolor <data.table[50x2]>
#> 9: Petal.Length  virginica <data.table[50x2]>
# Returns similar structure to Example 2