Export Nested Data with Advanced Grouping and Flexible Handling

Source: R/export_nest.R

The export_list function exports nested data from a data.frame or data.table with sophisticated grouping capabilities, supporting multiple nested column types and flexible file export options.

export_nest(
  nest_dt,
  group_cols = NULL,
  nest_col = NULL,
  export_path = tempdir(),
  file_type = "txt"
)

Arguments

nest_dt

A data.frame or data.table containing nested columns of data.frames, data.tables, or lists to be exported.

group_cols

Optional character vector specifying grouping columns. If NULL, uses all non-nested columns as grouping variables.

nest_col

Optional character string indicating the nested column to export. If NULL, automatically selects the first nested column.

export_path

Base directory path for file export. Defaults to a temporary directory created by tempdir().

file_type

File export format, either "txt" (tab-separated) or "csv". Defaults to "txt".

Value

An integer representing the total number of files exported successfully.

Details

Comprehensive Nested Data Export Features:

  • Automatic detection and handling of different nested column types

  • Flexible grouping strategies with intelligent column selection

  • Hierarchical directory structure generation based on grouping columns

  • Support for mixed nested column types (data.frame, data.table, list)

  • Multi-threaded file writing for enhanced performance

  • Informative messaging and warning system

Nested Column Detection Hierarchy:

  1. Prioritizes data.frame/data.table nested columns

  2. Falls back to regular list columns if no data.frame columns exist

Grouping Column Selection Strategy:

  1. When group_cols is NULL, uses all non-nested columns

  2. Provides warnings about unused non-nested columns

  3. Validates provided group columns

File Export Characteristics:

  • Supports "txt" (tab-separated) and "csv" formats

  • Uses multi-threading via parallel::detectCores()

  • Creates nested directory structure based on grouping variables

Note

Key Capabilities:

  • Handles complex nested data structures

  • Performs type conversion for nested content

  • Utilizes multi-threaded file export for optimal performance

  • Provides comprehensive column selection feedback

Examples

# Example 1: Basic nested data export workflow
# Step 1: Create nested data structure
dt_nest <- w2l_nest(
  data = iris,              # Input iris dataset
  cols2l = 1:2,             # Columns to be nested
  by = "Species"            # Grouping variable
)

# Step 2: Export nested data to files
export_nest(
  nest_dt = dt_nest,        # Input nested data.table
  nest_col = "data",        # Column containing nested data
  group_cols = c("name", "Species")  # Columns to create directory structure
)
#> [1] 6
# Returns the number of files created
# Creates directory structure: tempdir()/name/Species/data.txt

# Check exported files
list.files(
  path = tempdir(),         # Default export directory
  pattern = "txt",          # File type pattern to search
  recursive = TRUE          # Search in subdirectories
)
#> [1] "Sepal.Length/setosa/data.txt"     "Sepal.Length/versicolor/data.txt"
#> [3] "Sepal.Length/virginica/data.txt"  "Sepal.Width/setosa/data.txt"     
#> [5] "Sepal.Width/versicolor/data.txt"  "Sepal.Width/virginica/data.txt"  
# Returns list of created files and their paths

# Clean up exported files
files <- list.files(
  path = tempdir(),         # Default export directory
  pattern = "txt",          # File type pattern to search
  recursive = TRUE,         # Search in subdirectories
  full.names = TRUE         # Return full file paths
)
file.remove(files)          # Remove all exported files
#> [1] TRUE TRUE TRUE TRUE TRUE TRUE