Export a List of Data Frames with Hierarchical Directory Management

Source: R/export_list.R

Exports every element of a named (or unnamed) list of data.frame / data.table objects to txt or csv files. Element names may contain forward-slashes (/) to encode arbitrary subdirectory depth, e.g. "group_a/subject_01/results" writes <path>/group_a/subject_01/results.txt. Unnamed elements are automatically labelled split_<i>.

export_list(
  data,
  path = tempdir(),
  file_type = "txt",
  na = "NA",
  quote = FALSE,
  ...
)

Arguments

data

A non-empty list whose elements are data.frame, data.table, or any object coercible via data.table::as.data.table().

path

Single character string - the root export directory. Created recursively if absent. Defaults to tempdir().

file_type

"txt" (tab-separated, default) or "csv" (comma-separated). Case-insensitive.

na

Single string written for missing values. Default "NA". Use e.g. "-9999" for DMU.

quote

Passed to fwrite. Default FALSE: with a non-empty na, fwrite's "auto" would quote every header and character field, which command-line breeding programs (HIBLUP, DMU, ...) cannot parse. Use "auto" if values may contain the separator.

...

Further arguments passed to fwrite (e.g. col.names = FALSE).

Value

An invisible named character vector of the file paths written, with length equal to the number of successfully exported elements. The total count is accessible via length() on the return value.

Details

Performance design:

  • All element names are resolved and path components split in a single vectorised pass before the write loop, so no string work occurs inside the hot path.

  • Unique subdirectories are collected and created in one batch (k dir.create() syscalls, where k \(\le\) n).

  • The field separator is resolved once at function entry.

  • as.data.table() on an existing data.table is a reference-pass (no copy).

Name handling: each /-separated component of an element name is sanitised separately (invalid characters become _; .. cannot escape path). If two elements map to the same file (compared case-insensitively), the function stops instead of overwriting.

Error handling: Individual element failures emit a warning and are skipped; the remaining elements continue to be processed.

See also

fwrite

Examples

# Example: Export split data to files
out_dir <- file.path(tempdir(), "mintyr_export_list")

# Step 1: Create split data structure
dt_split <- w2l_split(
  data = iris,              # Input iris dataset
  cols = 1:2,               # Columns to be split
  by = "Species"            # Grouping variable
)

# Step 2: Export split data to files
files <- export_list(
  data = dt_split,          # Input list of data.tables
  path = out_dir
)
#> [ export_list ] Export complete. 6 / 6 file(s) written to: /tmp/RtmpT1dTzO/mintyr_export_list
# Returns (invisibly) a named vector of the written file paths
files
#>                                              Sepal.Length_setosa 
#>     "/tmp/RtmpT1dTzO/mintyr_export_list/Sepal.Length_setosa.txt" 
#>                                          Sepal.Length_versicolor 
#> "/tmp/RtmpT1dTzO/mintyr_export_list/Sepal.Length_versicolor.txt" 
#>                                           Sepal.Length_virginica 
#>  "/tmp/RtmpT1dTzO/mintyr_export_list/Sepal.Length_virginica.txt" 
#>                                               Sepal.Width_setosa 
#>      "/tmp/RtmpT1dTzO/mintyr_export_list/Sepal.Width_setosa.txt" 
#>                                           Sepal.Width_versicolor 
#>  "/tmp/RtmpT1dTzO/mintyr_export_list/Sepal.Width_versicolor.txt" 
#>                                            Sepal.Width_virginica 
#>   "/tmp/RtmpT1dTzO/mintyr_export_list/Sepal.Width_virginica.txt" 

# Clean up
unlink(out_dir, recursive = TRUE)