R/validate_truncation.R
validate_truncation.RdRemove all visits of part of the pigs over some days of the stage (late
entry, early exit or a gap), rerun the pipeline from clean_weight() to
calc_fcr() with each QC setting, and compare ADG, ADFI and FCR with the
result of all data. Nothing of the hidden days is used, so the result
measures what really happens when a pig misses these days.
Result of prep_visits().
A QC object from qc_config(), or a named list of them
to compare, e.g. list(lm = qc_config(impute_method = "loess_lm"), pop = qc_config()).
Days removed, counted on the stage of the first setting (from its full-data run):
"late<d>": the first d days of the stage and everything before
(late entry), e.g. "late20";
"early<d>": the last d days of the stage and everything after
(early exit);
"gap<d>": d consecutive days in the middle of the stage.
Share of the pigs truncated in one run (default 0.5). The
pigs are split at random into round(1 / share) groups, each group is
truncated in turn, so every pig is truncated once per repetition.
Number of repetitions, each with a new random split. Default 5.
Share of the best pigs for the selection agreement top_agree
(default 0.2: the best 20%). Best = highest ADG, lowest FCR, highest
ADFI.
Pigs to truncate. Default NULL: pigs whose stage is fully
covered by records in the first setting (adg_type == "observed") and
that have an FCR.
Random seed.
0 (silent) or 1 (progress, default).
Number of CPU cores: runs of the pipeline in parallel.
A list of class ppt_validation with
summary: one row per setting, scenario and trait (adg, adfi,
fcr): the mean over repetitions of bias (truncated - full),
rmse, mae_pct (mean absolute error, % of the full value),
pearson and spearman correlation with the full value, slope of
the regression of the truncated on the full value (below 1: the
differences between pigs shrink), top_agree (share of the best top
pigs on full data that are also among the best after truncation),
n (pigs with both values) and n_lost (pigs that lost their value);
columns ending in _sd give the SD over repetitions;
per_rep: the same measures for every repetition;
per_pig: full and truncated value of every pig, trait, setting,
scenario and repetition;
settings.
The reference is the result of all data with the same setting, not the
unknown true value: the measures tell how stable a setting is when data
are missing, which is what matters to compare settings. With few pigs
(e.g. 20), differences of a few hundredths in a correlation are noise;
look at the _sd columns and use more repetitions.
Pigs whose test becomes too short after truncation (e.g. below
min_test_days) are removed by clean_weight() and counted in
n_lost.
validate_adfi(), validate_adg() for faster checks of single
steps.
visits <- prep_visits(nedap, verbose = 0)
# 2 settings x 1 scenario x 1 repetition; use more repetitions on real data
res <- validate_truncation(
visits,
settings = list(lm = qc_config(impute_method = "loess_lm"), pop = qc_config()),
scenarios = "late20", reps = 1
)
#>
#> ── Validating the pipeline by truncation ───────────────────────────────────────
#> ⠙ [1/2] Full-data run of each setting ■■■■■■■■■■■■■■■ 0/2 | ETA ?
#> ⠹ [1/2] Full-data run of each setting ■■■■■■■■■■■■■■■ 1/2 | ETA 1s
#> ⠹ [1/2] Full-data run of each setting ■■■■■■■■■■■■■■■ 2/2 | ETA 0s
#> ✔ [1/2] Full-data run of each setting (2 settings) 2.4s
#> ℹ 21 pigs with a fully covered stage · 2 settings · 1 scenario · 1 repetition · 4 runs of the pipeline
#> [2/2] Truncated runs ■■■■■■■■■■■■■■■ 3/4 | ETA 1s
#> [2/2] Truncated runs ■■■■■■■■■■■■■■■ 4/4 | ETA 0s
#> ✔ [2/2] Truncated runs (4 runs) 4.6s
#> ✔ Done in 7.0s · details in $summary
#> FCR: rank correlation with the full-data FCR
#> late20
#> lm 0.916
#> pop 0.922*
#> FCR: mean absolute error (%)
#> late20
#> lm 4.59
#> pop 1.83*
res$summary[, .(setting, trait, mae_pct, spearman, slope, top_agree)]
#> setting trait mae_pct spearman slope top_agree
#> <char> <char> <num> <num> <num> <num>
#> 1: lm adg 1.847664 0.9129870 0.9403050 0.75
#> 2: pop adg 1.847664 0.9129870 0.9403050 0.75
#> 3: lm adfi 5.924236 0.9701299 1.1338426 0.75
#> 4: pop adfi 1.794426 0.9688312 0.9211916 0.75
#> 5: lm fcr 4.590688 0.9155844 1.3116640 0.50
#> 6: pop fcr 1.833390 0.9220779 1.0095083 0.75