Add artificial errors to body weight records that passed the checks, then
run clean_weight() with each outlier method and measure how many
artificial errors are found, how many good records are removed, and how
far ADG moves from the ADG of the clean records.
validate_outliers(
visits,
qc = qc_config(),
methods = list(`track+rlm+loess` = c("track", "rlm", "loess"), track = "track", rlm =
"rlm", loess = "loess", `rlm+loess` = c("rlm", "loess"), stl = "stl", sd = "sd"),
rate = 0.05,
size = c(3, 15),
direction = c("both", "down", "up"),
reps = 3,
max_pigs = NULL,
seed = 1,
reference = c("consensus", "qc"),
verbose = 1,
cores = 1
)Result of prep_visits().
QC settings from qc_config(): every setting except
weight_outlier and weight_track.
Named list of outlier method sets to compare. Each element
holds values of weight_outlier, plus "track" to track the weight
back from the end of the test first (weight_track = TRUE); sets
without "track" run without tracking. "track" alone is the tracking
without any other method.
Share of clean records that receive an artificial error. Default 0.05.
Range of the error size in kg. Default c(3, 15).
"both" (default): errors are added or subtracted at
random; "down": weights too low only (pig partly on the scale);
"up": too high only (two pigs on the scale).
Number of repetitions. Default 3.
Use a random sample of at most this many pigs, to save
time on large data (a few hundred pigs are enough). Default NULL: all.
Random seed, for reproducible results.
Which records count as clean, i.e. receive the
artificial errors and give the reference ADG: "consensus" (default)
the records kept by qc and by every method in methods, so that
no method is favoured; "qc" the records kept by qc (before 1.5.0),
which favours the method of qc.
0 (silent), 1 (summary and progress, default).
Number of CPU cores for clean_weight() and calc_adg().
A list of class ppt_validation with
summary: one row per method: sensitivity (share of artificial errors
found), false_positive_rate (share of clean records removed),
precision, adg_bias and adg_rmse (g/d, against the ADG of the
clean records), adg_spearman (rank correlation) and n_pigs;
per_rep: the same for each repetition;
settings.
The clean records are found by the methods themselves, so they are not
free of error. With reference = "consensus" a record is clean only when
no method flags it: every method starts from the same records, and a
method is not judged "wrong" for flagging a record that qc happened to
keep. The reference ADG is the ADG of the clean records, without any
outlier method. The row none shows the result when only weight_range
is applied (errors that push a weight outside it are still found).
Errors placed at random times shift single records but hardly move a growth curve fitted to hundreds of records, so ADG differences between methods are small; they mostly reflect which good records each method removes.
visits <- prep_visits(nedap, verbose = 0)
w <- clean_weight(visits, verbose = 0)
# 1. weight outlier methods: 5% artificial errors of 3-15 kg
vo <- validate_outliers(visits, reps = 1)
#>
#> ── Validating weight outlier methods ───────────────────────────────────────────
#> ℹ 7 methods · 5% artificial errors of 3-15 kg (both) · 1 repetition · reference: consensus
#> ⠙ [1/3] Clean records ■■■■■■■■■■■■■■■ 0/8 | ETA ?
#> ⠹ [1/3] Clean records ■■■■■■■■■■■■■■■ 1/8 | ETA 6s
#> ⠸ [1/3] Clean records ■■■■■■■■■■■■■■■ 2/8 | ETA 4s
#> ⠼ [1/3] Clean records ■■■■■■■■■■■■■■■ 3/8 | ETA 3s
#> ⠴ [1/3] Clean records ■■■■■■■■■■■■■■■ 4/8 | ETA 2s
#> ⠦ [1/3] Clean records ■■■■■■■■■■■■■■■ 5/8 | ETA 2s
#> ⠧ [1/3] Clean records ■■■■■■■■■■■■■■■ 6/8 | ETA 1s
#> ⠇ [1/3] Clean records ■■■■■■■■■■■■■■■ 7/8 | ETA 1s
#> ⠇ [1/3] Clean records ■■■■■■■■■■■■■■■ 8/8 | ETA 0s
#> ✔ [1/3] Clean records (8 cleanings) 4.4s
#> → [2/3] Reference ADG of the clean records ...
#> ✔ [2/3] Reference ADG of the clean records 0.7s
#> [3/3] Spoiled data, each method ■■■■■■■■■■■■■■■ 2/8 | ETA 6s
#> [3/3] Spoiled data, each method ■■■■■■■■■■■■■■■ 6/8 | ETA 2s
#> [3/3] Spoiled data, each method ■■■■■■■■■■■■■■■ 8/8 | ETA 0s
#> ✔ [3/3] Spoiled data, each method (8 runs) 6.5s
#> ✔ Done in 11.6s · details in $summary
#> 25,579 clean records of 29 pigs, 5% spoiled
#> errors found good removed ADG bias g/d ADG RMSE g/d
#> none 1.1% 0.00% -0.1 1.0
#> track+rlm+loess 94.3% 0.54% 0.6 2.1
#> track 67.9% 0.04% 0.1 1.0
#> rlm 90.0% 0.20% 0.3 1.5
#> loess 91.0% 0.00% 0.0* 0.7
#> rlm+loess 92.7% 0.20% 0.3 1.5
#> stl 95.3%* 0.29% 0.1 0.7*
#> sd 59.8% 0.00%* -0.2 1.1
vo$summary
#> method sensitivity false_positive_rate precision adg_bias
#> <char> <num> <num> <num> <num>
#> 1: none 0.01147666 0.000000e+00 1.0000000 -0.11659561
#> 2: track+rlm+loess 0.94338179 5.355966e-03 0.9046222 0.60896392
#> 3: track 0.67865340 3.707976e-04 0.9899554 0.08810530
#> 4: rlm 0.89977047 1.977587e-03 0.9607843 0.34852591
#> 5: loess 0.91048202 4.119974e-05 0.9991604 0.01913494
#> 6: rlm+loess 0.92654935 1.977587e-03 0.9618745 0.32561561
#> 7: stl 0.95256312 2.883982e-03 0.9467681 0.14346024
#> 8: sd 0.59831676 0.000000e+00 1.0000000 -0.24175573
#> adg_rmse adg_spearman n_pigs
#> <num> <num> <num>
#> 1: 1.0261373 0.9995074 29
#> 2: 2.1436768 0.9990148 29
#> 3: 0.9870266 0.9995074 29
#> 4: 1.5429075 0.9990148 29
#> 5: 0.7442768 0.9995074 29
#> 6: 1.5014817 0.9990148 29
#> 7: 0.7311463 0.9990148 29
#> 8: 1.1217394 0.9990148 29
# 2. growth curve models
va <- validate_adg(w, models = c("poly1", "poly2", "poly3"))
#>
#> ── Validating growth curves ────────────────────────────────────────────────────
#> ℹ 29 pigs · 3 models · 5-fold CV in 7-day blocks · truncation 10, 20 days
#> ⠙ [1/3] Re-cleaning CV folds ■■■■■■■■■■■■■■■ 0/5 | ETA ?
#> ⠹ [1/3] Re-cleaning CV folds ■■■■■■■■■■■■■■■ 1/5 | ETA 3s
#> ⠸ [1/3] Re-cleaning CV folds ■■■■■■■■■■■■■■■ 2/5 | ETA 2s
#> ⠼ [1/3] Re-cleaning CV folds ■■■■■■■■■■■■■■■ 3/5 | ETA 1s
#> ⠴ [1/3] Re-cleaning CV folds ■■■■■■■■■■■■■■■ 4/5 | ETA 1s
#> ⠴ [1/3] Re-cleaning CV folds ■■■■■■■■■■■■■■■ 5/5 | ETA 0s
#> ✔ [1/3] Re-cleaning CV folds (5 folds) 3.1s
#> ⠙ [2/3] Re-cleaning truncations ■■■■■■■■■■■■■■■ 0/4 | ETA ?
#> ⠹ [2/3] Re-cleaning truncations ■■■■■■■■■■■■■■■ 1/4 | ETA 2s
#> ⠸ [2/3] Re-cleaning truncations ■■■■■■■■■■■■■■■ 2/4 | ETA 1s
#> ⠼ [2/3] Re-cleaning truncations ■■■■■■■■■■■■■■■ 3/4 | ETA 1s
#> ⠼ [2/3] Re-cleaning truncations ■■■■■■■■■■■■■■■ 4/4 | ETA 0s
#> ✔ [2/3] Re-cleaning truncations (4 truncations) 2.5s
#> [3/3] Fitting the curves ■■■■■■■■■■■■■■■ 6/29 | ETA 4s
#> [3/3] Fitting the curves ■■■■■■■■■■■■■■■ 23/29 | ETA 1s
#> [3/3] Fitting the curves ■■■■■■■■■■■■■■■ 29/29 | ETA 0s
#> ✔ [3/3] Fitting the curves (29 pigs x 3 models) 5.4s
#> ✔ Done in 11.1s · details in $summary
#> Cross-validation error (median over pigs, kg) and ADG error after truncation
#> (RMSE, g/d)
#> CV kg CV > 10 kg late10 early10 late20 early20
#> poly1+start 1.76 0.0% 23.6* 21.5* 29.0 51.9
#> poly2+start 1.75 0.0% 31.3 23.0 28.2* 49.9*
#> poly3+start 1.61* 0.0% 37.2 23.9 36.2 60.1
va$summary
#> Key: <model>
#> model cv_rmse cv_mae cv_share_over bias_late10 rmse_late10
#> <char> <num> <num> <num> <num> <num>
#> 1: poly1+start 1.757314 1.414229 0 11.42217 23.62835
#> 2: poly2+start 1.752470 1.400581 0 24.01446 31.34774
#> 3: poly3+start 1.608416 1.264054 0 26.10989 37.18375
#> spearman_late10 n_late10 bias_early10 rmse_early10 spearman_early10
#> <num> <int> <num> <num> <num>
#> 1: 0.9600985 29 4.304331 21.47237 0.9965517
#> 2: 0.9694581 29 7.430822 22.99317 0.9960591
#> 3: 0.9586207 29 4.828772 23.93255 0.9960591
#> n_early10 bias_late20 rmse_late20 spearman_late20 n_late20 bias_early20
#> <int> <num> <num> <num> <int> <num>
#> 1: 29 -5.101753 29.02241 0.9389163 29 0.2360863
#> 2: 29 9.869637 28.23587 0.9467980 29 14.0993443
#> 3: 29 16.996585 36.22437 0.9147783 29 -3.1912060
#> rmse_early20 spearman_early20 n_early20
#> <num> <num> <int>
#> 1: 51.85206 0.9714286 29
#> 2: 49.89048 0.9758621 29
#> 3: 60.12357 0.9325123 29
# 3. weight fill methods
vw <- validate_weight_fill(w, methods = c("linear", "poly2"),
scenarios = c("blocks3", "late20"), reps = 1)
#>
#> ── Validating weight fill methods ──────────────────────────────────────────────
#> ℹ 29 pigs · 2 methods · 2 scenarios · 1 repetition
#> ✔ [1/4] Reference ADG and hidden days (29 pigs) 0.3s
#> ⠙ [2/4] Re-cleaning masks ■■■■■■■■■■■■■■■ 0/2 | ETA ?
#> ⠹ [2/4] Re-cleaning masks ■■■■■■■■■■■■■■■ 1/2 | ETA 1s
#> ⠹ [2/4] Re-cleaning masks ■■■■■■■■■■■■■■■ 2/2 | ETA 0s
#> ✔ [2/4] Re-cleaning masks (2 masks) 1.1s
#> ✔ [3/4] Filling the hidden days (29 pigs x 2 masks x 2 methods) 1.0s
#> ✔ Done in 2.5s · details in $summary
#> Daily weight error on the hidden days (RMSE, kg)
#> blocks3 late20
#> linear+start 0.93* 2.43*
#> poly2+start 1.42 2.70
#> ADG error (RMSE, g/d)
#> blocks3 late20
#> linear+start 5.1* 50.4*
#> poly2+start 9.9 51.7
vw$summary
#> scenario method daily_rmse daily_mae daily_bias adg_bias adg_rmse
#> <char> <char> <num> <num> <num> <num> <num>
#> 1: blocks3 linear+start 0.9280818 0.6722406 0.01704015 -0.7973775 5.118390
#> 2: blocks3 poly2+start 1.4165707 1.1360263 -0.03049167 -5.8986291 9.871986
#> 3: late20 linear+start 2.4261210 1.6707774 0.35260669 -6.7740568 50.394127
#> 4: late20 poly2+start 2.7036040 2.1173202 0.75794465 -10.8701574 51.672192
#> adg_spearman n_pigs
#> <num> <int>
#> 1: 0.9970443 29
#> 2: 0.9931034 29
#> 3: 0.9216749 29
#> 4: 0.9128079 29
# 4. feed intake imputation
adfi <- calc_adfi(visits, calc_adg(w, verbose = 0), verbose = 0)
vf <- validate_adfi(adfi, w, scenarios = c("random30", "late20"), reps = 2)
#>
#> ── Validating feed intake imputation ───────────────────────────────────────────
#> ℹ 29 pigs with coverage >= 0.8 · 3 methods · 2 scenarios · 2 repetitions · body weight without the hidden days
#> → [1/3] Leave-one-out population curves ...
#> ✔ [1/3] Leave-one-out population curves 0.0s
#> ⠙ [2/3] Body weight per mask ■■■■■■■■■■■■■■■ 0/4 | ETA ?
#> ⠹ [2/3] Body weight per mask ■■■■■■■■■■■■■■■ 1/4 | ETA 2s
#> ⠸ [2/3] Body weight per mask ■■■■■■■■■■■■■■■ 2/4 | ETA 2s
#> ⠼ [2/3] Body weight per mask ■■■■■■■■■■■■■■■ 3/4 | ETA 1s
#> ⠼ [2/3] Body weight per mask ■■■■■■■■■■■■■■■ 4/4 | ETA 0s
#> ✔ [2/3] Body weight per mask (4 masks) 3.3s
#> ✔ [3/3] Imputing the hidden days (29 pigs x 4 masks x 3 methods) 1.9s
#> ✔ Done in 5.3s · details in $summary
#> ADFI error (mean absolute error, % of the recorded ADFI)
#> random30 late20
#> loess_pop 1.05 1.34*
#> loess_lm 1.05* 4.49
#> linear 1.06 4.49
#> Rank correlation of ADFI with the recorded ADFI
#> random30 late20
#> loess_pop 0.990 0.980*
#> loess_lm 0.990* 0.954
#> linear 0.983 0.954
vf$summary
#> scenario method mean_abs_error_pct p95_abs_error_pct bias_pct
#> <char> <char> <num> <num> <num>
#> 1: random30 loess_pop 1.054756 2.747063 0.08312500
#> 2: random30 loess_lm 1.049116 2.652649 0.09695811
#> 3: random30 linear 1.059801 3.446624 0.12860912
#> 4: late20 loess_pop 1.336754 3.504007 -0.29738898
#> 5: late20 loess_lm 4.485321 7.063380 4.26201154
#> 6: late20 linear 4.485321 7.063380 4.26201154
#> spearman daily_rmse n_pigs
#> <num> <num> <int>
#> 1: 0.9896552 454.9470 29
#> 2: 0.9899015 454.8497 29
#> 3: 0.9834975 491.0849 29
#> 4: 0.9802956 321.3658 29
#> 5: 0.9536946 678.6888 29
#> 6: 0.9536946 678.6888 29