Convert raw NEDAP or FIRE station data into a standard visit table with the same columns and units for both station types.
prep_visits(data, station = NULL, format = NULL, qc = qc_config(), verbose = 1)Station data: the standard table from read_station(), or
a raw export already in R, e.g. the example datasets nedap and
fire or a table read from a database. Raw data is mapped to the
standard fields with format.
"nedap", "fire" or NULL (default): taken from the
"station" attribute set by read_station(), or detected from the
column names.
Station format for raw data whose columns differ from
the usual export, as in read_station(). NULL (default) uses the
built-in format. Ignored for the output of read_station(), which is
already standard.
QC settings from qc_config(). Used here: duration_max (to
tell a FIRE midnight crossing from a recording error) and the station
transfer parameters misread_max_visits, max_jump_kg.
0 (silent), 1 (summary, default) or 2 (details).
A data.table, one row per visit, ordered by location and
entry_time:
responder: pig ID (character, NA for unidentified visits);
responder_raw: transponder number as recorded;
breed: kept when the input has a breed column; otherwise add it
with add_breed();
identified: FALSE for visits without a transponder read;
location: station (feeder) ID, character;
date, seq_days (day number from the first day in the data, 1 =
first day), seq_in_day (visit number of the pig that day),
seq_in_location (visit number on the feeder that day);
entry_time, exit_time: POSIXct, UTC; duration: seconds;
feed_intake: g; weight: body weight, kg (NA if not recorded);
trough_in, trough_out: trough weight at entrance and exit, g
(FIRE only, NA for NEDAP); topup: feed dispensed, g (FIRE only);
source_file, if present in the input.
Attributes:
"station": station type;
"stations": one row per pig with its station history (see below);
"removed": log of visits turned into unidentified visits as
transponder misreads;
"prep_summary": number of rows dropped at each step.
A pig can be recorded at several stations because it was really moved: it escaped to another pen and stayed there, was found and put back, or was isolated for treatment and returned to the wrong pen. These visits are real and belong to the same pig. A transponder misread, in contrast, gives a few scattered visits at another station while the pig keeps visiting its own station.
So the identified visits of each pig are put in time order and cut into runs of consecutive visits at the same station:
misread: a run of at most misread_max_visits visits. Shortest
runs are dropped first, so that the runs around them join. The visits
stay in the data as unidentified visits (identified = FALSE,
responder = NA, responder_raw kept): they did happen at that
feeder and are needed for the feeder error checks and the
unidentified feed of the feeder.
transfer: every remaining run is a real stay at a station. A pig
can move any number of times, also back to a station it used before
(location_path such as "101>102>101"). All visits are kept with
their real location.
at each transfer the body weight before and after (median of the 20
valid weights next to it) is compared, allowing for growth over the
gap. A jump above max_jump_kg gives a warning, and
clean_weight() flags the pig as low confidence.
A gap without visits (e.g. isolation for treatment) never splits a pig
into two animals: the days are missing, and clean_weight() flags the
pig as low confidence when the longest gap exceeds max_gap_days.
The "stations" attribute has one row per pig: location_path,
main_location (station with most visits), n_locations,
n_transfers, transfer, misreads (visits turned unidentified),
longest_gap_days and weight_jump_kg (largest jump beyond the
expected growth, NA without transfer).
All stations are assumed to be in separate pens. Stations in the same pen, which a pig visits in turn, look like many transfers; a warning is given when a pig moves more than 3 times.
read_station() for the previous step.
visits <- prep_visits(fire)
#>
#> ── Preparing FIRE visits ───────────────────────────────────────────────────────
#> Visits read 9,794
#> └ unidentified (kept, identified = FALSE) 51 0.5%
#> └ crossing midnight (exit time corrected) 68 0.7%
#> └ body weight 0 (set to NA) 478 4.9%
#> ℹ No pig was seen at more than one station
#> ✔ 26 pigs at 2 stations, 9,794 visits over 66 days 0.1s
#> ℹ No breed column. To use breed in the feed intake model and breed-specific limits, add it with `add_breed()`.
head(visits)
#> responder responder_raw identified location date seq_days seq_in_day
#> <char> <char> <lgcl> <char> <Date> <int> <int>
#> 1: 35877 35877 TRUE 101 2023-12-19 1 1
#> 2: 35873 35873 TRUE 101 2023-12-19 1 1
#> 3: 35878 35878 TRUE 101 2023-12-19 1 1
#> 4: 35855 35855 TRUE 101 2023-12-19 1 1
#> 5: 35877 35877 TRUE 101 2023-12-19 1 2
#> 6: 35853 35853 TRUE 101 2023-12-19 1 1
#> seq_in_location entry_time exit_time duration feed_intake
#> <int> <POSc> <POSc> <num> <num>
#> 1: 1 2023-12-19 14:15:39 2023-12-19 14:18:02 143 46
#> 2: 2 2023-12-19 14:18:03 2023-12-19 14:23:05 302 248
#> 3: 3 2023-12-19 14:23:15 2023-12-19 14:28:45 330 201
#> 4: 4 2023-12-19 14:29:05 2023-12-19 14:34:29 324 121
#> 5: 5 2023-12-19 14:34:30 2023-12-19 14:34:37 7 0
#> 6: 6 2023-12-19 14:34:38 2023-12-19 14:36:26 108 0
#> weight trough_in trough_out topup
#> <num> <num> <num> <num>
#> 1: 67.6 678 632 286
#> 2: 60.8 632 384 0
#> 3: 70.8 670 469 286
#> 4: 51.2 755 634 286
#> 5: NA 634 634 0
#> 6: 88.6 634 634 0
# Unidentified visits are kept for the feeder error checks
table(visits$identified)
#>
#> FALSE TRUE
#> 51 9743
# Station history of each pig
head(attr(visits, "stations"))
#> responder location_path main_location n_locations n_transfers transfer
#> <char> <char> <char> <int> <int> <lgcl>
#> 1: 25822 101 101 1 0 FALSE
#> 2: 35853 101 101 1 0 FALSE
#> 3: 35855 101 101 1 0 FALSE
#> 4: 35862 101 101 1 0 FALSE
#> 5: 35865 101 101 1 0 FALSE
#> 6: 35869 101 101 1 0 FALSE
#> misreads longest_gap_days weight_jump_kg
#> <int> <num> <num>
#> 1: 0 0.7254051 NA
#> 2: 0 0.6270370 NA
#> 3: 0 0.6021412 NA
#> 4: 0 0.5934722 NA
#> 5: 0 0.5991898 NA
#> 6: 0 0.6719097 NA
# NEDAP data gets the same columns and units
nedap_visits <- prep_visits(nedap, verbose = 0)
summary(nedap_visits$weight)
#> Min. 1st Qu. Median Mean 3rd Qu. Max. NAs
#> 20.50 43.00 80.00 77.17 106.50 147.75 391