Package 'epinowcast'

Description

This function calculates and adds the maximum observed delay for each group and reference date in the provided dataset. It first checks the validity of the observation indicator and then computes the maximum delay. If an observation indicator is provided, it further adjusts the maximum observed delay for unobserved data to be negative 1 (indicating no maximum observed).

Usage

add_max_observed_delay(new_confirm, observation_indicator = NULL)

Arguments

Value

A data.table with the original columns of new_confirm and an additional "max_obs_delay" column representing the maximum observed delay for each group and reference date. If an observation indicator is provided, unobserved data will have a "max_obs_delay" value of -1.

See Also

Helper functions for model modules add_pmfs(), convolution_matrix(), enw_reference_by_report(), enw_reps_with_complete_refs(), extract_obs_metadata(), extract_sparse_matrix(), latest_obs_as_matrix(), simulate_double_censored_pmf()

Description

This function allows the addition of probability mass functions (PMFs) to produce a new PMF. This is useful for example in the context of reporting delays where the PMF of the sum of two Poisson distributions is the convolution of the PMFs.

Usage

add_pmfs(pmfs)

Arguments

Value

A vector describing the probability mass function of the sum of the

See Also

Helper functions for model modules add_max_observed_delay(), convolution_matrix(), enw_reference_by_report(), enw_reps_with_complete_refs(), extract_obs_metadata(), extract_sparse_matrix(), latest_obs_as_matrix(), simulate_double_censored_pmf()

Examples

# Sample and analytical PMFs for two Poisson distributions
x <- rpois(10000, 5)
xpmf <- dpois(0:20, 5)
y <- rpois(10000, 7)
ypmf <- dpois(0:20, 7)
# Add sampled Poisson distributions up to get combined distribution
z <- x + y
# Analytical convolution of PMFs
conv_pmf <- add_pmfs(list(xpmf, ypmf))
conv_cdf <- cumsum(conv_pmf)
# Empirical convolution of PMFs
cdf <- ecdf(z)(0:42)
# Compare sampled and analytical CDFs
plot(conv_cdf)
lines(cdf, col = "black")

Description

This function takes a data.table and applies a rolling sum over a given timestep, aggregating by specified columns. It's particularly useful for aggregating observations over certain periods.

Usage

aggregate_rolling_sum(dt, internal_timestep, by = NULL)

Arguments

Value

A modified data.table with aggregated observations.

See Also

Utility functions coerce_date(), coerce_dt(), date_to_numeric_modulus(), get_internal_timestep(), is.Date(), stan_fns_as_string()

Description

Converts formulas to strings

Usage

as_string_formula(formula)

Arguments

Value

A character string of the supplied formula

See Also

Functions used to help convert formulas into model designs construct_re(), construct_rw(), enw_formula(), enw_manual_formula(), parse_formula(), remove_rw_terms(), re(), rw_terms(), rw(), split_formula_to_terms()

Examples

epinowcast:::as_string_formula(~ 1 + age_group)

Description

This function verifies if the difference in calendar dates in the provided observations corresponds to the provided timestep of "month".

Usage

check_calendar_timestep(dates, date_var, exact = TRUE)

Arguments

Value

This function is used for its side effect of stopping if the check fails. If the check passes, the function returns invisibly.

See Also

Functions used for checking inputs check_group_date_unique(), check_group(), check_max_delay(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Description

Check observations for reserved grouping variables

Usage

check_group(obs)

Arguments

Value

The obs object, which will be modifiable in place.

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_max_delay(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Description

This function checks that the input data is stratified by reference_date, report_date, and .group. It does this by counting the number of observations for each combination of these variables, and throwing a warning if any combination has more than one observation.

Usage

check_group_date_unique(obs)

Arguments

See Also

Functions used for checking inputs check_calendar_timestep(), check_group(), check_max_delay(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Description

Check if maximum delay specified by the user is long enough and raise potential warnings. This is achieved by computing the share of reference dates where the cumulative case count is below some aspired coverage.

Usage

check_max_delay(
  data,
  max_delay = data$max_delay,
  cum_coverage = 0.8,
  maxdelay_quantile_outlier = 0.97,
  warn = TRUE,
  warn_internal = FALSE
)

Arguments

Details

The coverage is with respect to the maximum observed case count for the corresponding reference date. As the maximum observed case count is likely smaller than the true overall case count for not yet fully observed reference dates (due to right truncation), only reference dates that are more than the maximum observed delay ago are included. Still, because we can only use the maximum observed delay, not the unknown true maximum delay, the computed coverage values should be interpreted with care, as they are only proxies for the true coverage.

Value

A data.table with the share of reference dates where the cumulative case count is below cum_coverage, stratified by group.

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Examples

pobs <- enw_example(type = "preprocessed_observations")
check_max_delay(pobs, max_delay = 20, cum_coverage = 0.8)

Description

Check a model module contains the required components

Usage

check_module(module)

Arguments

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_max_delay(), check_modules_compatible(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Description

Check that model modules have compatible specifications

Usage

check_modules_compatible(modules)

Arguments

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_max_delay(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Description

This function verifies if the difference in numeric dates in the provided observations corresponds to the provided timestep.

Usage

check_numeric_timestep(dates, date_var, timestep, exact = TRUE)

Arguments

Value

This function is used for its side effect of stopping if the check fails. If the check passes, the function returns invisibly.

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_max_delay(), check_modules_compatible(), check_module(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Description

This function verifies if the observation_indicator within the provided new_confirm observations is logical. The check is performed to ensure that the observation_indicator is of the correct type.

Usage

check_observation_indicator(new_confirm, observation_indicator = NULL)

Arguments

Value

This function is used for its side effect of checking the observation indicator in new_confirm. If the check passes, the function returns invisibly. Otherwise, it stops and returns an error message.

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_max_delay(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Description

Check required quantiles are present

Usage

check_quantiles(posterior, req_probs = c(0.5, 0.95, 0.2, 0.8))

Arguments

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_max_delay(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_timestep_by_date(), check_timestep_by_group(), check_timestep()

Description

This function verifies if the difference in dates in the provided observations corresponds to the provided timestep. If the exact argument is set to TRUE, the function checks if all differences exactly match the timestep; otherwise, it checks if the sum of the differences modulo the timestep equals zero. If the check fails, the function stops and returns an error message.

Usage

check_timestep(
  obs,
  date_var,
  timestep = "day",
  exact = TRUE,
  check_nrow = TRUE
)

Arguments

Value

This function is used for its side effect of stopping if the check fails. If the check passes, the function returns invisibly.

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_max_delay(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep_by_group()

Description

This function verifies if the difference in dates within each date in the provided observations corresponds to the provided timestep. This check is performed for both report_date and reference_date and for each group in obs.

Usage

check_timestep_by_date(obs, timestep = "day", exact = TRUE)

Arguments

Value

This function is used for its side effect of checking the timestep by date in obs. If the check passes for all dates, the function returns invisibly. Otherwise, it stops and returns an error message.

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_max_delay(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_group(), check_timestep()

Description

This function verifies if the difference in dates within each group in the provided observations corresponds to the provided timestep. This check is performed for the specified date_var and for each group in obs.

Usage

check_timestep_by_group(obs, date_var, timestep = "day", exact = TRUE)

Arguments

Value

This function is used for its side effect of checking the timestep by group in obs. If the check passes for all groups, the function returns invisibly. Otherwise, it stops and returns an error message.

See Also

Functions used for checking inputs check_calendar_timestep(), check_group_date_unique(), check_group(), check_max_delay(), check_modules_compatible(), check_module(), check_numeric_timestep(), check_observation_indicator(), check_quantiles(), check_timestep_by_date(), check_timestep()

Description

Provides consistent coercion of inputs to IDate with error handling

Usage

coerce_date(dates = NULL)

Arguments

Details

If any of the elements of dates cannot be coerced, this function will result in an error, indicating all indices which cannot be coerced to IDate.

Internal methods of epinowcast assume dates are represented as IDate.

Value

An IDate vector.

See Also

Utility functions aggregate_rolling_sum(), coerce_dt(), date_to_numeric_modulus(), get_internal_timestep(), is.Date(), stan_fns_as_string()

Examples

# works
coerce_date(c("2020-05-28", "2020-05-29"))
# does not, indicates index 2 is problem
tryCatch(
  coerce_date(c("2020-05-28", "2020-o5-29")),
  error = function(e) {
    print(e)
  }
)

Description

Provides consistent coercion of inputs to data.table with error handling, column checking, and optional selection.

Usage

coerce_dt(
  data,
  select = NULL,
  required_cols = select,
  forbidden_cols = NULL,
  group = FALSE,
  dates = FALSE,
  copy = TRUE,
  msg_required = "The following columns are required: ",
  msg_forbidden = "The following columns are forbidden: "
)

Arguments

Details

This function provides a single-point function for getting a "local" version of data provided by the user, in the internally used data.table format. It also enables selectively copying versus not, as well as checking for the presence and/or absence of various columns.

While it is intended to address garbage in from the user, it does not generally attempt to address garbage in from the developer - e.g. if asking for overlapping required and forbidden columns (though that will lead to an always-error condition).

Value

A data.table; the returned object will be a copy, unless copy = FALSE, in which case modifications are made in-place

See Also

Utility functions aggregate_rolling_sum(), coerce_date(), date_to_numeric_modulus(), get_internal_timestep(), is.Date(), stan_fns_as_string()

Description

Constructs random effect terms

Usage

construct_re(re, data)

Arguments

Value

A list containing the transformed data ("data"), fixed effects terms ("terms") and a data.frame specifying the random effect structure between these terms (effects). Note that if the specified random effect was not a factor it will have been converted into one.

See Also

Functions used to help convert formulas into model designs as_string_formula(), construct_rw(), enw_formula(), enw_manual_formula(), parse_formula(), remove_rw_terms(), re(), rw_terms(), rw(), split_formula_to_terms()

Examples

# Simple examples
form <- epinowcast:::parse_formula(~ 1 + (1 | day_of_week))
data <- enw_example("prepr")$metareference[[1]]
random_effect <- re(form$random[[1]])
epinowcast:::construct_re(random_effect, data)

# A more complex example
form <- epinowcast:::parse_formula(
  ~ 1 + disp + (1 + gear | cyl) + (0 + wt | am)
)
random_effect <- re(form$random[[1]])
epinowcast:::construct_re(random_effect, mtcars)

random_effect2 <- re(form$random[[2]])
epinowcast:::construct_re(random_effect2, mtcars)

Description

This function takes random walks as defined by rw(), produces the required additional variables (denoted using a "c" prefix and constructed using enw_add_cumulative_membership()), and then returns the extended data.frame along with the new fixed effects and the random effect structure.

Usage

construct_rw(rw, data)

Arguments

Value

A list containing the following:

• data: The input data.frame with the addition of the new variables required by the specified random walk. These are added using enw_add_cumulative_membership(). -terms: A character vector of new fixed effects terms to add to a model formula.

• effects: A data.frame describing the random effect structure of the new effects.

See Also

Functions used to help convert formulas into model designs as_string_formula(), construct_re(), enw_formula(), enw_manual_formula(), parse_formula(), remove_rw_terms(), re(), rw_terms(), rw(), split_formula_to_terms()

Examples

data <- enw_example("preproc")$metareference[[1]]

epinowcast:::construct_rw(rw(week), data)

epinowcast:::construct_rw(rw(week, day_of_week), data)

Description

This function allows the construction of convolution matrices which can be be combined with a vector of primary events to produce a vector of secondary events for example in the form of a renewal equation or to simulate reporting delays. Time-varying delays are supported as well as distribution padding (to allow for use in renewal equation like approaches).

Usage

convolution_matrix(dist, t, include_partial = FALSE)

Arguments

Value

A matrix with each column indicating a primary event and each row indicating a secondary event.

See Also

Helper functions for model modules add_max_observed_delay(), add_pmfs(), enw_reference_by_report(), enw_reps_with_complete_refs(), extract_obs_metadata(), extract_sparse_matrix(), latest_obs_as_matrix(), simulate_double_censored_pmf()

Examples

# Simple convolution matrix with a static distribution
convolution_matrix(c(1, 2, 3), 10)
# Include partially reported convolutions
convolution_matrix(c(1, 2, 3), 10, include_partial = TRUE)
# Use a list of distributions
convolution_matrix(rep(list(c(1, 2, 3)), 10), 10)
# Use a time-varying list of distributions
convolution_matrix(c(rep(list(c(1, 2, 3)), 10), list(c(4, 5, 6))), 11)

Description

This function processes a date column in a data.table, converting it to a numeric representation and then computing the modulus with the provided timestep.

Usage

date_to_numeric_modulus(dt, date_column, timestep)

Arguments

Value

A modified data.table with two new columns: one for the numeric representation of the date minus the minimum date and another for its modulus with the timestep.

See Also

Utility functions aggregate_rolling_sum(), coerce_date(), coerce_dt(), get_internal_timestep(), is.Date(), stan_fns_as_string()

Description

Calculate cumulative reported cases from incidence of new reports

Usage

enw_add_cumulative(obs, by = NULL, copy = TRUE)

Arguments

Value

The input data.frame with a new variable confirm.

See Also

Data converters enw_add_incidence(), enw_aggregate_cumulative(), enw_cumulative_to_incidence(), enw_incidence_to_cumulative(), enw_incidence_to_linelist(), enw_linelist_to_incidence()

Examples

# Default reconstruct incidence
dt <- germany_covid19_hosp[location == "DE"][age_group == "00+"]
dt <- enw_add_incidence(dt)
dt <- dt[, confirm := NULL]
enw_add_cumulative(dt)

# Make use of maximum reported to calculate empirical daily reporting
enw_add_cumulative(dt)

Description

This function adds a cumulative membership effect to a data frame. This is useful for specifying models such as random walks (using rw()) where these features can be used in the design matrix with the appropriate formula. Supports grouping via the optional .group column. Note that cumulative membership is indexed to start with zero (i.e. the first observation is assigned a cumulative membership of zero).

Usage

enw_add_cumulative_membership(metaobs, feature, copy = TRUE)

Arguments

Value

A data.frame with a new columns ⁠cfeature$⁠ that contain the cumulative membership effect for each value of feature. For example if the original feature was week (with numeric entries ⁠1, 2, 3⁠) then the new columns will be cweek1, cweek2, and cweek3.

See Also

Functions used to formulate models enw_add_pooling_effect(), enw_design(), enw_effects_metadata(), enw_one_hot_encode_feature()

Examples

metaobs <- data.frame(week = 1:2)
enw_add_cumulative_membership(metaobs, "week")

metaobs <- data.frame(week = 1:3, .group = c(1,1,2))
enw_add_cumulative_membership(metaobs, "week")

Description

This helper function takes a data.frame or data.table of observations and adds the delay (numeric, in days) between reference_date and report_date for each observation.

Usage

enw_add_delay(obs, timestep = "day", copy = TRUE)

Arguments

Value

A data.table of observations with a new column delay.

See Also

Preprocessing functions enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

obs <- data.frame(report_date = as.Date("2021-01-01") + -2:0)
obs$reference_date <- as.Date("2021-01-01")
enw_add_delay(obs)

Description

Calculate incidence of new reports from cumulative reports

Usage

enw_add_incidence(obs, set_negatives_to_zero = TRUE, by = NULL, copy = TRUE)

Arguments

Value

The input data.frame with a new variable new_confirm. If max_confirm is present in the data.frame, then the proportion reported on each day (prop_reported) will also be added.

See Also

Data converters enw_add_cumulative(), enw_aggregate_cumulative(), enw_cumulative_to_incidence(), enw_incidence_to_cumulative(), enw_incidence_to_linelist(), enw_linelist_to_incidence()

Examples

# Default reconstruct incidence
dt <- germany_covid19_hosp[location == "DE"][age_group == "00+"]
enw_add_incidence(dt)

# Make use of maximum reported to calculate empirical daily reporting
dt <- enw_add_max_reported(dt)
enw_add_incidence(dt)

Description

Add the latest observations to the nowcast output. This is useful for plotting the nowcast against the latest observations.

Usage

enw_add_latest_obs_to_nowcast(nowcast, obs)

Arguments

Value

A data.frame of nowcast output with the latest observations added.

See Also

Functions used for postprocessing of model fits enw_nowcast_samples(), enw_nowcast_summary(), enw_posterior(), enw_pp_summary(), enw_quantiles_to_long(), enw_summarise_samples()

Examples

fit <- enw_example("nowcast")
obs <- enw_example("obs")
nowcast <- summary(fit, type = "nowcast")
enw_add_latest_obs_to_nowcast(nowcast, obs)

Description

This is a helper function which adds the maximum (in the sense of latest observed) number of reported cases for each reference_date and computes the proportion of already reported cases for each combination of reference_date and report_date.

Usage

enw_add_max_reported(obs, copy = TRUE)

Arguments

Value

A data.table with new columns max_confirm and cum_prop_reported. max_confirm is the maximum number of cases reported for a certain reference_date. cum_prop_reported is the proportion of cases for a certain reference_date that are reported until a given report_day, relative to all cases so far observed for this reference_date.

See Also

Preprocessing functions enw_add_delay(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

obs <- data.frame(report_date = as.Date("2021-01-01") + 0:2)
obs$reference_date <- as.Date("2021-01-01")
obs$confirm <- 1:3
enw_add_max_reported(obs)

Description

If not already present, annotates time series data with metadata commonly used in models: day of week, and days, weeks, and months since start of time series.

Usage

enw_add_metaobs_features(
  metaobs,
  holidays = NULL,
  holidays_to = "Sunday",
  datecol = "date"
)

Arguments

Details

Effects models often need to include covariates for time-based features, such as day of the week (e.g. to reflect different care-seeking and/or reporting behaviour).

This function is called from within enw_preprocess_data() to systematically annotate metaobs with these commonly used metadata, if not already present.

However, it can also be used directly on other data.

Value

A copy of the metaobs input, with additional columns:

• day_of_week, a factor of values as output from weekdays() and possibly as holiday_to if distinct from weekdays values

• day, numeric, 0 based from start of time series

• week, numeric, 0 based from start of time series

• month, numeric, 0 based from start of time series

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

# make some example date
nat_germany_hosp <- subset(
  germany_covid19_hosp,
  location == "DE" & age_group == "80+"
)[1:40]

basemeta <- enw_add_metaobs_features(
  nat_germany_hosp,
  datecol = "report_date"
)
basemeta

# with holidays - n.b.: holidays not found are silently ignored
holidaymeta <- enw_add_metaobs_features(
  nat_germany_hosp,
  datecol = "report_date",
  holidays = c(
    "2021-04-04", "2021-04-05",
    "2021-05-01", "2021-05-13",
    "2021-05-24"
  ),
  holidays_to = "Holiday"
)
holidaymeta
subset(holidaymeta, day_of_week == "Holiday")

Description

This function adds a pooling effect to the metadata returned by enw_effects_metadata(). It does this updating the fixed column to 0 for the effects that match the string argument and adding a new column var_name that is 1 for the effects that match the string argument and 0 otherwise.

Usage

enw_add_pooling_effect(effects, var_name = "sd", finder_fn = startsWith, ...)

Arguments

Value

A data.table with the following columns:

• effects: the name of the effect

• fixed: a logical indicating whether the effect is fixed (1) or random (0).

• Argument supplied to var_name: a logical indicating whether the effect should be pooled (1) or not (0).

See Also

Functions used to formulate models enw_add_cumulative_membership(), enw_design(), enw_effects_metadata(), enw_one_hot_encode_feature()

Examples

data <- data.frame(a = 1:3, b = as.character(1:3), c = c(1,1,2))
design <- enw_design(a ~ b + c, data)$design
effects <- enw_effects_metadata(design)
enw_add_pooling_effect(effects, prefix = "b")

Description

This function aggregates observations over a specified timestep, ensuring alignment on the same day of week for report and reference dates. It is useful for aggregating data to a weekly timestep, for example which may be desirable if testing using a weekly timestep or if you are very concerned about runtime. Note that the start of the timestep will be determined by min_date + a single timestep (i.e. the first timestep will be "2022-10-23" if the minimum reference date is "2022-10-16").

Usage

enw_aggregate_cumulative(
  obs,
  timestep = "day",
  by = NULL,
  min_reference_date = min(obs$reference_date, na.rm = TRUE),
  copy = TRUE
)

Arguments

Value

A data.table with aggregated observations.

See Also

Data converters enw_add_cumulative(), enw_add_incidence(), enw_cumulative_to_incidence(), enw_incidence_to_cumulative(), enw_incidence_to_linelist(), enw_linelist_to_incidence()

Examples

nat_hosp <- germany_covid19_hosp[location == "DE"][age_group == "00+"]
enw_aggregate_cumulative(nat_hosp, timestep = "week")

Description

Assign a group to each row of a data.table. If by is specified, then each unique combination of the columns in by will be assigned a unique group. If by is not specified, then all rows will be assigned to the same group.

Usage

enw_assign_group(obs, by = NULL, copy = TRUE)

Arguments

Value

A data.table with a .group column added ordered by .group and the existing key of obs.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

obs <- data.frame(x = 1:3, y = 1:3)
enw_assign_group(obs)
enw_assign_group(obs, by = "x")

Description

Ensures that all reference and report dates are present for all groups based on the maximum and minimum dates found in the data. This function may be of use to users when preprocessing their data. In general all features that you may consider using as grouping variables or as covariates need to be included in the by variable.

Usage

enw_complete_dates(
  obs,
  by = NULL,
  max_delay,
  min_date = min(obs$reference_date, na.rm = TRUE),
  max_date = max(obs$report_date, na.rm = TRUE),
  timestep = "day",
  missing_reference = TRUE,
  completion_beyond_max_report = FALSE,
  flag_observation = FALSE
)

Arguments

Value

A data.table with completed entries for all combinations of reference dates, groups and possible report dates.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

obs <- data.frame(
  report_date = c("2021-10-01", "2021-10-03"), reference_date = "2021-10-01",
  confirm = 1
)
enw_complete_dates(obs)

# Allow completion beyond the maximum date found in the data
enw_complete_dates(obs, completion_beyond_max_report = TRUE, max_delay = 10)

Description

This function is used internally by enw_preprocess_data() to combine various pieces of processed observed data into a single object. It is exposed to the user in order to allow for modular data preprocessing though this is not currently recommended. See documentation and code of enw_preprocess_data() for more on the expected inputs.

Usage

enw_construct_data(
  obs,
  new_confirm,
  latest,
  missing_reference,
  reporting_triangle,
  metareport,
  metareference,
  metadelay,
  max_delay,
  timestep,
  by
)

Arguments

Value

A data.table containing processed observations as a series of nested data.frames as well as variables containing metadata. These are:

• obs: (observations with the addition of empirical reporting proportions and restricted to the specified maximum delay).

• new_confirm: Incidence of notifications by reference and report date. Empirical reporting distributions are also added.

• latest: The latest available observations.

• missing_reference: Observations missing reference dates.

• reporting_triangle: Incident observations by report and reference date in the standard reporting triangle matrix format.

• metareference: Metadata reference dates derived from observations.

• metrareport: Metadata for report dates.

• metadelay: Metadata for reporting delays produced using enw_metadata_delay().

• max_delay: Maximum delay to be modelled by epinowcast.

• time: Numeric, number of timepoints in the data.

• snapshots: Numeric, number of available data snapshots to use for nowcasting.

• groups: Numeric, Number of groups/strata in the supplied observations (set using by).

• max_date: The maximum available report date.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

pobs <- enw_example("preprocessed")
enw_construct_data(
  obs = pobs$obs[[1]],
  new_confirm = pobs$new_confirm[[1]],
  latest = pobs$latest[[1]],
  missing_reference = pobs$missing_reference[[1]],
  reporting_triangle = pobs$reporting_triangle[[1]],
  metareport = pobs$metareport[[1]],
  metareference = pobs$metareference[[1]],
  metadelay = pobs$metadelay[[1]],
  max_delay = pobs$max_delay,
  timestep = pobs$timestep[[1]],
  by = c()
)

Description

This function is a wrapper around stats::model.matrix() that can optionally return a sparse design matrix defined as the unique number of rows in the design matrix and an index vector that allows the full design matrix to be reconstructed. This is useful for models that have many repeated rows in the design matrix and that are computationally expensive to fit. This function also allows for the specification of contrasts for categorical variables.

Usage

enw_design(formula, data, no_contrasts = FALSE, sparse = TRUE, ...)

Arguments

Value

A list containing the formula, the design matrix, and the index.

See Also

Functions used to formulate models enw_add_cumulative_membership(), enw_add_pooling_effect(), enw_effects_metadata(), enw_one_hot_encode_feature()

Examples

data <- data.frame(a = 1:3, b = as.character(1:3), c = c(1,1,2))
enw_design(a ~ b + c, data)
enw_design(a ~ b + c, data, no_contrasts = TRUE)
enw_design(a ~ b + c, data, no_contrasts = c("b"))
enw_design(a ~ c, data, sparse = TRUE)
enw_design(a ~ c, data, sparse = FALSE)

Description

This function extracts metadata from a design matrix and returns a data.table with the following columns:

• effects: the name of the effect

• fixed: a logical indicating whether the effect is fixed (1) or random (0).

It automatically drops the intercept (defined as "(Intercept)").

This function is useful for constructing a model design object for random effects when used in combination with ewn_add_pooling_effect.

Usage

enw_effects_metadata(design)

Arguments

Value

A data.table with the following columns:

• effects: the name of the effect

• fixed: a logical indicating whether the effect is fixed (1) or random (0)

See Also

Functions used to formulate models enw_add_cumulative_membership(), enw_add_pooling_effect(), enw_design(), enw_one_hot_encode_feature()

Examples

data <- data.frame(a = 1:3, b = as.character(1:3), c = c(1,1,2))
design <- enw_design(a ~ b + c, data)$design
enw_effects_metadata(design)

Description

Loads examples of nowcasts produced using example scripts. Used to streamline examples, in package tests and to enable users to explore package functionality without needing to install cmdstanr.

Usage

enw_example(
  type = c("nowcast", "preprocessed_observations", "observations", "script")
)

Arguments

Value

Depending on type, a data.table of the requested output OR the file name(s) to generate these outputs (type = "script")

See Also

Package data sets germany_covid19_hosp

Examples

# Load the nowcast
enw_example(type = "nowcast")

# Load the preprocessed observations
enw_example(type = "preprocessed_observations")

# Load the latest observations
enw_example(type = "observations")

# Load the script used to generate these examples
# Optionally source this script to regenerate the example
readLines(enw_example(type = "script"))

Description

Expectation model module

Usage

enw_expectation(
  r = ~0 + (1 | day:.group),
  generation_time = 1,
  observation = ~1,
  latent_reporting_delay = 1,
  data,
  ...
)

Arguments

Value

A list containing the supplied formulas, data passed into a list describing the models, a data.frame describing the priors used, and a function that takes the output data and priors and returns a function that can be used to sample from a tightened version of the prior distribution.

See Also

Model modules enw_fit_opts(), enw_missing(), enw_obs(), enw_reference(), enw_report()

Examples

enw_expectation(data = enw_example("preprocessed"))

Description

Extend a time series with additional dates. This is useful when extending the report dates of a time series to include future dates for nowcasting purposes or to include additional dates for backcasting when using a renewal process as the expectation model.

Usage

enw_extend_date(
  metaobs,
  days = 20,
  direction = c("end", "start"),
  timestep = "day"
)

Arguments

Value

A data.table with the same columns as metaobs but with additional rows for each date in the range of date to date + days (or date - days if direction = "start"). An additional variable observed is added with a value of FALSE for all new dates and TRUE for all existing dates.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

metaobs <- data.frame(date = as.Date("2021-01-01") + 0:4)
enw_extend_date(metaobs, days = 2)
enw_extend_date(metaobs, days = 2, direction = "start")

Description

This is a helper function which allows users to filter datasets by reference date. This is useful, for example, when evaluating nowcast performance against fully observed data. Users may wish to combine this function with enw_filter_report_dates(). Note that by definition it is assumed that report dates must be equal or greater than the corresponding reference date (i.e a report cannot happen before the event being reported occurs). This means that this function will also filter out any report dates that are earlier than their corresponding reference date.

Usage

enw_filter_reference_dates(
  obs,
  earliest_date,
  include_days,
  latest_date,
  remove_days
)

Arguments

Value

A data.table filtered by report date

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

# Filter by date
enw_filter_reference_dates(
  germany_covid19_hosp,
  earliest_date = "2021-09-01",
  latest_date = "2021-10-01"
)
#
# Filter by days
enw_filter_reference_dates(
  germany_covid19_hosp,
  include_days = 10, remove_days = 10
)

Description

This is a helper function which allows users to create truncated data sets at past time points from a given larger data set. This is useful when evaluating nowcast performance against fully observed data. Users may wish to combine this function with enw_filter_reference_dates().

Usage

enw_filter_report_dates(obs, latest_date, remove_days)

Arguments

Value

A data.table filtered by report date

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

# Filter by date
enw_filter_report_dates(germany_covid19_hosp, latest_date = "2021-09-01")

# Filter by days
enw_filter_report_dates(germany_covid19_hosp, remove_days = 10)

Description

Format model fitting options for use with stan

Usage

enw_fit_opts(
  sampler = epinowcast::enw_sample,
  nowcast = TRUE,
  pp = FALSE,
  likelihood = TRUE,
  likelihood_aggregation = c("snapshots", "groups"),
  threads_per_chain = 1L,
  debug = FALSE,
  output_loglik = FALSE,
  ...
)

Arguments

Value

A list containing the specified sampler function, data as a list specifying the fitting options to use, and additional arguments to pass to the sampler function when it is called.

See Also

Model modules enw_expectation(), enw_missing(), enw_obs(), enw_reference(), enw_report()

Examples

# Default options along with settings to pass to enw_sample
enw_fit_opts(iter_sampling = 1000, iter_warmup = 1000)

Description

Flags observations based on the 'confirm' column. If the '.observed' column does not exist, it is created. Observations are flagged as observed (TRUE) if 'confirm' is not NA.

Usage

enw_flag_observed_observations(obs, copy = TRUE)

Arguments

Value

A data.table with an additional column '.observed' indicating observed observations.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

dt <- data.frame(id = 1:3, confirm = c(NA, 1, 2))
enw_flag_observed_observations(dt)

Description

This function allows models to be defined using a flexible formula interface that supports fixed effects, random effects (using lme4 syntax). Note that the returned fixed effects design matrix is sparse and so the index supplied is required to link observations to the appropriate design matrix row.

Usage

enw_formula(formula, data, sparse = TRUE)

Arguments

Value

A list containing the following:

• formula: The user supplied formula

• parsed_formula: The formula as parsed by parse_formula()

• extended_formula: The flattened version of the formula with both user supplied terms and terms added for the user supplied complex model components.

• fixed: A list containing the fixed effect formula, sparse design matrix, and the index linking the design matrix with observations.

• random: A list containing the random effect formula, sparse design matrix, and the index linking the design matrix with random effects.

See Also

Functions used to help convert formulas into model designs as_string_formula(), construct_re(), construct_rw(), enw_manual_formula(), parse_formula(), remove_rw_terms(), re(), rw_terms(), rw(), split_formula_to_terms()

Examples

# Use meta data for references dates from the Germany COVID-19
# hospitalisation data.
obs <- enw_filter_report_dates(
  germany_covid19_hosp[location == "DE"],
  remove_days = 40
)
obs <- enw_filter_reference_dates(obs, include_days = 40)
pobs <- enw_preprocess_data(
  obs, by = c("age_group", "location"), max_delay = 20
  )
data <- pobs$metareference[[1]]

# Model with fixed effects for age group
enw_formula(~ 1 + age_group, data)

# Model with random effects for age group
enw_formula(~ 1 + (1 | age_group), data)

# Model with a random effect for age group and a random walk
enw_formula(~ 1 + (1 | age_group) + rw(week), data)

# Model defined without a sparse fixed effects design matrix
enw_formula(~1, data[1:20, ])

# Model using an interaction in the right hand side of a random effect
# to specify an independent random effect per strata.
enw_formula(~ (1 + day | week:month), data = data)

Description

Format formula data for use with stan

Usage

enw_formula_as_data_list(formula, prefix, drop_intercept = FALSE)

Arguments

Value

A list defining the model formula. This includes:

• ⁠prefix_fintercept:⁠ Is an intercept present for the fixed effects design matrix.

• prefix_fdesign: The fixed effects design matrix

• prefix_fnrow: The number of rows of the fixed design matrix

• prefix_findex: The index linking design matrix rows to observations

• prefix_fnindex: The length of the index

• prefix_fncol: The number of columns (i.e effects) in the fixed effect design matrix (minus 1 if drop_intercept = TRUE).

• prefix_rdesign: The random effects design matrix

• prefix_rncol: The number of columns (i.e random effects) in the random effect design matrix (minus 1 as the intercept is dropped).

See Also

Functions used to help convert models into the format required for stan enw_get_cache(), enw_model(), enw_priors_as_data_list(), enw_replace_priors(), enw_sample(), enw_set_cache(), enw_stan_to_r(), enw_unset_cache(), remove_profiling(), write_stan_files_no_profile()

Examples

f <- enw_formula(~ 1 + (1 | cyl), mtcars)
enw_formula_as_data_list(f, "mtcars")

# A missing formula produces the default list
enw_formula_as_data_list(prefix = "missing")

Description

Retrieves the user set cache location for Stan models. This path can be set through the enw_cache_location function call. If no environmental variable is available the output from tempdir() will be returned.

Usage

enw_get_cache()

Value

A string representing the file path for the cache location

See Also

Functions used to help convert models into the format required for stan enw_formula_as_data_list(), enw_model(), enw_priors_as_data_list(), enw_replace_priors(), enw_sample(), enw_set_cache(), enw_stan_to_r(), enw_unset_cache(), remove_profiling(), write_stan_files_no_profile()

Description

Imputes NA values in the 'confirm' column. NA values are replaced with the last available observation or 0.

Usage

enw_impute_na_observations(obs, by = NULL, copy = TRUE)

Arguments

Value

A data.table with imputed 'confirm' column where NA values have been replaced with zero.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

dt <- data.frame(
 id = 1:3, confirm = c(NA, 1, 2),
 reference_date = as.Date("2021-01-01")
)
enw_impute_na_observations(dt)

Description

This function takes a data.table of aggregate counts or something coercible to a data.table (such as a data.frame) and converts it to a line list where each row represents a case.

Usage

enw_incidence_to_linelist(
  obs,
  reference_date = "reference_date",
  report_date = "report_date"
)

Arguments

Value

A data.table with the following variables: id, reference_date, report_date, and any other variables in the obs object. Rows in obs will be duplicated based on the new_confirm column. reference_date and report_date may be renamed if reference_date and report_date are supplied.

See Also

Data converters enw_add_cumulative(), enw_add_incidence(), enw_aggregate_cumulative(), enw_cumulative_to_incidence(), enw_incidence_to_cumulative(), enw_linelist_to_incidence()

Examples

incidence <- enw_add_incidence(germany_covid19_hosp)
incidence <- enw_filter_reference_dates(
  incidence[location == "DE"], include_days = 10
)
enw_incidence_to_linelist(incidence, reference_date = "onset_date")

Description

Filter observations for the latest available reported data for each reference date. Note this is not the same as filtering for the maximum report date in all cases as data may only be updated up to some maximum number of days.

Usage

enw_latest_data(obs)

Arguments

Value

A data.table of observations filtered for the latest available data for each reference date.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

# Filter for latest reported data
enw_latest_data(germany_covid19_hosp)

Description

This function takes a line list (i.e. tabular data where each row represents a case) and aggregates to a count (new_confirm) of cases by user-specified reference_dates and report_dates. This is enables the use of enw_preprocess_data() and other epinowcast() preprocessing functions.

Usage

enw_linelist_to_incidence(
  linelist,
  reference_date = "reference_date",
  report_date = "report_date",
  by = NULL,
  max_delay,
  completion_beyond_max_report = FALSE,
  copy = TRUE
)

Arguments

Value

A data.table with the following variables: reference_date, report_date, new_confirm, confirm, delay, and any variables specified in by.

See Also

Data converters enw_add_cumulative(), enw_add_incidence(), enw_aggregate_cumulative(), enw_cumulative_to_incidence(), enw_incidence_to_cumulative(), enw_incidence_to_linelist()

Examples

linelist <- data.frame(
  onset_date = as.Date(c("2021-01-02", "2021-01-03", "2021-01-02")),
  report_date = as.Date(c("2021-01-03", "2021-01-05", "2021-01-04"))
)
enw_linelist_to_incidence(linelist, reference_date = "onset_date")

# Specify a custom maximum delay and allow completion beyond the maximum
# observed delay
enw_linelist_to_incidence(
 linelist, reference_date = "onset_date", max_delay = 5,
 completion_beyond_max_report = TRUE
)

Description

For most typical use cases enw_formula() should provide sufficient flexibility to allow models to be defined. However, there may be some instances where more manual model specification is required. This function supports this by allowing the user to supply vectors of fixed, random, and customised random effects (where they are not first treated as fixed effect terms). Prior to ⁠1.0.0⁠ this was the main interface for specifying models and it is still used internally to handle some parts of the model specification process.

Usage

enw_manual_formula(
  data,
  fixed = NULL,
  random = NULL,
  custom_random = NULL,
  no_contrasts = FALSE,
  add_intercept = TRUE
)

Arguments

Value

A list specifying the fixed effects (formula, design matrix, and design matrix index), and random effects (formula and design matrix).

See Also

Functions used to help convert formulas into model designs as_string_formula(), construct_re(), construct_rw(), enw_formula(), parse_formula(), remove_rw_terms(), re(), rw_terms(), rw(), split_formula_to_terms()

Examples

data <- enw_example("prep")$metareference[[1]]
enw_manual_formula(data, fixed = "week", random = "day_of_week")

Description

Extract metadata from raw data, either by reference or by report date. For the target date chosen (reference or report), confirm, ⁠max_confirm``, and ⁠cum_prop_reported' are dropped and the first observation for each group and date is retained.

Usage

enw_metadata(obs, target_date = c("reference_date", "report_date"))

Arguments

Value

A data.table with columns:

• date, a Date column

• .group, a grouping column

and the first observation for each group and date. The data.table is sorted by .group and date.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

obs <- data.frame(
  reference_date = as.Date("2021-01-01"),
  report_date = as.Date("2022-01-01"), x = 1:10
)
enw_metadata(obs, target_date = "reference_date")

Description

Calculate delay metadata based on the supplied maximum delay and independent of other metadata or date indexing. These data are meant to be used in conjunction with metadata on the date of reference. Users can build additional features with this data.frame or regenerate it using this function in the output of enw_preprocess_data().

Usage

enw_metadata_delay(max_delay = 20, breaks = 4, timestep = "day")

Arguments

Value

A data.frame of delay metadata. This includes:

• delay: The numeric delay from reference date to report.

• delay_cat: The categorised delay. This may be useful for model building.

• delay_week: The numeric week since the delay was reported. This again may be useful for model building.

• delay_head: A logical variable defining if the delay is in the lower 25% of the potential delays. This may be particularly useful when building models that assume a parametric distribution in order to increase the weight of the head of the reporting distribution in a pragmatic way.

• delay_tail: A logical variable defining if the delay is in the upper 75% of the potential delays. This may be particularly useful when building models that assume a parametric distribution in order to increase the weight of the tail of the reporting distribution in a pragmatic way.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

enw_metadata_delay(max_delay = 20, breaks = 4)

Description

Missing reference data model module

Usage

enw_missing(formula = ~1, data)

Arguments

Value

A list containing the supplied formulas, data passed into a list describing the models, a data.frame describing the priors used, and a function that takes the output data and priors and returns a function that can be used to sample from a tightened version of the prior distribution.

See Also

Model modules enw_expectation(), enw_fit_opts(), enw_obs(), enw_reference(), enw_report()

Examples

# Missingness model with a fixed intercept only
enw_missing(data = enw_example("preprocessed"))

# No missingness model specified
enw_missing(~0, data = enw_example("preprocessed"))

Description

Returns reports with missing reference dates as well as calculating the proportion of reports for a given reference date that were missing.

Usage

enw_missing_reference(obs)

Arguments

Value

A data.table of missing counts and proportions by report date and group.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_preprocess_data(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

obs <- data.frame(
  report_date = c("2021-10-01", "2021-10-03"), reference_date = "2021-10-01",
  confirm = 1
)
obs <- rbind(
  obs,
  data.frame(report_date = "2021-10-04", reference_date = NA, confirm = 4)
)
obs <- enw_complete_dates(obs)
obs <- enw_assign_group(obs)
obs <- enw_add_incidence(obs)
enw_missing_reference(obs)

Description

Load and compile the nowcasting model

Usage

enw_model(
  model = system.file("stan", "epinowcast.stan", package = "epinowcast"),
  include = system.file("stan", package = "epinowcast"),
  compile = TRUE,
  threads = TRUE,
  profile = FALSE,
  target_dir = epinowcast::enw_get_cache(),
  stanc_options = list(),
  cpp_options = list(),
  verbose = TRUE,
  ...
)

Arguments

Value

A cmdstanr model.

See Also

Functions used to help convert models into the format required for stan enw_formula_as_data_list(), enw_get_cache(), enw_priors_as_data_list(), enw_replace_priors(), enw_sample(), enw_set_cache(), enw_stan_to_r(), enw_unset_cache(), remove_profiling(), write_stan_files_no_profile()

Examples

mod <- enw_model()

Description

A generic wrapper around posterior::draws_df() with opinionated defaults to extract the posterior samples for the nowcast ("pp_inf_obs" from the stan code). The functionality of this function can be used directly on the output of epinowcast() using the supplied summary.epinowcast() method.

Usage

enw_nowcast_samples(fit, obs, max_delay = NULL, timestep = "day")

Arguments

Value

A data.frame of posterior samples for the nowcast prediction. This uses observed data where available and the posterior prediction where not.

See Also

Functions used for postprocessing of model fits enw_add_latest_obs_to_nowcast(), enw_nowcast_summary(), enw_posterior(), enw_pp_summary(), enw_quantiles_to_long(), enw_summarise_samples()

Examples

fit <- enw_example("nowcast")
enw_nowcast_samples(
  fit$fit[[1]],
  fit$latest[[1]],
  fit$max_delay,
  "day"
  )

Description

A generic wrapper around enw_posterior() with opinionated defaults to extract the posterior prediction for the nowcast ("pp_inf_obs" from the stan code). The functionality of this function can be used directly on the output of epinowcast() using the supplied summary.epinowcast() method.

Usage

enw_nowcast_summary(
  fit,
  obs,
  max_delay = NULL,
  timestep = "day",
  probs = c(0.05, 0.2, 0.35, 0.5, 0.65, 0.8, 0.95)
)

Arguments

Value

A data.frame summarising the model posterior nowcast prediction. This uses observed data where available and the posterior prediction where not.

See Also

summary.epinowcast()

Functions used for postprocessing of model fits enw_add_latest_obs_to_nowcast(), enw_nowcast_samples(), enw_posterior(), enw_pp_summary(), enw_quantiles_to_long(), enw_summarise_samples()

Examples

fit <- enw_example("nowcast")
enw_nowcast_summary(
  fit$fit[[1]],
  fit$latest[[1]],
  fit$max_delay
  )

Description

Setup observation model and data

Usage

enw_obs(family = c("negbin", "poisson"), observation_indicator = NULL, data)

Arguments

Value

A list as required by stan.

See Also

Model modules enw_expectation(), enw_fit_opts(), enw_missing(), enw_reference(), enw_report()

Examples

enw_obs(data = enw_example("preprocessed"))

Description

This function takes a data.frame and a categorical variable, performs one-hot encoding, and column-binds the encoded variables back to the data.frame.

Usage

enw_one_hot_encode_feature(metaobs, feature, contrasts = FALSE)

Arguments

See Also

Functions used to formulate models enw_add_cumulative_membership(), enw_add_pooling_effect(), enw_design(), enw_effects_metadata()

Examples

metaobs <- data.frame(week = 1:2)
enw_one_hot_encode_feature(metaobs, "week")
enw_one_hot_encode_feature(metaobs, "week", contrasts = TRUE)

metaobs <- data.frame(week = 1:6)
enw_one_hot_encode_feature(metaobs, "week")
enw_one_hot_encode_feature(metaobs, "week", contrasts = TRUE)

Description

Plot nowcast quantiles

Usage

enw_plot_nowcast_quantiles(nowcast, latest_obs = NULL, log = FALSE, ...)

Arguments

Value

A ggplot2 plot.

See Also

Plotting functions enw_plot_obs(), enw_plot_pp_quantiles(), enw_plot_quantiles(), enw_plot_theme(), plot.epinowcast()

Examples

nowcast <- enw_example("nowcast")
nowcast <- summary(nowcast, probs = c(0.05, 0.2, 0.8, 0.95))
enw_plot_nowcast_quantiles(nowcast)

Description

Generic quantile plot

Usage

enw_plot_obs(obs, latest_obs = NULL, log = TRUE, ...)

Arguments

Value

A ggplot2 plot.

See Also

Plotting functions enw_plot_nowcast_quantiles(), enw_plot_pp_quantiles(), enw_plot_quantiles(), enw_plot_theme(), plot.epinowcast()

Examples

nowcast <- enw_example("nowcast")
obs <- enw_example("obs")

# Plot observed data by reference date
enw_plot_obs(obs, x = reference_date)

# Plot observed data by reference date with more recent data
enw_plot_obs(nowcast$latest[[1]], obs, x = reference_date)

Description

Plot posterior prediction quantiles

Usage

enw_plot_pp_quantiles(pp, log = FALSE, ...)

Arguments

Value

A ggplot2 plot.

See Also

Plotting functions enw_plot_nowcast_quantiles(), enw_plot_obs(), enw_plot_quantiles(), enw_plot_theme(), plot.epinowcast()

Examples

nowcast <- enw_example("nowcast")
nowcast <- summary(
 nowcast, type = "posterior_prediction", probs = c(0.05, 0.2, 0.8, 0.95)
)
enw_plot_pp_quantiles(nowcast) +
 ggplot2::facet_wrap(ggplot2::vars(reference_date), scales = "free")

Description

Generic quantile plot

Usage

enw_plot_quantiles(posterior, latest_obs = NULL, log = FALSE, ...)

Arguments

Value

A ggplot2 plot.

See Also

enw_plot_nowcast_quantiles(), enw_plot_pp_quantiles()

Plotting functions enw_plot_nowcast_quantiles(), enw_plot_obs(), enw_plot_pp_quantiles(), enw_plot_theme(), plot.epinowcast()

Examples

nowcast <- enw_example("nowcast")
nowcast <- summary(nowcast, probs = c(0.05, 0.2, 0.8, 0.95))
enw_plot_quantiles(nowcast, x = reference_date)

Description

Package plot theme

Usage

enw_plot_theme(plot)

Arguments

Value

ggplot2 plot object.

See Also

Plotting functions enw_plot_nowcast_quantiles(), enw_plot_obs(), enw_plot_pp_quantiles(), enw_plot_quantiles(), plot.epinowcast()

Description

A generic wrapper around posterior::summarise_draws() with opinionated defaults.

Usage

enw_posterior(fit, variables = NULL, probs = c(0.05, 0.2, 0.8, 0.95), ...)

Arguments

Value

A data.frame summarising the model posterior.

See Also

Functions used for postprocessing of model fits enw_add_latest_obs_to_nowcast(), enw_nowcast_samples(), enw_nowcast_summary(), enw_pp_summary(), enw_quantiles_to_long(), enw_summarise_samples()

Examples

fit <- enw_example("nowcast")
enw_posterior(fit$fit[[1]], variables = "expr_beta")

Description

This function summarises posterior predictives for observed data (by report and reference date). The functionality of this function can be used directly on the output of epinowcast() using the supplied summary.epinowcast() method.

Usage

enw_pp_summary(fit, diff_obs, probs = c(0.05, 0.2, 0.35, 0.5, 0.65, 0.8, 0.95))

Arguments

Value

A data.table summarising the posterior predictions.

See Also

Functions used for postprocessing of model fits enw_add_latest_obs_to_nowcast(), enw_nowcast_samples(), enw_nowcast_summary(), enw_posterior(), enw_quantiles_to_long(), enw_summarise_samples()

Examples

fit <- enw_example("nowcast")
enw_pp_summary(fit$fit[[1]], fit$new_confirm[[1]], probs = c(0.5))

Description

This function preprocesses raw observations under the assumption they are reported as cumulative counts by a reference and report date and is used to assign groups. It also constructs data objects used by visualisation and modelling functions including the observed empirical probability of a report on a given day, the cumulative probability of report, the latest available observations, incidence of observations, and metadata about the date of reference and report (used to construct models). This function wraps other preprocessing functions that may be instead used individually if required. Note that internally reports beyond the user specified delay are dropped for modelling purposes with the cum_prop_reported and max_confirm variables allowing the user to check the impact this may have (if cum_prop_reported is significantly below 1 a longer max_delay may be appropriate). Also note that if missing reference or report dates are suspected to occur in your data then these need to be completed with enw_complete_dates().

Usage

enw_preprocess_data(
  obs,
  by = NULL,
  max_delay,
  timestep = "day",
  set_negatives_to_zero = TRUE,
  ...,
  copy = TRUE
)

Arguments

Details

If max_delay is numeric, it will be internally coerced to integer using as.integer()).

Value

A data.table containing processed observations as a series of nested data.frames as well as variables containing metadata. These are:

• obs: (observations with the addition of empirical reporting proportions and restricted to the specified maximum delay).

• new_confirm: Incidence of notifications by reference and report date. Empirical reporting distributions are also added.

• latest: The latest available observations.

• missing_reference: Observations missing reference dates.

• reporting_triangle: Incident observations by report and reference date in the standard reporting triangle matrix format.

• metareference: Metadata reference dates derived from observations.

• metrareport: Metadata for report dates.

• metadelay: Metadata for reporting delays produced using enw_metadata_delay().

• max_delay: Maximum delay to be modelled by epinowcast.

• time: Numeric, number of timepoints in the data.

• snapshots: Numeric, number of available data snapshots to use for nowcasting.

• groups: Numeric, Number of groups/strata in the supplied observations (set using by).

• max_date: The maximum available report date.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_reporting_triangle_to_long(), enw_reporting_triangle()

Examples

library(data.table)

# Filter example hospitalisation data to be national and over all ages
nat_germany_hosp <- germany_covid19_hosp[location == "DE"]
nat_germany_hosp <- nat_germany_hosp[age_group == "00+"]

# Preprocess with default settings
pobs <- enw_preprocess_data(nat_germany_hosp)
pobs

Description

Converts priors defined in a data.frame into a list format for use by stan. In addition it adds "_p" to all variable names in order too allow them to be distinguished from their standard usage within modelling code.

Usage

enw_priors_as_data_list(priors)

Arguments

Value

A named list with each entry specifying a prior as a length two vector (specifying the mean and standard deviation of the prior).

See Also

Functions used to help convert models into the format required for stan enw_formula_as_data_list(), enw_get_cache(), enw_model(), enw_replace_priors(), enw_sample(), enw_set_cache(), enw_stan_to_r(), enw_unset_cache(), remove_profiling(), write_stan_files_no_profile()

Examples

priors <- data.frame(variable = "x", mean = 1, sd = 2)
enw_priors_as_data_list(priors)

Description

Convert summarised quantiles from wide to long format

Usage

enw_quantiles_to_long(posterior)

Arguments

Value

A data.frame of quantiles in long format.

See Also

Functions used for postprocessing of model fits enw_add_latest_obs_to_nowcast(), enw_nowcast_samples(), enw_nowcast_summary(), enw_posterior(), enw_pp_summary(), enw_summarise_samples()

Examples

fit <- enw_example("nowcast")
posterior <- enw_posterior(fit$fit[[1]], var = "expr_lelatent_int[1,1]")
enw_quantiles_to_long(posterior)

Description

Reference date logit hazard reporting model module

Usage

enw_reference(
  parametric = ~1,
  distribution = c("lognormal", "none", "exponential", "gamma", "loglogistic"),
  non_parametric = ~0,
  data
)

Arguments

Value

A list containing the supplied formulas, data passed into a list describing the models, a data.frame describing the priors used, and a function that takes the output data and priors and returns a function that can be used to sample from a tightened version of the prior distribution.

See Also

Model modules enw_expectation(), enw_fit_opts(), enw_missing(), enw_obs(), enw_report()

Examples

# Parametric model with a lognormal distribution
enw_reference(
 parametric = ~1, distribution = "lognormal",
 data = enw_example("preprocessed")
)

# Non-parametric model with a random effect per delay
enw_reference(
 parametric = ~ 0, non_parametric = ~ 1 + (1 | delay),
 data = enw_example("preprocessed")
)

# Combined parametric and non-parametric model
enw_reference(
 parametric = ~ 1, non_parametric = ~ 0 + (1 | delay_cat),
 data = enw_example("preprocessed")
)

Description

Construct a lookup of references dates by report

Usage

enw_reference_by_report(
  missing_reference,
  reps_with_complete_refs,
  metareference,
  max_delay
)

Arguments

Value

A wide data.frame with each row being a complete report date and' the columns being the observation index for each reporting delay

See Also

Helper functions for model modules add_max_observed_delay(), add_pmfs(), convolution_matrix(), enw_reps_with_complete_refs(), extract_obs_metadata(), extract_sparse_matrix(), latest_obs_as_matrix(), simulate_double_censored_pmf()

Description

This function is used internally by epinowcast to replace default model priors with users specified ones (restricted to normal priors with specified mean and standard deviations). A common use would be extracting the posterior from a previous epinowcast() run (using summary(nowcast, type = fit)) and using this a prior.

Usage

enw_replace_priors(priors, custom_priors)

Arguments

Value

A data.table of prior definitions (variable, mean and sd).

See Also

Functions used to help convert models into the format required for stan enw_formula_as_data_list(), enw_get_cache(), enw_model(), enw_priors_as_data_list(), enw_sample(), enw_set_cache(), enw_stan_to_r(), enw_unset_cache(), remove_profiling(), write_stan_files_no_profile()

Examples

# Update priors from a data.frame
priors <- data.frame(variable = c("x", "y"), mean = c(1, 2), sd = c(1, 2))
custom_priors <- data.frame(variable = "x[1]", mean = 10, sd = 2)
enw_replace_priors(priors, custom_priors)

# Update priors from a previous model fit
default_priors <- enw_reference(
 distribution = "lognormal",
 data = enw_example("preprocessed"),
)$priors
print(default_priors)

fit_priors <- summary(
 enw_example("nowcast"), type = "fit",
 variables = c("refp_mean_int", "refp_sd_int", "sqrt_phi")
)
fit_priors

enw_replace_priors(default_priors, fit_priors)

Description

Report date logit hazard reporting model module

Usage

enw_report(non_parametric = ~0, structural = ~0, data)

Arguments

Value

A list containing the supplied formulas, data passed into a list describing the models, a data.frame describing the priors used, and a function that takes the output data and priors and returns a function that can be used to sample from a tightened version of the prior distribution.

See Also

Model modules enw_expectation(), enw_fit_opts(), enw_missing(), enw_obs(), enw_reference()

Examples

enw_report(data = enw_example("preprocessed"))

Description

Constructs the reporting triangle with each row representing a reference date and columns being observations by report date

Usage

enw_reporting_triangle(obs)

Arguments

Value

A data.frame with each row being a reference date, and columns being observations by reporting delay.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle_to_long()

Examples

obs <- enw_example("preprocessed")$new_confirm
enw_reporting_triangle(obs)

Description

Recast the reporting triangle from wide to long format

Usage

enw_reporting_triangle_to_long(obs)

Arguments

Value

A long format reporting triangle as a data.frame with additional variables new_confirm and delay.

See Also

Preprocessing functions enw_add_delay(), enw_add_max_reported(), enw_add_metaobs_features(), enw_assign_group(), enw_complete_dates(), enw_construct_data(), enw_extend_date(), enw_filter_delay(), enw_filter_reference_dates(), enw_filter_report_dates(), enw_flag_observed_observations(), enw_impute_na_observations(), enw_latest_data(), enw_metadata_delay(), enw_metadata(), enw_missing_reference(), enw_preprocess_data(), enw_reporting_triangle()

Examples

obs <- enw_example("preprocessed")$new_confirm
rt <- enw_reporting_triangle(obs)
enw_reporting_triangle_to_long(rt)

Description

Identify report dates with complete (i.e up to the maximum delay) reference dates

Usage

enw_reps_with_complete_refs(new_confirm, max_delay, by = NULL, copy = TRUE)

Arguments

Value

A data.frame containing a report_date variable, and grouping variables specified for report dates that have complete reporting.