IOVsearch#

The IOVsearch tool is a general tool to decide the best IOV structure given a start model. This includes adding IOV as well as testing to remove associated IIVs.

Running#

The IOVsearch tool is available both in Pharmpy/pharmr and from the command line.

To initiate IOVsearch in Python/R:

from pharmpy.modeling import read_model
from pharmpy.tools import read_modelfit_results, run_iovsearch

start_model = read_model('path/to/model')
start_model_results = read_modelfit_results('path/to/model')
res = run_iovsearch(model=start_model,
                    results=start_model_results,
                    column='OCC',
                    list_of_parameters=None,
                    distribution='same-as-iiv',
                    rank_type='bic',
                    cutoff=None)

This will take an input model model and use the column 'OCC' as the occasion column. IOV will be tested on parameters in list_of_parameters, which when none will be all parameters with IIV. The IOVs will have the same distribution as the IIVs. The candidate models will be ranked using bic with default cutoff, which for BIC is none.

You can limit the search to only certain parameters by giving a list:

res = run_iovsearch(model=start_model,
                    results=start_model_results,
                    column='OCC',
                    list_of_parameters=['CL', 'V'])

To run IOVsearch from the command line, the example code is redefined accordingly:

pharmpy run iovsearch path/to/model --column 'OCC' --distribution 'same-as-iiv' --rank_type 'bic'

Arguments#

Mandatory#

Argument

Description

model

Start model

results

ModelfitResults of start model

Optional#

Argument

Description

column

Name of column in dataset to use as occasion column (default is ‘OCC’). Note that this only makes sense for discrete occasion data.

list_of_parameters

List of parameters to test IOV on, if none all parameters with IIV will be tested (default)

distribution

Which distribution added IOVs should have (default is same as IIVs)

rank_type

Which selection criteria to rank models on, e.g. OFV (default is BIC).

cutoff

cutoff for the ranking function, exclude models that are below cutoff (default is none).

strictness

strictness criteria for model selection. Default is “minimization_successful or (rounding_errors and sigdigs>= 0.1)”

Algorithm#

The current algorithm uses a brute force approach where it first tests which IOVs should be used, then if any associated IIVs can be removed.

digraph G { draw [ label = "Input model"; shape = rect; ]; add_iov [ label = "Add IOV to all given parameters or all parameters with IIV"; shape = rect; ]; remove_iov [ label = "Create candidates where each possible subset of IOV is removed"; shape = rect; ] better_iov [ label = "Any candidate better than input?"; shape = rect; ] best_model_iov_no [ label = "Select input model"; shape = rect; ] best_model_iov_yes [ label = "Select best candidate model"; shape = rect; ] remove_iiv [ label = "Create candidates where each possible subset\n of IIVs connected to IOV is removed"; shape = rect; ] better_iiv [ label = "Any candidate better than previous?"; shape = rect; ] best_model_iiv_yes [ label = "Select best candidate model"; shape = rect; ] best_model_iiv_no [ label = "Select model with all IIVs"; shape = rect; ] done [ label = "Best model"; shape = rect; ] draw -> add_iov; add_iov -> remove_iov[label = "Fit model"]; remove_iov -> better_iov[label = "Fit models"]; better_iov -> best_model_iov_yes[label = "Yes"]; better_iov -> best_model_iov_no [label = "No"]; best_model_iov_no -> done; best_model_iov_yes -> remove_iiv[label = "Fit models"]; remove_iiv -> better_iiv; better_iiv -> best_model_iiv_yes[label = "Yes"]; better_iiv -> best_model_iiv_no[label = "No"]; best_model_iiv_yes -> done; best_model_iiv_no -> done; }

Distribution of new IOVs#

The distribution option determines how the added IOVs should be distributed. The different options are described below.

Distribution

Description

'same-as-iiv'

Copies the distribution of IIV etas (default)

'disjoint'

Disjoint normal distributions

'joint'

Joint normal distribution

'explicit'

Explicit mix of joint and disjoint distribution

By default, or when specifying 'same-as-iiv', you get the same covariance structure for added IOVs as the one that already exists for IIVs. If you want a different structure, you can specify 'disjoint' to force zero covariance between added IOVs, or 'joint' to force nonzero covariance. To get full control over the covariance you can specify 'explicit' and give the structure explicitly as in the following example:

res = run_iovsearch(model=start_model,
                    results=start_model_results,
                    column='OCC',
                    list_of_parameters=[['CL', 'V'], ['KA']],
                    distribution='explicit')

In this example, the newly added clearance (CL) and volume (V) IOVs will have nonzero mutual covariance, but will have zero covariance with the absorption constant (KA) IOV.

Comparing and ranking candidates#

The supplied rank_type will be used to compare a set of candidate models and rank them. A cutoff may also be provided if the user does not want to use the default. The following rank functions are available:

Rank type

Description

'ofv'

ΔOFV. Default is to not rank candidates with ΔOFV < cutoff (default 3.84)

'aic'

ΔAIC. Default is to rank all candidates if no cutoff is provided.

'bic'

ΔBIC (random). Default is to rank all candidates if no cutoff is provided.

Information about how BIC is calculated can be found in pharmpy.modeling.calculate_bic().

Results#

The results object contains various summary tables which can be accessed in the results object, as well as files in .csv/.json format. The name of the selected best model (based on the input selection criteria) is also included.

Consider a IOVsearch run:

res = run_iovsearch(column='VISI',
                    model=start_model,
                    results=start_model_results,
                    list_of_parameters=None,
                    rank_type='bic',
                    cutoff=None,
                    distribution='same-as-iiv')

The summary_tool table contains information such as which feature each model candidate has, the difference to the start model (in this case comparing BIC), and final ranking:

description n_params d_params dbic bic rank parent_model
step model
1 iovsearch_run7 IIV([CL]+[VC]+[MAT]);IOV([CL]) 8 -2 3.540508 -1265.694052 1 iovsearch_run1
iovsearch_run5 IIV([CL]+[VC]+[MAT]);IOV([MAT]) 8 -2 3.230772 -1265.384316 2 iovsearch_run1
iovsearch_run6 IIV([CL]+[VC]+[MAT]);IOV([VC]) 8 -2 3.230394 -1265.383938 3 iovsearch_run1
mox2 7 0 0.000000 -1262.153544 4 mox2
iovsearch_run3 IIV([CL]+[VC]+[MAT]);IOV([CL]+[MAT]) 9 -1 -0.750000 -1261.403544 5 iovsearch_run1
iovsearch_run4 IIV([CL]+[VC]+[MAT]);IOV([CL]+[VC]) 9 -1 -0.750400 -1261.403145 6 iovsearch_run1
iovsearch_run2 IIV([CL]+[VC]+[MAT]);IOV([VC]+[MAT]) 9 -1 -1.060114 -1261.093431 7 iovsearch_run1
iovsearch_run1 IIV([CL]+[VC]+[MAT]);IOV([CL]+[VC]+[MAT]) 10 3 -5.043077 -1257.110467 8 mox2
2 iovsearch_run7 IIV([CL]+[VC]+[MAT]);IOV([CL]) 8 0 0.000000 -1265.694052 1 iovsearch_run1
iovsearch_run8 IIV([VC]+[MAT]);IOV([CL]) 7 -1 -149.782429 -1115.911624 2 iovsearch_run7

To see information about the actual model runs, such as minimization status, estimation time, and parameter estimates, you can look at the summary_models table. The table is generated with pharmpy.tools.summarize_modelfit_results().

description minimization_successful errors_found warnings_found ofv runtime_total estimation_runtime POP_CL_estimate POP_VC_estimate POP_MAT_estimate IIV_CL_estimate IIV_VC_estimate IIV_MAT_estimate RUV_PROP_estimate OMEGA_IOV_1_estimate OMEGA_IOV_2_estimate OMEGA_IOV_3_estimate
step model
0 mox2 True 0 0 -1292.186761 4.0 0.10 24.5328 104.230 0.433676 0.481858 0.593654 0.000130 0.209972 NaN NaN NaN
1 iovsearch_run1 IIV([CL]+[VC]+[MAT]);IOV([CL]+[VC]+[MAT]) True 0 1 -1300.015062 4.0 0.39 24.1954 104.242 0.440613 0.474140 0.586866 0.311861 0.194680 0.002735 0.000006 1.000000e-07
iovsearch_run2 IIV([CL]+[VC]+[MAT]);IOV([VC]+[MAT]) True 0 1 -1299.707566 12.0 0.26 24.1403 104.665 0.439784 0.472835 0.590422 0.320387 0.195532 NaN 0.000001 1.000000e-06
iovsearch_run3 IIV([CL]+[VC]+[MAT]);IOV([CL]+[MAT]) True 0 1 -1300.017679 11.0 0.26 24.1951 104.241 0.440613 0.474125 0.586859 0.311859 0.194676 0.002735 NaN 1.000000e-06
iovsearch_run4 IIV([CL]+[VC]+[MAT]);IOV([CL]+[VC]) True 0 1 -1300.017280 12.0 0.26 24.1950 104.242 0.440611 0.474111 0.586860 0.311903 0.194675 0.002735 0.000001 NaN
iovsearch_run5 IIV([CL]+[VC]+[MAT]);IOV([MAT]) True 0 1 -1299.707992 11.0 0.17 24.1402 104.665 0.439785 0.472835 0.590425 0.320393 0.195531 NaN NaN 1.000000e-06
iovsearch_run6 IIV([CL]+[VC]+[MAT]);IOV([VC]) True 0 1 -1299.707614 12.0 0.18 24.1403 104.665 0.439783 0.472840 0.590424 0.320395 0.195532 NaN 0.000001 NaN
iovsearch_run7 IIV([CL]+[VC]+[MAT]);IOV([CL]) True 0 1 -1300.017728 11.0 0.04 24.1954 104.242 0.440613 0.474140 0.586866 0.311861 0.194680 0.002735 NaN NaN
2 iovsearch_run8 IIV([VC]+[MAT]);IOV([CL]) True 0 0 -1145.944840 4.0 0.16 24.2313 104.887 0.438628 NaN 0.572627 0.290631 0.197005 0.483460 NaN NaN

Finally, you can see a summary of different errors and warnings in summary_errors. See pharmpy.tools.summarize_errors() for information on the content of this table.

time message
model category error_no
iovsearch_run1 WARNING 0 2022-12-14 16:07:14.279 PARAMETER ESTIMATE IS NEAR ITS BOUNDARY
iovsearch_run2 WARNING 0 2022-12-14 16:07:14.279 PARAMETER ESTIMATE IS NEAR ITS BOUNDARY
iovsearch_run3 WARNING 0 2022-12-14 16:07:14.279 PARAMETER ESTIMATE IS NEAR ITS BOUNDARY
iovsearch_run4 WARNING 0 2022-12-14 16:07:14.279 PARAMETER ESTIMATE IS NEAR ITS BOUNDARY
iovsearch_run5 WARNING 0 2022-12-14 16:07:14.279 PARAMETER ESTIMATE IS NEAR ITS BOUNDARY
iovsearch_run6 WARNING 0 2022-12-14 16:07:14.279 PARAMETER ESTIMATE IS NEAR ITS BOUNDARY
iovsearch_run7 WARNING 0 2022-12-14 16:07:14.279 MINIMIZATION SUCCESSFUL\nHOWEVER, PROBLEMS OCCURRED WITH THE MINIMIZATION.