COVsearch#

The COVsearch tool is a general tool to identify covariates that explain some of the inter-individual variability.

Running#

The COVsearch tool is available both in Pharmpy/pharmr and from the command line.

To initiate COVsearch in Python/R:

from pharmpy.modeling import read_model
from pharmpy.tools import run_covsearch, read_modelfit_results

start_model = read_model('path/to/model')
start_model_results = read_modelfit_results('path/to/model')
res = run_covsearch(algorithm='scm-forward-then-backward',
                    model=start_model,
                    results=start_model_results,
                    search_space='COVARIATE?(@IIV, @CONTINUOUS, *); COVARIATE?(@IIV, @CATEGORICAL, CAT)',
                    p_forward=0.05,
                    p_backward=0.01,
                    max_steps=5)

In this example, we attempt up to five forward steps of the Stepwise Covariate Modeling (SCM) algorithm on the model start_model. The p-value threshold for these steps is 5% and the candidate search_space consists of all (*) supported effects (multiplicative) of continuous covariates on parameters with IIV, and a (multiplicative) categorical effect of categorical covariates on parameters with IIV. Once we have identified the best model with this method, we attempt up to k-1 backward steps of the SCM algorithm on this model, where k is the number of successful forward steps. The p-value threshold for the backward steps is 1% and the effects that are candidate for removal are only the ones that have been added by the forward steps.

To run COVsearch from the command line, the example code is redefined accordingly:

pharmpy run covsearch path/to/model --algorithm scm-forward-then-backward --search_space 'COVARIATE?(@IIV, @CONTINUOUS, *); COVARIATE?(@IIV, @CATEGORICAL, CAT)' --p_forward 0.05 --p_backward 0.01 --max_steps 5

Arguments#

Mandatory#

Argument

Description

search_space

The candidate parameter-covariate search_space to search through (required)

results

ModelfitResults of start model

model

Start model

Optional#

Argument

Description

p_forward

The p-value threshold for forward steps (default is 0.01)

p_backward

The p-value threshold for backward steps (default is 0.001)

max_steps

The maximum number of search algorithm steps to perform, or -1 for no maximum (default).

algorithm

The search algorithm to use (default is ‘scm-forward-then-backward’)

max_eval

Limit the number of function evaluations to 3.1 times that of the base model. Default is False.

adaptive_scope_reduction

Stash all non-significant parameter-covariate effects at each step in the forward search for later evaluation. As soon as all significant effects have been tested the stashed effects gets evaluated in a normal forward approach. Default is False

strictness

Strictness criteria for model selection. Default is “minimization_successful or (rounding_errors and sigdigs>= 0.1)”

Search space#

There are two kinds of candidate effects that can be described through the model feature language (MFL). For instance, say that we want to have want to forcefully add an exponential effect on volume through the weight covariate as well as testing adding an exponential effect on clearance with age as covariate. In this case, the following MFL specification can be used:

run_covsearch(
    ...
    search_space='COVARIATE(CL, WT, EXP);COVARIATE?(V,AGE,EXP)',
    ...
)

Note

COVARIATE(...) represent structural covariates while COVARIATE?(...) represent exploratory.

The search space is specified by first writing the parameters, then the covariates of interest, which effect, and, optionally, the operation to use for the covariate effect (‘*’ (default) or ‘+’). If the operation is omitted, the default operation will be used.

The MFL also provides additional features such as automatically- or manually-defined symbols. For instance the example above can be rewritten as

run_covsearch(
    ...
    effects='LET(CONTINUOUS, [AGE,WT]);COVARIATE?([CL, V], @CONTINUOUS, EXP)'
    ...
)

Notice how multiple statements are separated by semicolons ;. Omitting declaration of continuous covariates allows to let Pharmpy automatically derive which covariates should be referred to by @CONTINUOUS. For instance,

run_covsearch(
    ...
    search_space='COVARIATE?([CL, V], @CONTINUOUS, EXP)'
    ...
)

would test an exponential covariate effect on clearance and volume for each continuous covariate.

Note

Covariates that are already present in the model will be removed, unless they are also part of the search space. See Algorithm for more.

More automatic symbols are available. They are described in the MFL symbols section.

Wildcards#

In addition to symbols, using a wildcard * can help refer to computed list of values. For instance the MFL sentence COVARIATE?(*, *, *) represents “All continuous covariate effects of all covariates on all PK parameters”.

Type

Description of wildcard definition

Covariate

All covariates

Effect

All continuous effects

Parameter

All PK parameters

Note

Wildcard for effects cannot be used with structural covariates as only a single effect can be added per covariate for a certain parameter.

Algorithm#

The current default search algorithm ‘scm-forward-then-backward’ consists of the SCM method with forward steps followed by backward steps. The covariate effects that are added are dependent on the effects that are already present in the input model. All covariate effects that are initially part of the model but are not part of the search space will be removed before starting the search. Covariate effects that are part of both the search space and the model will be left in the model but are removed from the search space. In this initial stage, any structural covariates defined within the search space (see search_space) will be added as well. If any filtration is done, a new “filtered_input_model” is created, otherwise the input model will be used.

Note

If a filtered model is required, the changes made is reflected in its description.

digraph BST { node [fontname="Arial"]; base [label="Base model"] s0 [label="AddEffect(CL, SEX, CAT)"] s1 [label="AddEffect(CL, WT, EXP)"] s2 [label="AddEffect(V, SEX, CAT)"] s3 [label="AddEffect(V, WT, EXP)"] s4 [label="AddEffect(CL, SEX, CAT)"] s5 [label="AddEffect(CL, WT, EXP)"] s6 [label="AddEffect(V, SEX, CAT)"] s7 [label="AddEffect(CL, WT, EXP)"] s8 [label="AddEffect(V, SEX, CAT)"] s9 [label="Forward search best model"] s10 [label="RemoveEffect(V, WT, EXP)"] s11 [label="RemoveEffect(CL, SEX, CAT)"] s12 [label="Backward search best model"] base -> s0 base -> s1 base -> s2 base -> s3 s3 -> s4 s3 -> s5 s3 -> s6 s4 -> s7 s4 -> s8 s4 -> s9 s9 -> s10 s9 -> s11 s9 -> s12 }

To skip the backward steps use search algorithm ‘scm-forward’.

digraph BST { node [fontname="Arial"]; base [label="Base model"] s0 [label="AddEffect(CL, SEX, CAT)"] s1 [label="AddEffect(CL, WT, EXP)"] s2 [label="AddEffect(V, SEX, CAT)"] s3 [label="AddEffect(V, WT, EXP)"] s4 [label="AddEffect(CL, SEX, CAT)"] s5 [label="AddEffect(CL, WT, EXP)"] s6 [label="AddEffect(V, SEX, CAT)"] s7 [label="AddEffect(CL, WT, EXP)"] s8 [label="AddEffect(V, SEX, CAT)"] s9 [label="Forward search best model"] base -> s0 base -> s1 base -> s2 base -> s3 s3 -> s4 s3 -> s5 s3 -> s6 s4 -> s7 s4 -> s8 s4 -> s9 }

Adaptive scope reduction#

The adaptive scope reduction option integrates part of the SCM+ procedure within the covsearch tool. This option will modify the forward search such that only significant effects will be transferred to the next step. Insignificant effects are stored away. The number of possible steps in this search is dependent on the max_steps argument. With the resulting model from this search as input, a regular ‘scm-forward’ procedure is applied, using only the previously insignificant effects. The number of possible steps in this procedure is also determined by the max_steps argument.

If ‘scm-forward-then-backward’ is used, a subsequent backward search will follow.

Results#

The results object contains various summary tables which can be accessed in the results object, as well as files in .csv/.json format. The name of the selected best model (based on the input selection criteria) is also included.

Consider a covsearch run:

res = run_covsearch(model=start_model, results=start_model_results,
                    search_space='COVARIATE?([CL, MAT, VC], [AGE, WT], EXP);COVARIATE?([CL, MAT, VC], [SEX], CAT)')

The summary_tool table contains information such as which feature each model candidate has, the difference with the start model (in this case comparing BIC), and final ranking:

res.summary_tool
description pvalue goal_pvalue is_backward selected n_params d_params dofv ofv parent_model
step model
0 mox2 NaN NaN False True 7 0 0.000000 -1292.186761 mox2
1 covsearch_run1 (CL-AGE-exp) 1.047127e-03 0.05 False True 8 1 10.742325 -1302.929085 mox2
covsearch_run2 (CL-SEX-cat) 1.710000e-13 0.05 False False 8 1 54.315057 -1346.501817 mox2
covsearch_run3 (CL-WT-exp) 2.568110e-10 0.05 False False 8 1 39.978215 -1332.164976 mox2
covsearch_run4 (MAT-AGE-exp) 7.494490e-06 0.05 False False 8 1 20.062682 -1312.249442 mox2
covsearch_run5 (MAT-SEX-cat) 7.498453e-06 0.05 False False 8 1 20.061671 -1312.248431 mox2
covsearch_run6 (MAT-WT-exp) 4.459287e-06 0.05 False False 8 1 21.056538 -1313.243298 mox2
covsearch_run7 (VC-AGE-exp) 5.280678e-06 0.05 False False 8 1 20.732689 -1312.919450 mox2
covsearch_run8 (VC-SEX-cat) 3.923500e-11 0.05 False False 8 1 43.651626 -1335.838386 mox2
covsearch_run9 (VC-WT-exp) 1.003390e-10 0.05 False False 8 1 41.814832 -1334.001593 mox2
2 covsearch_run10 (CL-AGE-exp);(CL-SEX-cat) 3.984500e-11 0.05 False False 9 1 54.363759 -1346.550520 covsearch_run1
covsearch_run11 (CL-AGE-exp);(CL-WT-exp) 6.372293e-08 0.05 False False 9 1 39.989077 -1332.175838 covsearch_run1
covsearch_run12 (CL-AGE-exp);(MAT-AGE-exp) 2.176965e-03 0.05 False True 9 1 20.136326 -1312.323087 covsearch_run1
covsearch_run13 (CL-AGE-exp);(MAT-SEX-cat) 2.207550e-04 0.05 False False 9 1 24.387950 -1316.574711 covsearch_run1
covsearch_run14 (CL-AGE-exp);(MAT-WT-exp) 1.626878e-04 0.05 False False 9 1 24.961688 -1317.148448 covsearch_run1
covsearch_run15 (CL-AGE-exp);(VC-AGE-exp) 1.564495e-03 0.05 False False 9 1 20.743393 -1312.930153 covsearch_run1
covsearch_run16 (CL-AGE-exp);(VC-SEX-cat) 7.539960e-10 0.05 False False 9 1 48.617998 -1340.804759 covsearch_run1
covsearch_run17 (CL-AGE-exp);(VC-WT-exp) 5.762000e-11 0.05 False False 9 1 53.641996 -1345.828756 covsearch_run1
3 covsearch_run18 (CL-AGE-exp);(MAT-AGE-exp);(CL-SEX-cat) 1.823200e-11 0.05 False False 10 1 65.288366 -1357.475127 covsearch_run12
covsearch_run19 (CL-AGE-exp);(MAT-AGE-exp);(CL-WT-exp) 2.417058e-08 0.05 False False 10 1 51.263168 -1343.449928 covsearch_run12
covsearch_run20 (CL-AGE-exp);(MAT-AGE-exp);(MAT-SEX-cat) 9.759040e-03 0.05 False False 10 1 26.814686 -1319.001446 covsearch_run12
covsearch_run21 (CL-AGE-exp);(MAT-AGE-exp);(MAT-WT-exp) 2.469024e-02 0.05 False False 10 1 25.181808 -1317.368568 covsearch_run12
covsearch_run22 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp) 2.811054e-02 0.05 False True 10 1 24.957602 -1317.144362 covsearch_run12
covsearch_run23 (CL-AGE-exp);(MAT-AGE-exp);(VC-SEX-cat) 2.642452e-09 0.05 False False 10 1 55.567343 -1347.754104 covsearch_run12
covsearch_run24 (CL-AGE-exp);(MAT-AGE-exp);(VC-WT-exp) 2.840779e-09 0.05 False False 10 1 55.426389 -1347.613149 covsearch_run12
4 covsearch_run25 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(CL-SE... 1.261720e-10 0.05 False False 11 1 66.324567 -1358.511327 covsearch_run22
covsearch_run26 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(CL-WT... 2.651215e-07 0.05 False False 11 1 51.446030 -1343.632790 covsearch_run22
covsearch_run27 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 1.954671e-03 0.05 False True 11 1 34.549224 -1326.735985 covsearch_run22
covsearch_run28 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-W... 2.163414e-05 0.05 False False 11 1 42.997342 -1335.184102 covsearch_run22
covsearch_run29 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(VC-SE... 3.926594e-09 0.05 False False 11 1 59.617389 -1351.804150 covsearch_run22
covsearch_run30 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(VC-WT... 9.491049e-09 0.05 False False 11 1 57.900404 -1350.087164 covsearch_run22
5 covsearch_run31 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 7.423998e-16 0.05 False False 12 1 99.566615 -1391.753375 covsearch_run27
covsearch_run32 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 4.706676e-07 0.05 False False 12 1 59.929659 -1352.116420 covsearch_run27
covsearch_run33 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 6.374390e-03 0.05 False True 12 1 41.990522 -1334.177283 covsearch_run27
covsearch_run34 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 4.054865e-07 0.05 False False 12 1 60.217267 -1352.404028 covsearch_run27
covsearch_run35 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 1.105347e-07 0.05 False False 12 1 62.729333 -1354.916093 covsearch_run27
6 covsearch_run36 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 1.300000e-14 0.05 False False 13 1 101.379219 -1393.565980 covsearch_run33
covsearch_run37 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 4.029682e-07 0.05 False False 13 1 67.670588 -1359.857348 covsearch_run33
covsearch_run38 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 7.641559e-06 0.05 False True 13 1 62.016040 -1354.202801 covsearch_run33
covsearch_run39 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 4.748923e-06 0.05 False False 13 1 62.926496 -1355.113256 covsearch_run33
7 covsearch_run40 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 1.363918e-18 0.05 False False 14 1 139.462177 -1431.648937 covsearch_run38
covsearch_run41 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 3.203548e-06 0.05 False False 14 1 83.706655 -1375.893415 covsearch_run38
covsearch_run42 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 2.052323e-04 0.05 False True 14 1 75.798603 -1367.985364 covsearch_run38
8 covsearch_run43 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 3.567877e-17 0.05 False False 15 1 146.801019 -1438.987780 covsearch_run42
covsearch_run44 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 4.507358e-05 0.05 False True 15 1 92.443408 -1384.630169 covsearch_run42
9 covsearch_run45 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... 1.510000e-13 0.05 False True 16 1 146.997461 -1439.184221 covsearch_run44
10 covsearch_run46 (MAT-AGE-exp);(VC-AGE-exp);(MAT-SEX-cat);(MAT-... NaN 0.01 True False 15 -1 146.322869 -1438.509630 covsearch_run45
covsearch_run47 (CL-AGE-exp);(VC-AGE-exp);(MAT-SEX-cat);(MAT-W... NaN 0.01 True False 15 -1 146.925020 -1439.111781 covsearch_run45
covsearch_run48 (CL-AGE-exp);(MAT-AGE-exp);(MAT-SEX-cat);(MAT-... NaN 0.01 True False 15 -1 142.814402 -1435.001163 covsearch_run45
covsearch_run49 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-W... NaN 0.01 True False 15 -1 146.862830 -1439.049591 covsearch_run45
covsearch_run50 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... NaN 0.01 True False 15 -1 146.449696 -1438.636456 covsearch_run45
covsearch_run51 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... NaN 0.01 True False 15 -1 NaN NaN covsearch_run45
covsearch_run52 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... NaN 0.01 True False 15 -1 NaN NaN covsearch_run45
covsearch_run53 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... NaN 0.01 True False 15 -1 146.801017 -1438.987778 covsearch_run45
covsearch_run54 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... NaN 0.01 True False 15 -1 NaN NaN covsearch_run45

To see information about the actual model runs, such as minimization status, estimation time, and parameter estimates, you can look at the summary_models table. The table is generated with pharmpy.tools.summarize_modelfit_results().

res.summary_models
description minimization_successful errors_found warnings_found ofv runtime_total estimation_runtime POP_CL_estimate POP_VC_estimate POP_MAT_estimate ... RUV_PROP_estimate POP_CLAGE_estimate POP_CLSEX_estimate POP_CLWT_estimate POP_MATAGE_estimate POP_MATSEX_estimate POP_MATWT_estimate POP_VCAGE_estimate POP_VCSEX_estimate POP_VCWT_estimate
step model
0 mox2 True 0 0 -1292.186761 4.0 0.10 24.5328 104.2300 0.433676 ... 0.209972 NaN NaN NaN NaN NaN NaN NaN NaN NaN
1 covsearch_run1 (CL-AGE-exp) True 0 0 -1302.929085 16.0 0.31 24.2632 103.2400 0.447194 ... 0.197452 -0.005338 NaN NaN NaN NaN NaN NaN NaN NaN
covsearch_run2 (CL-SEX-cat) True 0 0 -1346.501817 16.0 0.26 22.8757 106.2690 0.429235 ... 0.192852 NaN 0.382904 NaN NaN NaN NaN NaN NaN NaN
covsearch_run3 (CL-WT-exp) True 0 0 -1332.164976 16.0 0.25 24.4824 106.0490 0.435646 ... 0.199549 NaN NaN -0.010536 NaN NaN NaN NaN NaN NaN
covsearch_run4 (MAT-AGE-exp) True 0 0 -1312.249442 16.0 0.30 24.1536 103.7690 0.430342 ... 0.193411 NaN NaN NaN 0.059279 NaN NaN NaN NaN NaN
covsearch_run5 (MAT-SEX-cat) True 0 0 -1312.248431 16.0 0.23 23.9763 102.8640 0.428158 ... 0.195608 NaN NaN NaN NaN 1.517590 NaN NaN NaN NaN
covsearch_run6 (MAT-WT-exp) True 0 0 -1313.243298 16.0 0.33 23.9807 103.1030 0.505090 ... 0.192557 NaN NaN NaN NaN NaN -0.019706 NaN NaN NaN
covsearch_run7 (VC-AGE-exp) True 0 0 -1312.919450 16.0 0.31 24.3823 103.4790 0.453248 ... 0.198783 NaN NaN NaN NaN NaN NaN 0.014651 NaN NaN
covsearch_run8 (VC-SEX-cat) True 0 0 -1335.838386 16.0 0.21 23.3323 98.5279 0.434088 ... 0.191425 NaN NaN NaN NaN NaN NaN NaN 0.953372 NaN
covsearch_run9 (VC-WT-exp) True 0 0 -1334.001593 4.0 0.14 23.7087 109.1750 0.459561 ... 0.192934 NaN NaN NaN NaN NaN NaN NaN NaN -0.013320
2 covsearch_run10 (CL-AGE-exp);(CL-SEX-cat) True 0 0 -1346.550520 14.0 0.16 22.8501 106.4740 0.428178 ... 0.192682 0.000659 0.387669 NaN NaN NaN NaN NaN NaN NaN
covsearch_run11 (CL-AGE-exp);(CL-WT-exp) True 0 0 -1332.175838 15.0 0.18 24.4821 106.0270 0.435683 ... 0.199545 -0.000303 NaN -0.010497 NaN NaN NaN NaN NaN NaN
covsearch_run12 (CL-AGE-exp);(MAT-AGE-exp) True 0 0 -1312.323087 15.0 0.22 24.1635 103.5920 0.433194 ... 0.193747 -0.000956 NaN NaN 0.056724 NaN NaN NaN NaN NaN
covsearch_run13 (CL-AGE-exp);(MAT-SEX-cat) True 0 0 -1316.574711 14.0 0.15 24.0931 101.1120 0.435607 ... 0.197305 -0.006150 NaN NaN NaN 1.520520 NaN NaN NaN NaN
covsearch_run14 (CL-AGE-exp);(MAT-WT-exp) True 0 0 -1317.148448 15.0 0.35 24.1216 101.2180 0.519385 ... 0.194664 -0.005918 NaN NaN NaN NaN -0.020136 NaN NaN NaN
covsearch_run15 (CL-AGE-exp);(VC-AGE-exp) True 0 0 -1312.930153 15.0 0.34 24.3873 103.4580 0.453399 ... 0.198812 0.000334 NaN NaN NaN NaN NaN 0.014954 NaN NaN
covsearch_run16 (CL-AGE-exp);(VC-SEX-cat) True 0 0 -1340.804759 15.0 0.24 23.4279 96.6231 0.442192 ... 0.192753 -0.006503 NaN NaN NaN NaN NaN NaN 0.984897 NaN
covsearch_run17 (CL-AGE-exp);(VC-WT-exp) True 0 0 -1345.828756 14.0 0.17 23.9115 106.9240 0.466270 ... 0.194238 -0.009567 NaN NaN NaN NaN NaN NaN NaN -0.012092
3 covsearch_run18 (CL-AGE-exp);(MAT-AGE-exp);(CL-SEX-cat) True 0 0 -1357.475127 15.0 0.20 22.7337 106.9590 0.409478 ... 0.189160 0.005291 0.396947 NaN 0.063392 NaN NaN NaN NaN NaN
covsearch_run19 (CL-AGE-exp);(MAT-AGE-exp);(CL-WT-exp) True 0 0 -1343.449928 15.0 0.22 24.3781 106.2760 0.415011 ... 0.195766 0.003125 NaN -0.010523 0.057943 NaN NaN NaN NaN NaN
covsearch_run20 (CL-AGE-exp);(MAT-AGE-exp);(MAT-SEX-cat) True 0 0 -1319.001446 15.0 0.20 24.1007 101.5110 0.446059 ... 0.195730 -0.004767 NaN NaN 0.019268 1.348000 NaN NaN NaN NaN
covsearch_run21 (CL-AGE-exp);(MAT-AGE-exp);(MAT-WT-exp) True 0 0 -1317.368568 15.0 0.43 24.1211 100.8490 0.522201 ... 0.195098 -0.006600 NaN NaN -0.008419 NaN -0.022336 NaN NaN NaN
covsearch_run22 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp) True 0 0 -1317.144362 15.0 0.28 24.2833 103.7400 0.430777 ... 0.195843 0.001364 NaN NaN 0.040813 NaN NaN 0.011936 NaN NaN
covsearch_run23 (CL-AGE-exp);(MAT-AGE-exp);(VC-SEX-cat) True 0 0 -1347.754104 15.0 0.35 23.3688 96.9010 0.444906 ... 0.189330 -0.003100 NaN NaN 0.045624 NaN NaN NaN 0.972469 NaN
covsearch_run24 (CL-AGE-exp);(MAT-AGE-exp);(VC-WT-exp) True 0 0 -1347.613149 15.0 0.41 23.9130 106.7450 0.468161 ... 0.193116 -0.008094 NaN NaN 0.023159 NaN NaN NaN NaN -0.011946
4 covsearch_run25 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(CL-SE... True 0 0 -1358.511327 12.0 0.23 22.8124 106.7120 0.407766 ... 0.189540 0.006380 0.385807 NaN 0.057041 NaN NaN 0.005508 NaN NaN
covsearch_run26 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(CL-WT... True 0 0 -1343.632790 13.0 0.22 24.3789 106.2080 0.416694 ... 0.196038 0.002606 NaN -0.010703 0.059912 NaN NaN -0.002344 NaN NaN
covsearch_run27 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1326.735985 12.0 0.22 24.1985 101.6770 0.441794 ... 0.197429 -0.000340 NaN NaN 0.003577 1.495110 NaN 0.013967 NaN NaN
covsearch_run28 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-W... True 0 0 -1335.184102 11.0 0.38 24.2994 94.1796 0.587939 ... 0.204599 -0.002549 NaN NaN -0.070972 NaN -0.034213 0.034044 NaN NaN
covsearch_run29 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(VC-SE... True 0 0 -1351.804150 12.0 0.20 23.4692 97.3027 0.440385 ... 0.190442 -0.000522 NaN NaN 0.033397 NaN NaN 0.009672 0.954734 NaN
covsearch_run30 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(VC-WT... True 0 0 -1350.087164 12.0 0.22 24.0941 106.6170 0.463834 ... 0.195705 -0.005726 NaN NaN 0.013863 NaN NaN 0.008593 NaN -0.010781
5 covsearch_run31 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1391.753375 10.0 0.27 22.6693 100.2570 0.413923 ... 0.192398 0.002984 0.554417 NaN -0.012482 2.367050 NaN 0.017124 NaN NaN
covsearch_run32 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1352.116420 10.0 0.35 24.3618 104.2080 0.427491 ... 0.196915 0.000299 NaN -0.011851 0.020109 1.442940 NaN -0.000315 NaN NaN
covsearch_run33 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1334.177283 12.0 0.31 24.2658 99.5620 0.521096 ... 0.198387 -0.001229 NaN NaN -0.030354 0.491875 -0.022223 0.019316 NaN NaN
covsearch_run34 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1352.404028 11.0 0.27 23.4546 96.9522 0.446657 ... 0.190453 -0.000216 NaN NaN 0.038044 -0.224276 NaN 0.009542 1.009540 NaN
covsearch_run35 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1354.916093 12.0 0.36 24.0732 105.0130 0.454492 ... 0.197082 -0.005988 NaN NaN 0.001885 0.848782 NaN 0.010323 NaN -0.009922
6 covsearch_run36 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1393.565980 9.0 0.31 22.7145 99.9771 0.451302 ... 0.191512 0.002433 0.519989 NaN -0.017435 1.646840 -0.008652 0.018053 NaN NaN
covsearch_run37 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1359.857348 9.0 0.43 24.3318 102.9710 0.477444 ... 0.194217 -0.000649 NaN -0.012179 -0.008419 0.585757 -0.019211 0.001906 NaN NaN
covsearch_run38 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1354.202801 9.0 0.33 23.4944 96.3068 0.502124 ... 0.191075 -0.001918 NaN NaN 0.000169 -0.367142 -0.014021 0.011399 0.989687 NaN
covsearch_run39 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1355.113256 9.0 0.32 24.0790 104.7890 0.462595 ... 0.197129 -0.005915 NaN NaN -0.002221 0.737357 -0.003565 0.010903 NaN -0.009576
7 covsearch_run40 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 1 -1431.648937 7.0 0.36 21.7389 89.8006 0.491811 ... 0.183751 0.001489 0.808497 NaN -0.009685 -0.033267 -0.014331 0.013699 0.986758 NaN
covsearch_run41 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1375.893415 8.0 0.37 23.6677 98.7770 0.477864 ... 0.187999 -0.001004 NaN -0.011109 0.009238 -0.267484 -0.015830 0.000202 0.869150 NaN
covsearch_run42 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1367.985364 8.0 0.35 23.6488 100.0460 0.472418 ... 0.194026 -0.005955 NaN NaN 0.012667 -0.165471 -0.002078 0.009019 0.632058 -0.008418
8 covsearch_run43 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1438.987780 8.0 0.52 21.9834 92.9929 0.474434 ... 0.187663 -0.002481 0.737404 NaN -0.005033 0.094740 -0.005773 0.012366 0.751771 -0.007571
covsearch_run44 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1384.630169 7.0 0.56 24.0359 100.2120 0.478021 ... 0.193344 -0.009365 NaN -0.007109 0.010833 -0.065816 -0.002607 0.002247 0.416489 -0.013879
9 covsearch_run45 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1439.184221 5.0 0.38 22.0706 93.1991 0.474688 ... 0.187822 -0.002901 0.709206 -0.000803 -0.004749 0.096523 -0.005768 0.011567 0.724866 -0.008196
10 covsearch_run46 (MAT-AGE-exp);(VC-AGE-exp);(MAT-SEX-cat);(MAT-... True 0 0 -1438.509630 17.0 1.02 22.0336 92.8083 0.475373 ... 0.187730 NaN 0.736659 -0.000429 -0.005557 0.098655 -0.006160 0.014272 0.746265 -0.007399
covsearch_run47 (CL-AGE-exp);(VC-AGE-exp);(MAT-SEX-cat);(MAT-W... True 0 0 -1439.111781 17.0 0.80 22.0636 93.2525 0.473063 ... 0.187425 -0.002946 0.706752 -0.000822 NaN 0.107862 -0.004566 0.010938 0.727224 -0.008331
covsearch_run48 (CL-AGE-exp);(MAT-AGE-exp);(MAT-SEX-cat);(MAT-... True 0 0 -1435.001163 18.0 0.99 22.0292 93.8033 0.473542 ... 0.184713 -0.006622 0.662625 -0.002097 0.012020 0.118838 -0.002378 NaN 0.666572 -0.010074
covsearch_run49 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-W... True 0 0 -1439.049591 18.0 0.89 22.0651 92.9375 0.483644 ... 0.187724 -0.002919 0.709124 -0.000793 -0.005781 NaN -0.007473 0.011606 0.751144 -0.008031
covsearch_run50 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1438.636456 17.0 0.86 22.0661 93.5310 0.462817 ... 0.187525 -0.003050 0.704774 -0.000804 0.002862 0.222520 NaN 0.010672 0.711725 -0.008833
covsearch_run51 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1414.872375 18.0 0.93 23.0432 102.1300 0.446808 ... 0.192812 -0.005142 0.427285 -0.003711 -0.005160 1.361570 0.000456 0.009445 NaN -0.012851
covsearch_run52 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1431.810457 7.0 0.39 21.7913 90.1894 0.489466 ... 0.183104 0.001493 0.779125 -0.001087 -0.008017 -0.030930 -0.014649 0.012169 0.967468 NaN
covsearch_run53 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1438.987778 17.0 0.76 21.9838 92.9945 0.474441 ... 0.187672 -0.002483 0.737404 NaN -0.005034 0.094706 -0.005771 0.012366 0.751794 -0.007573
covsearch_run54 (CL-AGE-exp);(MAT-AGE-exp);(VC-AGE-exp);(MAT-S... True 0 0 -1384.630168 15.0 0.68 24.0361 100.2120 0.478038 ... 0.193349 -0.009366 NaN -0.007108 0.010829 -0.065876 -0.002607 0.002248 0.416464 -0.013880

55 rows × 23 columns

You can see a summary of different errors and warnings in summary_errors. See pharmpy.tools.summarize_errors() for information on the content of this table.

res.summary_errors
time message
model category error_no
covsearch_run40 WARNING 0 2022-11-16 11:52:05.462 MINIMIZATION SUCCESSFUL\nHOWEVER, PROBLEMS OCC...

Finally, the results object provides the same attributes as provided by SCM

res.steps
reduced_ofv extended_ofv ofv_drop delta_df pvalue goal_pvalue is_backward extended_significant selected model covariate_effects
step parameter covariate extended_state
0 -1292.186761 -1292.186761 0.000000 0 NaN NaN False NaN True mox2 NaN
1 CL AGE * exp -1292.186761 -1302.929085 10.742325 1 1.047127e-03 0.05 False True True covsearch_run1 NaN
SEX * cat -1292.186761 -1346.501817 54.315057 1 1.710000e-13 0.05 False True False covsearch_run2 NaN
WT * exp -1292.186761 -1332.164976 39.978215 1 2.568110e-10 0.05 False True False covsearch_run3 NaN
MAT AGE * exp -1292.186761 -1312.249442 20.062682 1 7.494490e-06 0.05 False True False covsearch_run4 NaN
SEX * cat -1292.186761 -1312.248431 20.061671 1 7.498453e-06 0.05 False True False covsearch_run5 NaN
WT * exp -1292.186761 -1313.243298 21.056538 1 4.459287e-06 0.05 False True False covsearch_run6 NaN
VC AGE * exp -1292.186761 -1312.919450 20.732689 1 5.280678e-06 0.05 False True False covsearch_run7 NaN
SEX * cat -1292.186761 -1335.838386 43.651626 1 3.923500e-11 0.05 False True False covsearch_run8 NaN
WT * exp -1292.186761 -1334.001593 41.814832 1 1.003390e-10 0.05 False True False covsearch_run9 NaN
2 CL SEX * cat -1302.929085 -1346.550520 43.621435 1 3.984500e-11 0.05 False True False covsearch_run10 NaN
WT * exp -1302.929085 -1332.175838 29.246753 1 6.372293e-08 0.05 False True False covsearch_run11 NaN
MAT AGE * exp -1302.929085 -1312.323087 9.394001 1 2.176965e-03 0.05 False True True covsearch_run12 NaN
SEX * cat -1302.929085 -1316.574711 13.645626 1 2.207550e-04 0.05 False True False covsearch_run13 NaN
WT * exp -1302.929085 -1317.148448 14.219363 1 1.626878e-04 0.05 False True False covsearch_run14 NaN
VC AGE * exp -1302.929085 -1312.930153 10.001068 1 1.564495e-03 0.05 False True False covsearch_run15 NaN
SEX * cat -1302.929085 -1340.804759 37.875674 1 7.539960e-10 0.05 False True False covsearch_run16 NaN
WT * exp -1302.929085 -1345.828756 42.899671 1 5.762000e-11 0.05 False True False covsearch_run17 NaN
3 CL SEX * cat -1312.323087 -1357.475127 45.152040 1 1.823200e-11 0.05 False True False covsearch_run18 NaN
WT * exp -1312.323087 -1343.449928 31.126842 1 2.417058e-08 0.05 False True False covsearch_run19 NaN
MAT SEX * cat -1312.323087 -1319.001446 6.678359 1 9.759040e-03 0.05 False True False covsearch_run20 NaN
WT * exp -1312.323087 -1317.368568 5.045482 1 2.469024e-02 0.05 False True False covsearch_run21 NaN
VC AGE * exp -1312.323087 -1317.144362 4.821275 1 2.811054e-02 0.05 False True True covsearch_run22 NaN
SEX * cat -1312.323087 -1347.754104 35.431017 1 2.642452e-09 0.05 False True False covsearch_run23 NaN
WT * exp -1312.323087 -1347.613149 35.290062 1 2.840779e-09 0.05 False True False covsearch_run24 NaN
4 CL SEX * cat -1317.144362 -1358.511327 41.366965 1 1.261720e-10 0.05 False True False covsearch_run25 NaN
WT * exp -1317.144362 -1343.632790 26.488428 1 2.651215e-07 0.05 False True False covsearch_run26 NaN
MAT SEX * cat -1317.144362 -1326.735985 9.591623 1 1.954671e-03 0.05 False True True covsearch_run27 NaN
WT * exp -1317.144362 -1335.184102 18.039740 1 2.163414e-05 0.05 False True False covsearch_run28 NaN
VC SEX * cat -1317.144362 -1351.804150 34.659788 1 3.926594e-09 0.05 False True False covsearch_run29 NaN
WT * exp -1317.144362 -1350.087164 32.942802 1 9.491049e-09 0.05 False True False covsearch_run30 NaN
5 CL SEX * cat -1326.735985 -1391.753375 65.017391 1 7.423998e-16 0.05 False True False covsearch_run31 NaN
WT * exp -1326.735985 -1352.116420 25.380435 1 4.706676e-07 0.05 False True False covsearch_run32 NaN
MAT WT * exp -1326.735985 -1334.177283 7.441298 1 6.374390e-03 0.05 False True True covsearch_run33 NaN
VC SEX * cat -1326.735985 -1352.404028 25.668043 1 4.054865e-07 0.05 False True False covsearch_run34 NaN
WT * exp -1326.735985 -1354.916093 28.180108 1 1.105347e-07 0.05 False True False covsearch_run35 NaN
6 CL SEX * cat -1334.177283 -1393.565980 59.388697 1 1.300000e-14 0.05 False True False covsearch_run36 NaN
WT * exp -1334.177283 -1359.857348 25.680065 1 4.029682e-07 0.05 False True False covsearch_run37 NaN
VC SEX * cat -1334.177283 -1354.202801 20.025518 1 7.641559e-06 0.05 False True True covsearch_run38 NaN
WT * exp -1334.177283 -1355.113256 20.935973 1 4.748923e-06 0.05 False True False covsearch_run39 NaN
7 CL SEX * cat -1354.202801 -1431.648937 77.446136 1 1.363918e-18 0.05 False True False covsearch_run40 NaN
WT * exp -1354.202801 -1375.893415 21.690615 1 3.203548e-06 0.05 False True False covsearch_run41 NaN
VC WT * exp -1354.202801 -1367.985364 13.782563 1 2.052323e-04 0.05 False True True covsearch_run42 NaN
8 CL SEX * cat -1367.985364 -1438.987780 71.002416 1 3.567877e-17 0.05 False True False covsearch_run43 NaN
WT * exp -1367.985364 -1384.630169 16.644805 1 4.507358e-05 0.05 False True True covsearch_run44 NaN
9 CL SEX * cat -1384.630169 -1439.184221 54.554053 1 1.510000e-13 0.05 False True True covsearch_run45 NaN
10 CL AGE * exp -1439.184221 -1438.509630 -0.674591 -1 NaN 0.01 True True False covsearch_run46 NaN
MAT AGE * exp -1439.184221 -1439.111781 -0.072441 -1 NaN 0.01 True True False covsearch_run47 NaN
VC AGE * exp -1439.184221 -1435.001163 -4.183059 -1 NaN 0.01 True True False covsearch_run48 NaN
MAT SEX * cat -1439.184221 -1439.049591 -0.134631 -1 NaN 0.01 True True False covsearch_run49 NaN
WT * exp -1439.184221 -1438.636456 -0.547765 -1 NaN 0.01 True True False covsearch_run50 NaN
VC SEX * cat -1439.184221 -1414.872375 -24.311846 -1 NaN 0.01 True False False covsearch_run51 NaN
WT * exp -1439.184221 -1431.810457 -7.373764 -1 NaN 0.01 True False False covsearch_run52 NaN
CL WT * exp -1439.184221 -1438.987778 -0.196444 -1 NaN 0.01 True True False covsearch_run53 NaN
SEX * cat -1439.184221 -1384.630168 -54.554053 -1 NaN 0.01 True False False covsearch_run54 NaN 
res.ofv_summary
direction reduced_ofv extended_ofv ofv_drop delta_df pvalue goal_pvalue
[0, , , ] Forward -1292.186761 -1292.186761 0.000000 0 NaN NaN
[1, CL, AGE, * exp] Forward -1292.186761 -1302.929085 10.742325 1 1.047127e-03 0.05
[2, MAT, AGE, * exp] Forward -1302.929085 -1312.323087 9.394001 1 2.176965e-03 0.05
[3, VC, AGE, * exp] Forward -1312.323087 -1317.144362 4.821275 1 2.811054e-02 0.05
[4, MAT, SEX, * cat] Forward -1317.144362 -1326.735985 9.591623 1 1.954671e-03 0.05
[5, MAT, WT, * exp] Forward -1326.735985 -1334.177283 7.441298 1 6.374390e-03 0.05
[6, VC, SEX, * cat] Forward -1334.177283 -1354.202801 20.025518 1 7.641559e-06 0.05
[7, VC, WT, * exp] Forward -1354.202801 -1367.985364 13.782563 1 2.052323e-04 0.05
[8, CL, WT, * exp] Forward -1367.985364 -1384.630169 16.644805 1 4.507358e-05 0.05
[9, CL, SEX, * cat] Forward -1384.630169 -1439.184221 54.554053 1 1.510000e-13 0.05
[CL, AGE, * exp] Final included -1439.184221 -1438.509630 -0.674591 -1 NaN 0.01
[MAT, AGE, * exp] Final included -1439.184221 -1439.111781 -0.072441 -1 NaN 0.01
[VC, AGE, * exp] Final included -1439.184221 -1435.001163 -4.183059 -1 NaN 0.01
[MAT, SEX, * cat] Final included -1439.184221 -1439.049591 -0.134631 -1 NaN 0.01
[MAT, WT, * exp] Final included -1439.184221 -1438.636456 -0.547765 -1 NaN 0.01
[VC, SEX, * cat] Final included -1439.184221 -1414.872375 -24.311846 -1 NaN 0.01
[VC, WT, * exp] Final included -1439.184221 -1431.810457 -7.373764 -1 NaN 0.01
[CL, WT, * exp] Final included -1439.184221 -1438.987778 -0.196444 -1 NaN 0.01
[CL, SEX, * cat] Final included -1439.184221 -1384.630168 -54.554053 -1 NaN 0.01 
res.candidate_summary
N_test N_ok N_localmin N_failed StepIncluded
parameter covariate extended_state
CL AGE * exp 1 1 0 0 1.0
SEX * cat 9 9 0 0 9.0
WT * exp 8 8 0 0 8.0
MAT AGE * exp 2 2 0 0 2.0
SEX * cat 4 4 0 0 4.0
WT * exp 5 5 0 0 5.0
VC AGE * exp 3 3 0 0 3.0
SEX * cat 6 6 0 0 6.0
WT * exp 7 7 0 0 7.0