IIVsearch

The IIVsearch tool is a general tool to decide the best IIV structure given a start model. This includes deciding which IIV to keep and the covariance structure based on a chosen selection criteria.

Running

The IIVsearch tool is available both in Pharmpy/pharmr and from the command line.

To initiate IIVsearch in Python/R:

PythonR

from pharmpy.modeling import read_model
from pharmpy.tools import read_modelfit_results, run_iivsearch

start_model = read_model('path/to/model')
start_model_results = read_model_results('path/to/model')
res = run_iivsearch(algorithm='top_down_exhaustive',
                    model=start_model,
                    results=start_model_results,
                    iiv_strategy='no_add',
                    rank_type='bic',
                    cutoff=None)

This will take an input model model and run the brute force algorithm. IIVs on structural parameters (such as mean absorption time) will not be added to the input model since iiv_strategy is set to be ‘no_add’. The candidate models will be ranked using bic with default cutoff, which for BIC is none.

To run IIVsearch from the command line, the example code is redefined accordingly:

pharmpy run iivsearch path/to/model 'top_down_exhaustive' --iiv_strategy 'no_add' --rank_type 'bic'

Arguments

Mandatory

Argument	Description
algorithm	Algorithm to use (e.g. 'top_down_exhaustive')
model	Input model
results	ModelfitResults of input model

Optional

Argument	Description
iiv_strategy	If/how IIV should be added to start model (default is to not add). See Adding IIV to the start model
rank_type	Which selection criteria to rank models on, e.g. OFV (default is BIC)
linearize	Decide wether or not to linearize model before starting the search. See linearization tool for more. Default value is False.
cutoff	Cutoff for the ranking function, exclude models that are below cutoff (default is none)
keep	List of IIVs to keep, either by parameter name or ETA name. Default is [“CL”]
strictness	strictness criteria for model selection. Default is “minimization_successful or (rounding_errors and sigdigs>= 0.1)”
correlation_algorithm	Specify if an algorithm different from the argument algorithm should be used when searching for the correlation structure for the added IIVs. If not specified, the same algorithm will be used when searching for which IIVs to add as well as for the correlation structure.

Note

In this documentation, “base model” will be used to describe the model which all candidates are based on. Note that if you have set iiv_strategy to anything other than ‘no_add’, this model will be different to the input model. The term “base model” can thus be either the input model or a copy with added IIVs.

Algorithms

Different aspects of the IIV structure can be explored in the tool depending on which algorithm is chosen. The available algorithms can be seen in the table below.

Note

If only algorithm is specified, the same will be applied to correlation_algorithm if possible. If not, please see description below which would be used. We recommend setting both arguments if specific algorithms are wanted.

Algorithm	Description
'top_down_exhaustive'	Removes available IIV in all possible combinations. The covariance structure search will search all possible IIV variance and covariance structure
'bottom_up_stepwise'	Iteratively adds all available IIV, one at a time. After each addition, the best model is selected. The algorithm stops when no better model was found after adding a new ETA. The following covariance structure search uses the same approach as ‘top_down_exhaustive’
'skip'	Set this argument if you are certain to skip either the search for number of etas (algorithm='skip') or to skip the search for the best covariance structure (correlation_algorithm='skip'). However, if algorithm is set to skip, then correlation_algorithm need to be set to a valid value.

Top down exhaustive search

The top_down_exhaustive search combines the top down exhaustive approach for choosing number of etas with the brute force algorithm for the block structure, by first choosing the number of etas then the block structure.

The top_down_exhaustive algorithm for choosing number of etas will create candidate models for all combinations of removed IIVs. It will also create a naive pooled model meaning all the etas are fixed to 0. This can be useful in identifying local minima, since all other candidate models should have a lower OFV than the naive pooled model (which doesn’t have any inter-individual variability).

For the covariance structure search, the top_down_exhaustive algorithm will create candidates with all possible IIV variance and covariance structures from the IIVs in the base model.

Bottom up stepwise search

The bottom_up_stepwise algorithm differ from the top_down_exhaustive as the models are created in iterative steps, each adding a single ETA. The algorithm will create a model with all possible IIVs and in the first step it will remove all but one. This ETA will be on clearance (CL) if possible. If not, the first parameter in alphabetical order will have an ETA. This model is then run and its results are used to update the initial estimates of the model. In the next step, a candidate model is created for each remaining parameter that could have an ETA put on it. All models are run, and the best model is chosen for the next step, updating the initial values once more.

The candidate models are then compared using the specified rank type and if no better model can be found, the algorithm stops.

However, this algorithm is not supported to run for the covariance structure search and thusly it will use the algorithm top_down_exhaustive for this step.

Adding IIV to the start model

The iiv_strategy option determines whether or not IIV on the PK parameters should be added to the input model. The different strategies can be seen here:

Strategy	Description
'no_add'	Input model is kept as base model
'add_diagonal'	Diagonal IIV is added to all structural parameters
'fullblock'	IIV is added to all structural parameters, and all IIVs will be in a full block

Comparing and ranking candidates

The supplied rank_type will be used to compare a set of candidate models and rank them. A cutoff may also be provided if the user does not want to use the default. The following rank functions are available:

Rank type	Description
'ofv'	ΔOFV. Default is to not rank candidates with ΔOFV < cutoff (default 3.84)
'aic'	ΔAIC. Default is to rank all candidates if no cutoff is provided.
'bic'	ΔBIC (iiv). Default is to rank all candidates if no cutoff is provided.

Information about how BIC is calculated can be found in pharmpy.modeling.calculate_bic().

The IIVsearch results

The results object contains various summary tables which can be accessed in the results object, as well as files in .csv/.json format. The name of the selected best model (based on the input selection criteria) is also included.

Consider a iivsearch run:

PythonR

res = run_iivsearch(algorithm='td_brute_force',
                    model=start_model,
                    results=start_model_results,
                    iiv_strategy='no_add',
                    rank_type='bic',
                    cutoff=None)

The summary_tool table contains information such as which feature each model candidate has, the difference to the start model (in this case comparing BIC), and final ranking:

		description	n_params	d_params	dbic	bic	rank	parent_model
step	model
1	iivsearch_run2	[CL]+[VC]	6	-1	4.282024	-1283.597406	1	input
	input	[CL]+[VC]+[MAT]	7	0	0.000000	-1279.315382	2	input
	iivsearch_run1	[CL]+[MAT]	6	-1	-217.691571	-1061.623812	3	input
	iivsearch_run3	[CL]	5	-2	-385.641987	-893.673395	4	input
2	iivsearch_run4	[CL,VC]	7	1	81.971960	-1365.569365	1	iivsearch_run2
	iivsearch_run2	[CL]+[VC]	6	0	0.000000	-1283.597406	2	iivsearch_run2
3	iivsearch_run4	[CL,VC]	7	0	86.253983	-1365.569365	1	iivsearch_run4
	input	[CL]+[VC]+[MAT]	7	0	0.000000	-1279.315382	2	input

To see information about the actual model runs, such as minimization status, estimation time, and parameter estimates, you can look at the summary_models table. The table is generated with pharmpy.tools.summarize_modelfit_results().

		description	minimization_successful	errors_found	warnings_found	ofv	runtime_total	estimation_runtime	POP_CL_estimate	POP_CL_SE	POP_CL_RSE	...	IIV_VC_RSE	IIV_MAT_estimate	IIV_MAT_SE	IIV_MAT_RSE	RUV_PROP_estimate	RUV_PROP_SE	RUV_PROP_RSE	IIV_CL_IIV_VC_estimate	IIV_CL_IIV_VC_SE	IIV_CL_IIV_VC_RSE
step	model
0	input	[CL]+[VC]+[MAT]	True	0	0	-1292.186761	4.0	0.10	24.5328	NaN	NaN	...	NaN	0.00013	NaN	NaN	0.209972	NaN	NaN	NaN	NaN	NaN
1	iivsearch_run1	[CL]+[MAT]	True	0	0	-1070.204731	7.0	0.17	29.0366	NaN	NaN	...	NaN	3.31998	NaN	NaN	0.401593	NaN	NaN	NaN	NaN	NaN
	iivsearch_run2	[CL]+[VC]	True	0	1	-1292.178325	8.0	0.07	24.5333	NaN	NaN	...	NaN	NaN	NaN	NaN	0.209975	NaN	NaN	NaN	NaN	NaN
	iivsearch_run3	[CL]	True	0	0	-897.963854	8.0	0.10	29.1049	NaN	NaN	...	NaN	NaN	NaN	NaN	0.574966	NaN	NaN	NaN	NaN	NaN
2	iivsearch_run4	[CL,VC]	True	0	0	-1378.440744	4.0	0.11	24.9055	NaN	NaN	...	NaN	NaN	NaN	NaN	0.215885	NaN	NaN	0.498114	NaN	NaN

5 rows × 31 columns

Finally, you can see a summary of different errors and warnings in summary_errors. See pharmpy.tools.summarize_errors() for information on the content of this table.

				time	message
step	model	category	error_no
1	iivsearch_run2	WARNING	0	2024-10-22 13:15:02.368	MINIMIZATION SUCCESSFUL\nHOWEVER, PROBLEMS OCCURRED WITH THE MINIMIZATION.
2	iivsearch_run2	WARNING	0	2024-10-22 13:15:02.368	MINIMIZATION SUCCESSFUL\nHOWEVER, PROBLEMS OCCURRED WITH THE MINIMIZATION.