Estmethod

The Estmethod tool is a general tool to compare estimation methods and/or solvers for a given model.

Running

The Estmethod tool is available both in Pharmpy/pharmr and from the command line.

To initiate Estmethod in Python/R:

PythonR

from pharmpy.modeling import read_model
from pharmpy.tools import read_modelfit_results, run_estmethod

start_model = read_model('path/to/model')
start_model_results = read_modelfit_results('path/to/model')
res = run_estmethod(algorithm='exhaustive',
                    model=start_model,
                    results=start_model_results,
                    methods='all',
                    solvers=['LSODA', 'LSODI'])

This will take an input model start_model. The tool will use the ‘exhaustive’ algorithm and try all combinations between all available methods and solvers LSODA and LSODI.

To run Estmethod from the command line, the example code is redefined accordingly:

pharmpy run estmethod path/to/model 'exhaustive' --methods 'all' --solvers 'LSODA LSODI'

Arguments

For a more detailed description of each argument, see their respective chapter on this page.

Mandatory

Argument	Description
algorithm	Algorithm to use (e.g. 'reduced')
model	Start model
results	ModelfitResults of the start model

Optional

Argument	Description
methods	List of estimation methods to test (e.g. ['IMP'])
solvers	List of solvers to test (e.g. ['LSODA'])
parameter_uncertainty_methods	List of Parameter uncertainty methods to test.

Algorithms

There are a few ways Estmethod can test the different solvers/methods. The available algorithms can be seen in the table below.

Algorithm	Description
'exhaustive'	All combinations are tested (cartesian product)
'exhaustive_with_update'	All combinations are tested, but additionally creates candidates updated from FOCE
'exhaustive_only_eval'	All combinations are tested, but only does evaluation

Exhaustive

The exhaustive algorithm works by creating all combinations of methods and solvers. The candidates will have one estimation step and an evaluation step with IMP as method to be able to compare the candidates.

The following table contains the setting for the estimation step:

Setting	Value
interaction	True
maximum_evaluations	9999
auto	True
keep_every_nth_iter	10

Settings for evaluation step is the same as for estimation step, with the following additions:

Setting	Value
method	IMP
isample	100000
niter	10

Exhaustive (with update)

The exhaustive_with_update algorithm is similar to the exhaustive algorithm, but in addition to the candidate models that the exhaustive algorithm create, it will also create a set of candidates that will use the final estimates of a candidate with FOCE as the initial estimates.

Settings are the same as for exhaustive.

Exhaustive (only evaluation)

The exhaustive_only_eval algorithm has the same algorithm as exhaustive, but instead of estimating the candidate models it only evaluates.

Settings are the same as for exhaustive evaluation step, where the method is the method being examined.

Estimation methods

For a list of supported methods, see pharmpy.model.EstimationStep.supported_methods().

To test FOCE with LAPLACE, simply specify LAPLACE as input argument in the tool.

Solvers

Supported solvers are: CVODES, DGEAR, DVERK, IDA, LSODA and LSODI.

Parameter uncertainty methods

Method to use when estimating parameter uncertainty. For a list of supported parameter uncertainty methods, see here.

The Estmethod results

The results object contains various summary tables which can be accessed in the results object, as well as files in .csv/.json format.

Consider a Estmethod run with the exhaustive algorithm and testing FO and LSODA:

PythonR

res = run_estmethod(algorithm='exhaustive',
                    model=start_model,
                    results=start_model_results,
                    methods=['FO', 'IMP'])

The summary_tool table contains information such as which feature each model candidate has, the OFV, estimation runtime, and parent model:

	description	ofv	runtime_estimation	parent_model
model
estmethod_run2	IMP,IMP	-2422.465718	81.93	mox2
estmethod_run1	FO,IMP	-2271.769243	10.46	mox2

To see information about the actual model runs, such as minimization status, estimation time, and parameter estimates, you can look at the summary_models table. The table is generated with pharmpy.tools.summarize_modelfit_results().

		description	run_type	minimization_successful	errors_found	warnings_found	ofv	runtime_total	estimation_runtime	POP_CL_estimate	POP_VC_estimate	POP_MAT_estimate	IIV_CL_estimate	IIV_VC_estimate	IIV_MAT_estimate	RUV_PROP_estimate
model	step
mox2	1		estimation	True	0	0	-1292.186761	4.0	0.10	24.5328	104.2300	0.433676	0.481858	0.593654	0.000130	0.209972
estmethod_run1	1	FO,IMP	estimation	True	0	1	-1299.696027	75.0	10.46	24.2702	105.1840	0.439589	0.483291	0.598179	0.322572	0.197647
	2	FO,IMP	evaluation	True	0	1	-2271.769243	75.0	59.32	24.2702	105.1840	0.439589	0.483291	0.598179	0.322572	0.197647
estmethod_run2	1	IMP,IMP	estimation	True	0	0	-2420.117263	145.0	81.93	30.3241	81.6578	0.901289	0.419533	0.408292	0.865979	0.105366
	2	IMP,IMP	evaluation	True	0	0	-2422.465718	145.0	57.43	30.3241	81.6578	0.901289	0.419533	0.408292	0.865979	0.105366

You can also see a summary of the settings that were used:

		method	interaction	cov	evaluation	maximum_evaluations	laplace	isample	niter	auto	keep_every_nth_iter
mox2	1	FO	False	False	False	NaN	False	NaN	NaN	NaN	NaN
estmethod_run1	1	FO	True	False	False	9999.0	False	NaN	NaN	True	10.0
	2	IMP	True	False	True	9999.0	False	10000.0	10.0	NaN	10.0
estmethod_run2	1	IMP	True	False	False	9999.0	False	NaN	NaN	True	10.0
	2	IMP	True	False	True	9999.0	False	10000.0	10.0	NaN	10.0

Finally, you can see a summary of different errors and warnings in summary_errors. See pharmpy.tools.summarize_errors() for information on the content of this table.

			time	message
model	category	error_no
estmethod_run1	WARNING	0	2022-12-21 13:40:40.787	MINIMIZATION SUCCESSFUL\nHOWEVER, PROBLEMS OCCURRED WITH THE MINIMIZATION.