Bootstrap#

Pharmpy can do postprocessing for the PsN bootstrap tool.

\[\]

The Bootstrap postprocessing and results#

Parameter statistics#

The parameter_statistics table contains summary statistics for over the bootstrap runs for the model parameters.

Column	Description
`mean`	Mean over all bootstrap runs
`median`	Median over all bootstrap runs
`bias`	Difference between the mean and the value in the original model
`stderr`	Standard deviation over all bootstrap runs
`RSE`	Standard error divided by the mean

	mean	median	bias	stderr	RSE
POP_CL	0.005890	0.005891	NaN	0.000460	0.078170
POP_V	1.431004	1.422850	NaN	0.073015	0.051024
IVCL	0.149260	0.124740	NaN	0.112645	0.754686
IVV	0.189801	0.185916	NaN	0.045563	0.240056
SIGMA_1_1	0.015470	0.015184	NaN	0.003553	0.229641

The parameter_distribution table gives a numeric overview of the distributions of the bootstrap parameter estimates. For each parameter it contains the lowest and highest values, the median and values at some other selected percentiles. All percentiles are calculated using linear interpolation if it falls between two data points. If the two data points are \(x_0\) and \(x_1\) the percentile would be \(x_0 + (x_1 - x_0) f\) where \(f\) is \([np]\), the fractional part of the number of observations \(n\) multiplied by the percentile \(p\).

	min	0.05%	0.5%	2.5%	5%	median	95%	97.5%	99.5%	99.95%	max
POP_CL	0.004851	0.004857	0.004902	0.005031	0.005149	0.005891	0.006659	0.006846	0.007353	0.007460	0.007472
POP_V	1.277810	1.278405	1.283765	1.309488	1.326434	1.422850	1.547333	1.582182	1.631219	1.644090	1.645520
IVCL	0.000011	0.000097	0.000869	0.006275	0.013558	0.124740	0.353679	0.367954	0.416361	0.433175	0.435043
IVV	0.079940	0.080472	0.085258	0.101225	0.119080	0.185916	0.265224	0.282926	0.290666	0.291792	0.291917
SIGMA_1_1	0.007645	0.007719	0.008382	0.009481	0.009956	0.015184	0.021341	0.022691	0.025584	0.027052	0.027215

The parameter_estimates_histogram give histograms for the distributions of the parameter estimates:

The raw parameter data is available in parameter_estimates

	POP_CL	POP_V	IVCL	IVV	SIGMA_1_1
0	0.006755	1.52531	0.108662	0.258316	0.011408
1	0.005424	1.43599	0.270899	0.156400	0.014023
2	0.006585	1.38641	0.226891	0.162075	0.010971
3	0.005757	1.47090	0.000011	0.193243	0.019133
4	0.006875	1.50111	0.345382	0.176756	0.009135
...	...	...	...	...	...
95	0.005683	1.34027	0.163931	0.159868	0.010816
96	0.005795	1.34379	0.075865	0.194188	0.018041
97	0.006078	1.35172	0.040387	0.214176	0.015683
98	0.005353	1.44898	0.155477	0.258743	0.020128
99	0.006247	1.49069	0.125491	0.219146	0.018971

100 rows × 5 columns

OFV statistics#

Summary statistics for the objective function values of the bootstrap runs can be found in the ofv_statistics table, which has the following rows:

Row	Description
`bootstrap_bootdata_ofv`	OFVs from the bootstrap runs
`original_bootdata_ofv`	Sum of iOFVs from original modelfit of individuals included in each bootstrap run
`bootstrap_origdata_ofv`	OFVs from all dofv runs, i.e. evaluations on original data on boostrap models
`original_origdata_ofv`	OFV of original model
`delta_bootdata`	Difference between original_bootdata_ofv and bootstrap_bootdata_ofv for each model
`delta_origdata`	Difference between bootstrap_origdata_ofv and the OFV of the original model

Note that some of these rows will not be created if the bootstrap was run without the dofv option.

	mean	median	stderr
bootstrap_bootdata_ofv	730.08165	729.644638	44.737749
original_bootdata_ofv	NaN	NaN	NaN
bootstrap_origdata_ofv	NaN	NaN	NaN
delta_bootdata	NaN	NaN	NaN
delta_origdata	NaN	NaN	NaN

The ofv_distribution gives a numeric overview of the OFVs similar to the parameter_distriution described above.

	min	0.05%	0.5%	2.5%	5%	median	95%	97.5%	99.5%	99.95%	max
bootstrap_bootdata_ofv	593.389406	593.972104	599.216391	654.548507	660.573965	729.644638	795.776225	803.946508	816.836314	820.873796	821.322405
original_bootdata_ofv	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
bootstrap_origdata_ofv	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
delta_bootdata	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
delta_origdata	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

A histogram of the bootstrap ofv from ofv_plot:

The dofv_quantiles_plot show distribution of the delta-OFV metrics over the distribution quantiles. They are compared with a chi-square distribution.

The raw ofv data is available in ofvs.

	bootstrap_bootdata_ofv	original_bootdata_ofv	bootstrap_origdata_ofv	delta_bootdata	delta_origdata
0	664.272157	NaN	NaN	NaN	NaN
1	749.459450	NaN	NaN	NaN	NaN
2	749.061103	NaN	NaN	NaN	NaN
3	660.725470	NaN	NaN	NaN	NaN
4	679.585721	NaN	NaN	NaN	NaN
...	...	...	...	...	...
95	702.481949	NaN	NaN	NaN	NaN
96	748.074871	NaN	NaN	NaN	NaN
97	710.203812	NaN	NaN	NaN	NaN
98	787.180418	NaN	NaN	NaN	NaN
99	771.202034	NaN	NaN	NaN	NaN

100 rows × 5 columns

Covariance matrix#

A covariance matrix for the parameters is available in covariance_matrix:

	POP_CL	POP_V	IVCL	IVV	SIGMA_1_1
POP_CL	2.119974e-07	0.000011	-0.000002	0.000001	-3.491440e-07
POP_V	1.096413e-05	0.005331	0.000922	0.001519	3.460998e-05
IVCL	-2.365879e-06	0.000922	0.012689	-0.000981	-1.760432e-04
IVV	1.331894e-06	0.001519	-0.000981	0.002076	7.755294e-05
SIGMA_1_1	-3.491440e-07	0.000035	-0.000176	0.000078	1.262063e-05

Included individuals#

The included_individuals is a list of lists with all individuals that were included in each bootstrap run.

	0	1	2	3	4	5	6	7	8	9	...	49	50	51	52	53	54	55	56	57	58
0	28	8	57	59	56	22	6	30	47	16	...	2	20	23	38	56	1	52	32	11	31
1	22	48	13	7	40	24	10	18	34	32	...	48	15	55	9	12	27	27	35	49	15
2	25	24	43	25	39	23	58	3	10	5	...	10	49	22	50	50	46	4	57	29	55
3	29	16	44	34	10	14	30	31	43	22	...	12	30	8	35	30	17	49	9	14	29
4	27	19	30	36	38	22	9	15	50	23	...	45	40	46	9	43	56	31	42	57	57
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
95	48	1	31	1	41	50	40	59	23	23	...	57	29	53	8	27	13	13	36	17	29
96	31	2	56	43	9	47	31	5	9	52	...	46	44	1	21	15	6	51	8	11	26
97	4	5	53	55	11	27	56	6	2	4	...	12	4	50	58	8	7	42	54	19	29
98	12	52	2	31	23	55	38	4	28	56	...	18	5	14	57	48	40	16	30	12	13
99	58	51	21	9	39	28	22	30	43	18	...	6	32	7	23	56	57	53	38	4	27

100 rows × 59 columns

	0	1	2	3	4	5	6	7	8	9	...	49	50	51	52	53	54	55	56	57	58
0	28	8	57	59	56	22	6	30	47	16	...	2	20	23	38	56	1	52	32	11	31
1	22	48	13	7	40	24	10	18	34	32	...	48	15	55	9	12	27	27	35	49	15
2	25	24	43	25	39	23	58	3	10	5	...	10	49	22	50	50	46	4	57	29	55
3	29	16	44	34	10	14	30	31	43	22	...	12	30	8	35	30	17	49	9	14	29
4	27	19	30	36	38	22	9	15	50	23	...	45	40	46	9	43	56	31	42	57	57
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
95	48	1	31	1	41	50	40	59	23	23	...	57	29	53	8	27	13	13	36	17	29
96	31	2	56	43	9	47	31	5	9	52	...	46	44	1	21	15	6	51	8	11	26
97	4	5	53	55	11	27	56	6	2	4	...	12	4	50	58	8	7	42	54	19	29
98	12	52	2	31	23	55	38	4	28	56	...	18	5	14	57	48	40	16	30	12	13
99	58	51	21	9	39	28	22	30	43	18	...	6	32	7	23	56	57	53	38	4	27

	0	1	2	3	4	5	6	7	8	9	...	49	50	51	52	53	54	55	56	57	58
0	28	8	57	59	56	22	6	30	47	16	...	2	20	23	38	56	1	52	32	11	31
1	22	48	13	7	40	24	10	18	34	32	...	48	15	55	9	12	27	27	35	49	15
2	25	24	43	25	39	23	58	3	10	5	...	10	49	22	50	50	46	4	57	29	55
3	29	16	44	34	10	14	30	31	43	22	...	12	30	8	35	30	17	49	9	14	29
4	27	19	30	36	38	22	9	15	50	23	...	45	40	46	9	43	56	31	42	57	57
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
95	48	1	31	1	41	50	40	59	23	23	...	57	29	53	8	27	13	13	36	17	29
96	31	2	56	43	9	47	31	5	9	52	...	46	44	1	21	15	6	51	8	11	26
97	4	5	53	55	11	27	56	6	2	4	...	12	4	50	58	8	7	42	54	19	29
98	12	52	2	31	23	55	38	4	28	56	...	18	5	14	57	48	40	16	30	12	13
99	58	51	21	9	39	28	22	30	43	18	...	6	32	7	23	56	57	53	38	4	27

	0	1	2	3	4	5	6	7	8	9	...	49	50	51	52	53	54	55	56	57	58
0	28	8	57	59	56	22	6	30	47	16	...	2	20	23	38	56	1	52	32	11	31
1	22	48	13	7	40	24	10	18	34	32	...	48	15	55	9	12	27	27	35	49	15
2	25	24	43	25	39	23	58	3	10	5	...	10	49	22	50	50	46	4	57	29	55
3	29	16	44	34	10	14	30	31	43	22	...	12	30	8	35	30	17	49	9	14	29
4	27	19	30	36	38	22	9	15	50	23	...	45	40	46	9	43	56	31	42	57	57
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
95	48	1	31	1	41	50	40	59	23	23	...	57	29	53	8	27	13	13	36	17	29
96	31	2	56	43	9	47	31	5	9	52	...	46	44	1	21	15	6	51	8	11	26
97	4	5	53	55	11	27	56	6	2	4	...	12	4	50	58	8	7	42	54	19	29
98	12	52	2	31	23	55	38	4	28	56	...	18	5	14	57	48	40	16	30	12	13
99	58	51	21	9	39	28	22	30	43	18	...	6	32	7	23	56	57	53	38	4	27