Case deletion diagnostics

Pharmpy currently creates results after a PsN cdd run.

The cdd results

Case results

The case_results table contains the different results and metrics for each case.

	cook_score	jackknife_cook_score	delta_ofv	dofv_influential	covariance_ratio	skipped_individuals
1	0.130159	0.121389	0.015207	False	1.050505	[1]
2	0.348263	0.336159	0.144045	False	0.963884	[2]
3	0.192433	0.181125	0.031030	False	1.058896	[3]
4	0.163095	0.155901	0.026142	False	1.087449	[4]
5	0.422651	0.404915	0.190732	False	0.985403	[5]
6	0.374312	0.343311	0.097063	False	1.029066	[6]
7	0.169869	0.163883	0.026223	False	1.101636	[7]
8	0.263865	0.254943	0.058123	False	1.086424	[8]
9	0.757098	0.673587	0.477910	False	1.017369	[9]
10	0.150161	0.144191	0.023623	False	1.094799	[10]
11	0.361646	0.352901	0.217902	False	0.849310	[11]
12	0.255533	0.215005	0.035375	False	1.121111	[12]
13	0.191848	0.187019	0.038199	False	1.049151	[13]
14	0.376365	0.337975	0.096680	False	1.096456	[14]
15	0.142806	0.137340	0.014709	False	1.070088	[15]
16	0.193393	0.179744	0.031711	False	1.040691	[16]
17	0.131746	0.122842	0.020637	False	1.065881	[17]
18	1.176644	0.932149	1.036787	False	0.606158	[18]
19	0.128197	0.120498	0.019030	False	1.205007	[19]
20	0.141093	0.138101	0.029021	False	1.099191	[20]
21	0.176582	0.153583	0.035291	False	1.143483	[21]
22	0.104396	0.100064	0.015499	False	1.050691	[22]
23	0.552071	0.501458	0.278595	False	1.086007	[23]
24	0.242334	0.217068	0.047627	False	1.156273	[24]
25	0.792099	0.715951	0.798775	False	0.919336	[25]
26	0.135247	0.133493	0.021356	False	1.056714	[26]
27	0.442779	0.424958	0.125763	False	0.999221	[27]
28	0.132903	0.127188	0.010157	False	1.029600	[28]
29	0.081122	0.076790	0.005443	False	1.034017	[29]
30	0.183624	0.168501	0.027707	False	1.108785	[30]
31	0.113959	0.111168	0.012363	False	1.038495	[31]
32	0.532749	0.515747	0.325048	False	0.954054	[32]
33	0.094133	0.091212	0.014747	False	1.053618	[33]
34	0.447798	0.406737	0.194457	False	1.096334	[34]
35	0.364978	0.352973	0.124295	False	0.943556	[35]
36	0.270838	0.252922	0.063675	False	1.125423	[36]
37	0.129553	0.121963	0.014043	False	1.086167	[37]
38	0.254914	0.243250	0.064025	False	1.145822	[38]
39	0.207529	0.194633	0.046887	False	1.113939	[39]
40	0.239182	0.218112	0.044252	False	1.114399	[40]
41	0.181986	0.173826	0.020285	False	1.057664	[41]
42	0.604112	0.584208	0.604211	False	0.788608	[42]
43	0.407717	0.369648	0.154161	False	1.063991	[43]
44	0.215722	0.196744	0.043742	False	1.115559	[44]
45	0.214167	0.200091	0.036238	False	1.160549	[45]
46	0.094284	0.089002	0.007506	False	1.034685	[46]
47	0.074174	0.072365	0.007535	False	1.035618	[47]
48	0.743323	0.717468	0.654313	False	0.714976	[48]
49	0.153971	0.147360	0.014502	False	1.092664	[49]
50	0.284706	0.278460	0.072960	False	1.089158	[50]
51	0.331587	0.309374	0.116933	False	1.054565	[51]
52	0.447965	0.421612	0.129593	False	1.002555	[52]
53	0.184872	0.175196	0.029896	False	1.032356	[53]
54	0.430382	0.415446	0.141657	False	1.003056	[54]
55	0.409307	0.369197	0.189729	False	0.936719	[55]
56	0.084764	0.079718	0.007416	False	1.122771	[56]
57	0.107600	0.103145	0.013947	False	1.078285	[57]
58	0.154672	0.150761	0.028236	False	1.096841	[58]
59	0.155632	0.150461	0.034848	False	1.068821	[59]

Cook score

The Cook score for each case is calculated as:

\[\sqrt{(P_i - P_{orig})^T \operatorname{cov}(P_{orig})^{-1} (P_i - P_{orig})}\]

Where \(P_i\) is the estimated parameter vector for case \(i\), \(P_{orig}\) is the estimated parameter vector for the original model and \(\operatorname{cov}(P_{orig})\) is the covariance matrix of the estimated parameters.

Jackknife cookscore

This is the same as the Cook score above, but instead using the Jackknife covariance matrix.

\[\sqrt{(P_i - P_{orig})^T \operatorname{cov}^{\operatorname{jackknife}}(P_{orig})^{-1} (P_i - P_{orig})}\]

where

\[\operatorname{cov}_{j,k}^{\operatorname{jackknife}} = \frac{N - 1}{N}\sum_{i=1}^N(p_{i,j} - \overline{p}_j)(p_{i,k} - \overline{p}_k)\]

is the jackknife estimate of the covariance between \(p_{orig,j}\) and \(p_{orig,k}\) which is used to calculate the full jackknife covariance matrix.

\[\overline{p}_j = \frac{1}{N}\sum_{i=1}^N p_{i,j}\]

is the mean of parameter \(p_{i,j}\) over all case deleted datasets. \(j\) being the index in the parameter vector and \(i\) being the case index.

Covariance ratio

The covariance ratio for each case is calculated as:

\[\sqrt{\frac{\operatorname{det}({\operatorname{cov}(P_i))}}{\operatorname{det}(\operatorname{cov}(P_{orig}))}}\]

Delta OFV

For the delta OFV to be calculated the cases must correspond to individuals. Then it is calculated as

\[dOFV = OFV_{all} - iOFV_{k} - OFV_{k}\]

where \(OFV_{all}\) is the OFV of the full run with all individuals included, \(iOFV_k\) is the individual OFV of the k:th individual in the full run and \(OFV_k\) is the OFV of the run with the k:th individual removed. [dOFV]

Skipped individuals

A list of the individuals that were skipped for each case.

Case column

The Name of the case column in the dataset can be found in case_column.

PythonR

res.case_column

'ID'

References

[dOFV]

Rikard Nordgren, Sebastian Ueckert, Svetlana Freiberga and Mats O. Karlsson, “Faster methods for case deletion diagnostics: dOFV and linearized dOFV”, PAGE 27 (2018) Abstr 8683 https://www.page-meeting.org/?abstract=8683