Dynamic Probability Control Limits for Risk-Adjusted Bernoulli Cumulative Sum Charts

Transcription

1 Dynamic Probabiliy Conrol Limis for Risk-Adjused Bernoulli Cumulaive Sum Chars Xiang Zhang Disseraion submied o he faculy of he Virginia Polyechnic Insiue and Sae Universiy in parial fulfillmen of he requiremens for he degree of Docor of Philosophy In Saisics William H. Woodall, Commiee Chair Anne G. Ryan Driscoll Chrisopher T. Franck Yili Hong December 4 h, 2015 Blacksburg, VA Keywords: saisical process conrol, average run lengh (ARL), false alarm rae, run lengh disribuion, surgical oucome qualiy. Copyrigh 2015, Xiang Zhang

2 Dynamic Probabiliy Conrol Limis for Risk-Adjused Bernoulli Cumulaive Sum Chars Xiang Zhang ABSTRACT The risk-adjused Bernoulli cumulaive sum (CUSUM) char developed by Seiner e al. (2000) is an increasingly popular ool for monioring clinical and surgical performance. In pracice, however, use of a fixed conrol limi for he char leads o quie variable inconrol average run lengh (ARL) performance for paien populaions wih differen risk score disribuions. To overcome his problem, he simulaion-based dynamic probabiliy conrol limis (DPCLs) paien-by-paien for he risk-adjused Bernoulli CUSUM chars is deermined in his sudy. By mainaining he probabiliy of a false alarm a a consan level condiional on no false alarm for previous observaions, he risk-adjused CUSUM chars wih DPCLs have consisen in-conrol performance a he desired level wih approximaely geomerically disribued run lenghs. Simulaion resuls demonsrae ha he proposed mehod does no rely on any informaion or assumpions abou he paiens risk disribuions. The use of DPCLs for risk-adjused Bernoulli CUSUM chars allows each char o be designed for he corresponding paricular sequence of paiens for a surgeon or hospial. The effec of esimaion error on performance of risk-adjused Bernoulli CUSUM char wih DPCLs is also examined. Our simulaion resuls show ha he in-conrol performance of risk-adjused Bernoulli CUSUM char wih DPCLs is affeced by he esimaion error. The mos influenial facors are he specified desired inconrol average run lengh, he Phase I sample size and he overall adverse even rae.

3 However, he effec of esimaion error is uniformly smaller for he risk-adjused Bernoulli CUSUM char wih DPCLs han for he corresponding char wih a consan conrol limi under various realisic scenarios. In addiion, here is a subsanial reducion in he sandard deviaion of he in-conrol run lengh when DPCLs are used. Therefore, use of DPCLs has ye anoher advanage when designing a risk-adjused Bernoulli CUSUM char. These researches are resuls of join work wih Dr. William H. Woodall (Deparmen of Saisics, Virginia Tech). Moreover, DPCLs are adaped o design he risk-adjused CUSUM chars for muliresponses developed by Tang e al. (2015). I is shown ha he in-conrol performance of he chars wih DPCLs can be conrolled for differen paien populaions because hese limis are deermined for each specific sequence of paiens. Thus, he risk-adjused CUSUM char for muliresponses wih DPCLs is more pracical and should be applied o effecively monior surgical performance by hospials and healhcare praciioners. This research is a resul of join work wih Dr. William H. Woodall (Deparmen of Saisics, Virginia Tech) and Mr. Jusin Loda (Deparmen of Saisics, Virginia Tech). iii

4 Dedicaion To my parens and my advisor iv

5 Acknowledgemens Firs and foremos, I would like o sincerely and graefully hank my advisor, Dr. William H. Woodall, for his guidance, suppor, paience, undersanding and encouragemen hroughou he enire period of my graduae sudies a Virginia Tech. He has helped me o become more independen, more opimisic, and mos imporanly, more confiden abou my academic career and has augh me o always have a can-do aiude owards various scenarios. For everyhing you have done for me, Dr. Woodall, I hank you. I would like o hank he remaining members of my graduae commiee, Dr. A Anne G. Ryan Driscoll, Dr. Chrisopher T. Franck and Dr. Yili Hong. Your consideraion, suggesion and assisance are grealy appreciaed. I would also like o express my graiude o all he faculy, suff and graduae sudens in Deparmen of Saisics a Virginia Tech, who hroughou my educaional career have suppored and encouraged me o believe in my abiliies. They have guided me hrough many difficul siuaions, allowing me o reach his accomplishmen. Special hanks o my co-auhors, Wenmeng, Hongyue, Jusin and my friends, Xiaoning, Peng, Ma e al. for heir inpu and conribuions o my papers and researches. Finally, I would like o hank my family, especially my parens, for heir firm belief in me. I would be impossible for me o achieve any goals wihou heir uncondiional love and suppor. Thank you so much, Mom and Dad. I am so graeful o be your son and will keep doing my bes o make you proud. v

6 Table of conens Lis of figures... vii Lis of ables...x Chaper 1: Inroducion...1 Chaper 2: Dynamic probabiliy conrol limis for risk-adjused Bernoulli CUSUM chars...9 Chaper 3: Dynamic probabiliy conrol limis for lower and wo-sided risk-adjused Bernoulli CUSUM chars...39 Chaper 4: The effec of esimaion error on in-conrol performance of risk-adjused Bernoulli CUSUM char wih dynamic probabiliy conrol limis...71 Chaper 5: Dynamic probabiliy conrol limis for risk-adjused CUSUM chars based on muliresponses...91 Chaper 6: Conclusions and fuure work References (No included in Chapers 2 5) vi

7 Lis of figures 1.1 Parsonne score hisograms for five differen paien populaions sudied in Tian e al. (2015) (used wih permission of Tian e al.) The in-conrol ARLs of upper (circled) and lower (sarred) risk-adjused CUSUM chars given varying risk disribuions in Tian e al. (2015) (used wih permission of Tian e al.) (a) Esimaed surgical failure rae p, (b) DPCLs, and (c) = Pr{C,i > h (0.001)} of he firs 50 paiens in he firs sequence from risk disribuion Surgeon 1 (= 0.001) (a) Esimaed surgical failure rae p, (b) DPCLs, and (c) = Pr{C,i > h (0.001)} of he firs 1000 paiens in he firs sequence from risk disribuion Surgeon 1 (= 0.001) DPCLs comparison of wo sequences from wo differen risk disribuions ( High Risk and Low Risk ) (= 0.001) DPCLs where he risk disribuion shifed from Low Risk o High Risk afer he 500 h paien (= 0.001) Comparison of consan conrol limi (dashed line) and DPCLs (solid line) for comparable in-conrol ARLs (= )...31 vii

8 2.6 Comparison of he condiional false alarm raes (FARs) for he DPCLs conrol char (darker line) and he consan limi char (ligher line) shown in Figure 2.5 (= ) DPCLs of he firs 1000 paiens in a sequence from risk disribuion Surgeon 1 (= ) for lower risk-adjused Bernoulli CUSUM char Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion All wih = 0.005, 0.001, , Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion High Risk wih = 0.005, 0.001, , Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion Low Risk wih = 0.005, 0.001, , Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion Surgeon 1 wih = 0.005, 0.001, , Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion Surgeon 6 wih = 0.005, 0.001, , (a) Esimaed surgical failure rae p, (b) DPCLs, and (c) Pr C, i h (0.001) of he firs 1000 paiens in a sequence from risk disribuion Surgeon 1 (= 0.001) for lower risk-adjused Bernoulli CUSUM char An example of DPCLs for a wo-sided risk-adjused Bernoulli CUSUM char...62 viii

9 4.1 Comparison of consan conrol limi (dashed line) and DPCLs (solid line) for comparable in-conrol ARLs (= ) Hisograms of Parsonne scores for he five paien populaions The esimaed in-conrol ARLs of upper risk-adjused CUSUM chars for muliresponses wih consan conrol limis given varying paien populaions (a) DPCLs, and (b) = Pr{C,i > h (0.001)} of he firs 1000 paiens in a sequence from risk disribuion Surgeon DPCLs comparison of wo sequences from wo differen risk disribuions ( High Risk and Low Risk ) DPCLs where he risk disribuion shifed from Low Risk o High Risk afer he 500 h paien Comparison of consan conrol limi (dashed line) and DPCLs (solid line) for comparable in-conrol ARLs (= ) Comparison of he condiional false alarm raes (FARs) for he DPCLs conrol char (darker line) and he ones for he consan limi conrol char (ligher line) shown in Figure 5.6 (= ) ix

10 Lis of ables 1.1 Populaion means and in-conrol ARL comparison in Tian e al. (2015) (used wih permission of Tian e al.) Esimaed in-conrol performance of risk-adjused Bernoulli CUSUM chars wih DPCLs ( = 0.005) Esimaed in-conrol performance of risk-adjused Bernoulli CUSUM chars wih DPCLs ( = 0.001) Esimaed in-conrol performance of risk-adjused Bernoulli CUSUM chars wih DPCLs ( = ) Esimaed in-conrol performance of risk-adjused Bernoulli CUSUM chars wih DPCLs ( = ) Comparison of he esimaed ou-of-conrol ARL performance for he wo chars compared in Secion 4.2 wih differen Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = 0.005) Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = 0.001)...52 x

11 3.3 Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = ) Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = ) Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = and N = 1,000,000) Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = and N = 1,000,000) Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = and N = 1,000,000) Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = and N = 1,000,000) Esimaed in-conrol performance of wo-sided risk-adjused Bernoulli CUSUM chars wih DPCLs Esimaed in-conrol performance for each of he five Phase I sample sizes Esimaed in-conrol performance for each of he hree specified in-conrol ARL values Esimaed in-conrol performance for each of he hree overall adverse even raes...85 xi

12 4.4 Esimaed in-conrol performance for each of hree levels of paien variaion Mean Parsonne scores and esimaed in-conrol ARLs comparison Esimaed in-conrol performance of risk-adjused CUSUM chars for muliresponses wih DPCLs ( = 0.001) xii

13 Chaper 1 Inroducion Healhcare qualiy has been an imporan opic of discussion for many years. Recenly, increasing aenion has been placed upon monioring he qualiy of healhcare performance. The use of saisical process conrol echniques o appropriaely monior he qualiy of healhcare has garnered increasing ineres due o is impac and effeciveness in oher fields such as manufacuring and public healh surveillance. One characerisic of ineres which has received significan aenion is ha of surgical oucome qualiy. The imely deecion of any deerioraion in surgical performance is he key o promp invesigaions of possible causes and o avoid adverse consequences. Moreover, i is quie beneficial for healhcare praciioners and organizaions o obain evidence of improved surgical performance. The cumulaive sum (CUSUM) char is widely used for process monioring in indusrial qualiy conrol applicaions. For monioring a sequence of independen Bernoulli rials, he Bernoulli CUSUM char proposed by Reynolds and Soumbos (1999) is based on he assumpion ha he Bernoulli proporion is consan when he process is sable. These auhors did no allow for varying in-conrol probabiliy values since indusrial iems being produced are ypically assumed o be relaively homogeneous. The CUSUM char designed by Reynolds and Soumbos (1999) is generally no appropriae for monioring surgical performance when he pre-operaive risks vary considerably among paiens. Driven by he need for appropriaely monioring surgical performance in he presence of risk variaions in he underlying paien populaions, he risk- 1

14 adjused Bernoulli CUSUM char was developed by Seiner e al. (2000). Wih his char one adjuss for each paien s pre-operaive risk of surgical failure using a logisic regression model and hen applying he likelihood raio based scoring mehod o obain he monioring saisics. I has been shown ha he risk-adjused CUSUM char is suiable for deecing improvemen or deerioraion in surgical performance when here is a mix of paiens wih varying pre-operaive risks. A general review of risk-adjused charing was provided by Cook e al. (2008). Woodall e al. (2015) gave a horough review on monioring and improving surgical oucome qualiy. A number of praciioners and researchers have applied he risk-adjused Bernoulli CUSUM mehod for monioring clinical oucomes. Sherlaw-Johnson (2005) advised using he signal rule of he risk-adjused CUSUM in he background wih he perhaps more commonly used variable life adjused display (VLAD). This is he approach advocaed by he Clinical Pracice Improvemen Cenre (2008). Sherlaw-Johnson e al. (2005) combined he risk-adjused CUSUM char and he rocke ail char based on he VLAD for monioring by developing a scheme in which CUSUM signals were superimposed ono he rocke ail plos. In anoher applicaion, Harris e al. (2005) applied he risk-adjused CUSUM mehod rerospecively o he analysis of heir medical cener s experience wih rupured abdominal aoric aneurysms (RAAAs) while adjusing for he variabiliy in paiens comorbidiies and hemodynamic insabiliy. As anoher example, Novick e al. (2006) compared risk-adjused and non-riskadjused CUSUM analyses of coronary arery bypass surgery oucomes and found ha he riskadjused CUSUM mehod was advanageous over non-risk-adjused mehods by no incorrecly signaling a deerioraion in performance when preoperaive paien risk was high. As a final example, Moore e al. (2007) used he risk-adjused CUSUM mehod o assess shifs in he performance of an RAAA program over ime. 2

15 Seing appropriae conrol limis o ge desired in-conrol average run lengh (usually denoed as ARL 0 ) is of imporance in he design of any CUSUM procedure. The run lengh is defined in surgical performance monioring as he number of Bernoulli rials (i.e., paiens) observed unil a signal is given by he conrol char indicaing ha a process change has occurred. A relaively large ARL 0 value is desirable when here are no changes in he parameer of ineres, whereas a small ou-of-conrol average run lengh value when he parameer of ineres has acually changed indicaes good char performance. Moreover, geing he same or very close ARL 0 values is he prerequisie for comparing he ou-of-conrol performance of compeing conrol chars. In he cardiac surgery example shown in Seiner e al. (2000), he conrol limis for he proposed CUSUM chars were se a a specified level o give a relaively large ARL 0 value given he paien populaion and he fied logisic regression model used for risk-adjusmen. However, concerns abou he effec of differen risk disribuions on he performance of risk-adjused Bernoulli CUSUM chars have been brough up by several researchers. Seiner e al. (2001) showed ha he in-conrol average run lenghs (ARLs) of risk-adjused CUSUM chars wih he same risk adjusmen model and fixed conrol limis can vary by a facor of 10 for sequences of he highes and lowes risk paiens. Thus, hey suggesed ha he conrol limi of he monioring procedure be adjused if he paien mix changes dramaically. Mos recenly, Tian e al. (2015) examined he effec of varying paien populaion disribuions on he inconrol performance of he risk-adjused Bernoulli CUSUM chars. Figure 1.1 shows he five differen paien populaions examined in his sudy. Here he paien populaions are represened by heir Parsonne score disribuions. The simulaion resuls are shown in Table 1.1 and Figure 1.2. I can be clearly seen ha he in-conrol ARLs of he risk-adjused Bernoulli CUSUM char 3

16 wih consan conrol limis and a given risk-adjusmen model can vary by a facor of wo for differen realisic paien populaions. Figure 1.1. Parsonne score hisograms for five differen paien populaions sudied in Tian e al. (2015) (used wih permission of Tian e al.) 4

17 Table 1.1. Populaion means and in-conrol ARL comparison in Tian e al. (2015) (used wih permission of Tian e al.) Populaion Mean Parsonne Score Upper CUSUM ARL 0 (S.E.) Lower CUSUM ARL 0 (S.E.) All Phase I scores ,400.1 (73.6) 6,069.3 (59.6) Lower 50% of scores ,324.0 (120.1) 11,014.0 (105.5) Higher 50% of scores ,988.6 (50.9) 3,983.7 (39.8) Surgeon 1 scores ,474.1 (63.9) 5,148.4 (50.2) Surgeon 6 scores ,657.8 (94.2) 8,207.2 (78.7) Figure 1.2. The in-conrol ARLs of upper (circled) and lower (sarred) risk-adjused CUSUM chars given varying risk disribuions in Tian e al. (2015) (used wih permission of Tian e al.) Varying paien populaions are very common in applicaions. The paien mixes for differen hospials and differen surgeons can vary considerably and vary over ime. Therefore, use of consan conrol limis for risk-adjused Bernoulli CUSUM chars leads o quie variable in-conrol ARL performance. The char would need o be designed for each surgeon or hospial based on assumpions abou he paien populaion ha may be difficul o jusify. 5

18 To overcome his problem, he mehod o deermine dynamic probabiliy conrol limis (DPCLs) for he risk-adjused Bernoulli CUSUM chars is proposed in Chaper 2. The dynamic mehod of deermining probabiliy conrol limis was proposed by Shen e al. (2013) for an applicaion involving he exponenially weighed moving average (EWMA) char o monior Poisson coun daa wih ime-varying populaion sizes. The concep in non-dynamic applicaions was previously used by Margavio e al. (1995) and Hawkins (2003). By mainaining he condiional false alarm raes given here is no false for previous observaions a a consan level, he risk-adjused Bernoulli CUSUM char wih DPCLs can consisenly deliver a desirable inconrol performance wih approximaely geomerically disribued run lenghs for any sequence of paiens wihou regard o he populaion(s) from which he paiens come. Therefore, he use of DPCLs insead of consan conrol limis for risk-adjused Bernoulli CUSUM chars is more pracical since no assumpions abou paiens risk disribuions are needed. This sudy is a resul of join work wih Dr. William H. Woodall (Deparmen of Saisics, Virginia Tech). Chaper 3 is an exended work of Chaper 2 which applies DPCLs o he lower riskadjused Bernoulli CUSUM chars o monior he improvemen of surgical oucome qualiy. The in-conrol performance of wo-sided risk-adjused Bernoulli CUSUM chars wih DPCLs is also examined in Chaper 3. This sudy is a resul of join work wih Dr. William H. Woodall (Deparmen of Saisics, Virginia Tech). Jones and Seiner (2000) sudied he effec of esimaion error on he performance of riskadjused Bernoulli CUSUM char wih consan conrol limis sysemaically and found ha he effec could be subsanial. Chaper 4 focuses on examining he esimaion error on in-conrol performance of risk-adjused Bernoulli CUSUM char wih DPCLs. In pracice, differen Phase I daa would provide differen parameer esimaes and herefore differen risk-adjusmen models. 6

19 The in-conrol performance of risk-adjused Bernoulli CUSUM char wih DPCLs would also be affeced by his esimaion error, bu i will no be affeced by changes in he paien risk disribuion because he conrol limis are deermined depending on he specific observed sequence of paien risk scores. To make a fair comparison, we use he same daa and seings as in he sudy of Jones and Seiner (2000) excep for he way o deermine he conrol limis. This sudy is a resul of join work wih Dr. William H. Woodall (Deparmen of Saisics, Virginia Tech). Tang e al. (2015) developed he risk-adjused CUSUM char based on muliresponses for monioring a surgical process wih hree or more oucomes. Similar o he risk-adjused Bernoulli CUSUM char, here is a significan effec of varying risk disribuions on he in-conrol performance of risk-adjused CUSUM char for muliresponses when consan conrol limis are applied. In Chaper 5, he DPCLs approach is adaped o design he risk-adjused CUSUM char based on muliresponses and he in-conrol performance of he chars wih DPCLs is examined. This sudy is a resul of join work wih Dr. William H. Woodall (Deparmen of Saisics, Virginia Tech) and Mr. Jusin Loda (Deparmen of Saisics, Virginia Tech). The nex chaper, Chaper 2, is he manuscrip iled Dynamic probabiliy conrol limis for risk-adjused Bernoulli CUSUM chars which has been published in Saisics in Medicine (2015). This is followed by Chaper 3, he manuscrip iled Dynamic probabiliy conrol limis for lower and wo-sided risk-adjused Bernoulli CUSUM chars which has been submied o Journal of Qualiy Technology. Nex, Chaper 4 is he manuscrip iled The effec of esimaion error on in-conrol performance of risk-adjused Bernoulli CUSUM char wih dynamic probabiliy conrol limis which has been submied o The Inernaional Journal of Biosaisics and Chaper 5 is he manuscrip iled Dynamic probabiliy conrol limis for risk-adjused 7

20 CUSUM chars based on muliresponses which has been submied o Journal of he American Saisical Associaion. Finally, he conclusions and some fuure research ideas are discussed in Chaper 6. 8

21 Chaper 2 Dynamic probabiliy conrol limis for risk-adjused Bernoulli CUSUM chars ABSTRACT The risk-adjused Bernoulli cumulaive sum (CUSUM) char developed by Seiner e al. (2000) is an increasingly popular ool for monioring clinical and surgical performance. In pracice, however, use of a fixed conrol limi for he char leads o quie variable in-conrol average run lengh (ARL) performance for paien populaions wih differen risk score disribuions. To overcome his problem, we deermine simulaion-based dynamic probabiliy conrol limis (DPCLs) paien-by-paien for he risk-adjused Bernoulli CUSUM chars. By mainaining he probabiliy of a false alarm a a consan level condiional on no false alarm for previous observaions, our risk-adjused CUSUM chars wih DPCLs have consisen in-conrol performance a he desired level wih approximaely geomerically disribued run lenghs. Our simulaion resuls demonsrae ha our mehod does no rely on any informaion or assumpions abou he paiens risk disribuions. The use of DPCLs for risk-adjused Bernoulli CUSUM chars allows each char o be designed for he corresponding paricular sequence of paiens for a surgeon or hospial. Keywords: average run lengh (ARL); false alarm rae; run lengh disribuion; saisical process conrol; surgical performance. 9

22 1. Inroducion Appropriaely monioring he qualiy of healhcare performance has become increasingly imporan. In paricular, monioring he qualiy of surgical oucome performance has been receiving increasing ineres [1, 2]. I is imporan o deec any deerioraion in surgical performance in a imely manner o avoid adverse consequences. Also, i is quie beneficial o obain evidence of improved surgical performance. Generally, here is considerable variabiliy in prior risks of differen paien populaions. Thus, he sandard Bernoulli cumulaive sum (CUSUM) char which does no allow for heerogeneiy in paiens canno be applied effecively o monior surgical performance. Seiner e al. [3] developed he risk-adjused Bernoulli CUSUM char which involves adjusing for each paien s pre-operaive risk of surgical failure using a logisic regression model and hen applying a likelihood-raio based scoring mehod o obain he monioring saisics. I has been shown o be useful for deecing surgical deerioraion or improvemen in seings where here is a mix of paiens wih varying pre-operaive risks. The risk-adjused Bernoulli CUSUM char has been recommended by several researchers [2, 4-6] due o is advanages of design, performance and effeciveness in reducing he effec of paiens prior risks. In pracice, a number of praciioners have applied his char for various applicaions. Some of hem use he risk-adjused CUSUM chars direcly o assess he performance of cerain kinds of operaions [7-13] or o monior clinical performance of healhcare organizaions [14], while some run a risk-adjused CUSUM char wih he more easily inerpreable variable life adjused display (VLAD) char for monioring clinical oucomes [15-19]. 10

23 However, concerns abou he effec of differen risk disribuions on he performance of risk-adjused Bernoulli CUSUM chars have been brough up by several researchers. Seiner e al. [4] showed ha he in-conrol average run lenghs (ARLs) of risk-adjused CUSUM chars wih he same risk adjusmen model and fixed conrol limis can vary by a facor of 10 for sequences of he highes and lowes risk paiens. Thus, hey suggesed ha he conrol limi of he monioring procedure be adjused if he paien mix changes dramaically. The ARL is he expeced number of paiens unil a conrol char signal is given. Rogers e al. [1] saed ha a design based on prior daa migh no yield an in-conrol ARL accuraely if he risk profile changes over ime. In addiion, Loke and Gan [20] compared various heoreical risk disribuions and found ha he in-conrol ARLs of risk-adjused CUSUM chars were clearly affeced by changes in he risk disribuion. In fac, hey recommended a mehod based on he bea disribuion for monioring he risk disribuion. More recenly, Tian e al. [21] showed ha he in-conrol ARLs of risk-adjused CUSUM chars wih fixed conrol limis and a given riskadjusmen model can vary by a facor of wo for differen realisic paien populaions. Varying paien populaions are very common in applicaions. The paien mixes for differen hospials and differen surgeons can vary considerably and vary over ime. Therefore, use of fixed conrol limis for risk-adjused Bernoulli CUSUM chars leads o quie variable inconrol ARL performance. The char would need o be designed for each surgeon or hospial based on assumpions abou he paien populaion ha may be difficul o jusify. To address his problem, we propose o apply dynamic probabiliy conrol limis (DPCLs) o he risk-adjused Bernoulli CUSUM char for monioring he surgical performance for specific sequences of paiens. The dynamic mehod of deermining probabiliy conrol limis was proposed by Shen e al. [22] for an applicaion involving he exponenially weighed moving 11

24 average (EWMA) char o monior Poisson coun daa wih ime-varying populaion sizes. The concep in non-dynamic applicaions was previously used by Margavio e al. [23] and Hawkins [24]. The idea is o mainain he condiional probabiliy of a false alarm given here is no false alarm for previous observaions a a consan value. By applying he DPCLs o he risk-adjused Bernoulli CUSUM char, one can obain an in-conrol run lengh disribuion which is approximaely a geomeric disribuion wih a desired in-conrol ARL for any sequence of paiens wihou regard o he populaion(s) from which he paiens come. Thus, our mehod overcomes he primary shorcoming of he risk-adjused CUSUM char, herefore making i more pracical for monioring surgical performance since no assumpions abou paiens risk disribuions are needed. I is imporan o noe ha alhough we conrol he condiional false alarm rae o be consan, our mehod can be used o design he char wih any specified sequence of condiional false alarm raes. We noe ha he approach of Gombay e al. [25] is differen from ours in he sense ha hey conrol he overall probabiliy of a false alarm for a specified number of paiens. We insead assume an ongoing monioring scenario. The remainder of his paper is organized as follows. The risk-adjused Bernoulli CUSUM char and he proposed simulaion-based procedure for deermining he DPCLs are inroduced in Secion 2. Nex, he simulaion using he daa se examined by Seiner e al. [3] o validae our mehod is explained in Secion 3. This is followed by resuls demonsraing he desirable inconrol run lengh disribuions obained by applying he proposed DPCLs and a comparison of ou-of-conrol performance wih he use of fixed conrol limis in Secion 4. Finally, we give our conclusions and make some relaed remarks in Secion 5. 12

25 2. Mehod 2.1. Risk-adjused Bernoulli CUSUM char The one-sided abular CUSUM saisics can be wrien as follows, 1 C max 0, C W, = 1, 2, 3, (2.1) where C0 0 and W represens he weigh assigned o he resul for he h individual. To monior a specified change of he parameer of ineres, say from o 0, he opimal choice a for W is he log-likelihood raio W ln f y ; f y ; a 0, where y is he oucome for he h individual. The wo-sided CUSUM char is obained by running wo one-sided chars simulaneously, one wih a and one wih 0 a. The signs of he saisics can be changed 0 for one of he ses of cumulaive sum saisics for ease of ploing he chars one above he oher. Driven by he need o ake ino accoun he pre-operaive risks of an adverse even of ineres which vary considerably among differen paiens in pracice, Seiner e al. [3] developed a monioring approach in which one can adjus for each paien s prior risk of surgical failure. I is hus referred o as he risk-adjused Bernoulli CUSUM char. By denoing he adjused surgical failure rae for paien by p, we obain f y p 1 p 1 y y, where y = 1 if paien experiences he adverse even of ineres, such as deah or a surgical sie infecion wihin a specified ime period following surgery, and y = 0 oherwise. The surgical failure rae for each paien is deermined by assessing he pre-operaive risk of each paien by applying a mehod such as a logisic regression model based on Parsonne 13

26 scores [26]. Since p usually varies considerably from paien o paien, monioring for any change in p is no useful. We le R denoe he odds raio corresponding o failure. Then for paien, Rp : 1 p are he odds of failure and he corresponding probabiliy of failure is Rp 1 p Rp. The risk-adjused CUSUM char is designed o monior for an odds raio change from R o 0 performance, and se o a value R, where a R is usually se o 1 o reflec curren expeced surgical 0 R > a R for deecing performance deerioraion, and o a value 0 R < a R o deec process improvemen. Thus, he risk-adjused CUSUM char weighs can be 0 calculaed as W 1 p R p 1 p R p R 0 log if y 0, 1 p Ra p 0 a log if y 1. 1 p Ra p R0. (2.2) We le C, 1, 2,, represen he CUSUM saisics designed o deec an increase in he odds raio and C, 1, 2,, he CUSUM saisics designed o deec a decrease in he odds raio. Following he convenion we change he signs of he saisics for he lower CUSUM char and do no allow he values o exceed zero. The char signals when C h or C h which indicaes ha here has been eiher deerioraion or improvemen in he surgical performance, respecively. The conrol limis h and h are se o yield a suiably large in-conrol ARL when here are no changes in he odds raio of failure R. 0 However, as discussed in Secion 1, he in-conrol ARLs have been found o vary significanly as he paien risk disribuions change if we apply fixed conrol limis o he risk- 14

27 adjused Bernoulli CUSUM char. Moreover, accurae informaion abou he paien risk disribuions, which deermines he design of he char, is usually unavailable in pracice. In addiion, he risk disribuion may change over ime, as i would end o do for a beginning surgeon who gains more and more experience. More experienced surgeons end o operae on sicker paiens han new surgeons. To solve his design problem, we propose he use of dynamic probabiliy conrol limis for risk-adjused CUSUM chars in he nex secion Dynamic probabiliy conrol limis (DPCLs) The mehod of deermining dynamic probabiliy conrol limis was developed by Shen e al. [22] for he EWMA char for monioring Poisson coun daa wih ime-varying sample sizes. The main idea is o keep he probabiliy of obaining a false alarm consan from sample o sample condiional on no false alarm for he previous observaions. Specifically, for a one-sided riskadjused Bernoulli CUSUM char, he DPCLs he exen possible he following equaions: h,,...,,... saisfy o h1 h2 h k Pr C1 h1 S1, Pr C h Ck hk, k 1,..., 1, S, for 2, 3,, (2.3) where S1, S 2, is he sequence of observed risk scores, such as Parsonne scores, and is he predeermined condiional false alarm rae. For each paien, he observed risk-adjused CUSUM saisic C max 0, C 1 W is calculaed, where W is compued using equaion (2.2) based on he observed y and he value of p. The monioring process signals when C h( ). From equaion (2.3), i follows ha he in-conrol run lengh is approximaely he geomeric 15

28 disribuion wih parameer p h,,...,,... o be close o 1/. h1 h2 h k. Thus, we found he in-conrol ARL wih use of he DPCLs The compuaional procedure o obain he DPCLs is oulined below. For simpliciy, we only explain his procedure for he one-sided risk-adjused Bernoulli CUSUM char for deecing process deerioraion, i.e. R a > R 0. This compuaional algorihm can be applied similarly o he oher side of a wo-sided risk-adjused Bernoulli CUSUM char for also deecing shifs o a value R a < R 0. In general erms, he ieraive algorihm works as follows: we simulae a large number of CUSUM saisics for each paien. For a given paien, we randomly sample a large number of values of he immediaely previous CUSUM saisics from our simulaion which did no resul in an ou-of-conrol signal. We combine hese values wih randomly generaed Bernoulli random variables wih he parameer value based on he curren paien s risk score. We hen choose an upper percenile of hese updaed CUSUM values as he conrol limi for he curren paien. A he beginning, consider he firs paien wih some risk score S 1. Under expeced performance, he esimaed rae p can be calculaed from he risk-adjusmen model. Nex, we 1 generae N random variables Y 1,i which are independen and idenically disribued (i.i.d.) Bernoulli random variables wih probabiliy p 1, i = 1, 2,, N, where N is a sufficienly large number. Then we calculae he risk-adjused CUSUM saisics C 1,1, C 1,2, C 1,3,, C 1,N corresponding o he Y 1,i s, using he equaions (2.1) and (2.2), sor he N CUSUM saisics in ascending order C 1,(1), C 1,(2), C 1,(3),, C 1,(N) and ake he N N1 larges CUSUM saisic C 1,(N ) as he approximaed DPCL h 1 (). Noe ha he CUSUM saisic can only ake 16

29 wo values a = 1 (i.e. he firs paien) due o he binary propery of Bernoulli variables. Noe also ha for he firs few paiens here may no be a conrol limi such ha he approximaed condiional false alarm rae is or below. In hese cases, here is no conrol limi used and no possibiliy of he char signaling. Then, for paien 2 wih risk score S 2, we generae N i.i.d. Bernoulli random values Y 2,i wih probabiliy p, where 2 p is calculaed from our logisic regression model. Again, N 2 corresponding risk-adjused CUSUM saisics C 2,i, i = 1, 2,, N are compued from equaions (2.1) and (2.2) wih randomly chosen values from a vecor of C 1,i values which are less han or equal o h 1 (). Then we sor he N C 2,i values in ascending order and ake he upper percenile C 2,(N ) as he DPCL h 2 (). For he nex paien, we keep he CUSUM saisics which are less han or equal o h 2 () and repea he same procedure as before o obain h 3 (). The process is coninued ieraively o obain oher conrol limis. The algorihmic form of he simulaion procedure o obain he conrol limi for paien ( = 1, 2, 3, ), can be summarized as follows: 1) Generae N Bernoulli random variables Y,i (i = 1, 2,, N) wih in-conrol failure rae p obained from risk-adjusmen model and N CUSUM saisics C, i max 0, C 1, j W, i (j = 1, 2,, N) using he equaions (2.1) and (2.2) accordingly, where 1, jis randomly seleced from a vecor of CUSUM values -1 C C C 1, i such ha C 1, i h 1( ). 2) Sor he N CUSUM saisics in ascending order C,(1), C,(2), C,(3),, C,(N) and ake he N N 1 larges CUSUM saisic C,(N ) as he approximaed DPCL h (). 17

30 3) Calculae C max 0, C 1 W based on he surgical oucome and risk score of paien. If C > h (), an ou-of-conrol signal is issued. Oherwise, go back o sep 1. Due o he discreeness of Bernoulli random variables, for each paien = 1, 2,, Pr C, i h ( ) (denoed by ) is always conrolled o be less han or equal o and Pr C, i h ( ) (denoed by ) o be always greaer han or equal o. As he conrol limi deerminaion procedure progresses, an increasing number of differen CUSUM saisic values will be generaed. Afer a relaively small number of paiens, we will observe more k, i Pr C h ( ) values close o. As shown in Secion 4, he condiional false alarm rae, k however, does no converge o as he number of paiens increases. 18

31 3. Simulaion seings The previous sudy of Tian e al. [21] found ha he in-conrol ARL of he Bernoulli riskadjused CUSUM char wih fixed conrol limis varies considerably for differen paien populaions. Specifically, he in-conrol ARLs decrease as he mean risk score of he paien populaion increases. In our simulaion work, we examined he in-conrol performance of he Bernoulli risk-adjused CUSUM char wih our proposed DPCLs for specific sequences of paiens from differen paien populaions. We used he same daa se of paiens from a seven-year sudy used by Seiner e al. [3]. The 2,218 paiens from he firs wo years were reaed as in-conrol group and used o fi he following logisic regression risk model: logi p X, (2.4) where X is he Parsonne score of paien and p is he probabiliy of deah wihin hiry days following surgery. Since he Parsonne score is he only explanaory variable in his logisic regression model o deermine he probabiliy of deah, we can use he differen Parsonne score disribuions o represen he differen paien populaions. We also use he same crieria as Tian e al. [21] o differeniae he Parsonne score disribuions. We randomly chose several sequences of 20,000 Parsonne scores wih replacemen from each of he following five differen Phase I risk disribuions: 1) All: 2,218 scores for all paiens (Mean = ), 2) High Risk: he highes 50% of he scores (Mean = ), 3) Low Risk: he lowes 50% of he scores (Mean = ), 19

32 4) Surgeon 1: 565 scores for all of surgeon 1 s paiens (Mean = ), 5) Surgeon 6: 474 scores for all of surgeon 6 s paiens (Mean = ). Then for each sequence of scores, we calculaed he DPCLs applying he algorihm explained in Secion 2.2. Here, he condiional false alarm rae was chosen and N, he number of CUSUM saisics simulaed, was se o be a suiably large number. The risk-adjused CUSUM saisics were obained using equaions (2.1) and (2.2) wih R 0 = 1 and R a = 2. We also recorded, i h and Pr C, i h ( ) Pr C ( ) for = 1, 2, as a check for accuracy of he procedure. Afer we obained a sequence of DPCLs for a specific sequence of scores seleced from a paricular risk disribuion, 100,000 conrol chars were simulaed o esimae he in-conrol ARL (ARL 0 ). Also, he 10 h, 25 h, 50 h, 75 h, 90 h perceniles of he run lengh (Q 0.10, Q 0.25, Q 0.50, Q 0.75, Q 0.90 ) and sandard deviaion of he run lengh (SDRL) were esimaed in order o compare wih hose of he geomeric disribuion. 20

33 4. Resuls 4.1. Esimaed in-conrol performance We firs se he condiional false alarm rae o be For each of he five risk disribuions described in he previous secion, en differen sequences of 20,000 scores were randomly chosen and used o consruc he conrol char. Here we used N = 100,000. The simulaion resuls are lised in Table 2.1 for he firs five of he sequences along wih he geomeric disribuion wih p = = for comparison. Theoreically, he in-conrol run lengh of he risk-adjused Bernoulli CUSUM char wih our proposed DPCLs for any sequence of risk scores should follow an approximae geomeric disribuion. We noe from Table 2.1, ha he esimaed ARL 0, SDRL and Q 0.10, Q 0.25, Q 0.50, Q 0.75, Q 0.90 values for each sequence examined are close o he corresponding heoreical values of a geomeric disribuion wih p = The SDRL value is approximaely equal o he ARL 0 value for each sequence, as i would be for he geomeric disribuion. However, we do observe sligh deviaions of he run lengh disribuions from he heoreical geomeric disribuion, which is due o he discree naure of he Bernoulli random variables. The average value of Pr C, i h (0.005) (denoed by ) and he average value of Pr C, i h (0.005) (denoed by ) are also repored for each sequence of scores. We can see ha he values vary from o and he values vary from o Thus he averages are close o he desired condiional false alarm rae of =

34 Table 2.1. Esimaed in-conrol performance of risk-adjused Bernoulli CUSUM chars wih DPCLs ( = 0.005) Risk disribuion All High Risk Low Risk Surgeon 1 Surgeon 6 Sequence index ARL 0 (S.E.) SDRL Q 0.10 Q 0.25 Q 0.50 Q 0.75 Q (0.66) E E (0.67) E E (0.65) E E (0.67) E E (0.67) E E (0.66) E E (0.66) E E (0.65) E E (0.66) E E (0.65) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E (0.66) E E-3 Geomeric / 200 (/) / / Secondly, we considered a lower condiional false alarm rae of = We again randomly chose en differen sequences of 20,000 scores from each of he five risk disribuions and conduced he same simulaion procedures as wih = and N = 100,000. The resuls are summarized in Table 2.2 for he firs five of he sequences and compared wih he geomeric disribuion wih p = = The esimaed ARL 0, SDRL and Q 0.10, Q 0.25, Q 0.50, Q 0.75, Q 0.90 values for each sequence examined are even closer o he heoreical reference values of he 22

35 geomeric disribuion wih p = wih respec o relaive error. The SDRL and ARL 0 values are very close o each oher for each sequence of scores. The average Pr C, i h (0.001) values ) vary from o and he average Pr C, i h (0.001) ( values ( ) vary from o Boh averages are very close o he desired condiional false alarm rae of Table 2.2. Esimaed in-conrol performance of risk-adjused Bernoulli CUSUM chars wih DPCLs ( = 0.001) Risk disribuion All High Risk Low Risk Surgeon 1 Surgeon 6 Sequence index ARL 0 (S.E.) SDRL Q 0.10 Q 0.25 Q 0.50 Q 0.75 Q (3.26) E E (3.15) E E (3.18) E E (3.22) E E (3.20) E E (3.19) E E (3.18) E E (3.18) E E (3.17) E E (3.21) E E (3.19) E E (3.14) E E (3.20) E E (3.19) E E (3.16) E E (3.18) E E (3.21) E E (3.20) E E (3.22) E E (3.21) E E (3.22) E E (3.22) E E (3.18) E E (3.18) E E (3.17) E E-3 Geomeric / 1000 (/) / / 23

36 We also decreased boh of he condiional false alarm raes by a facor of 10 and examined he in-conrol performance wih he proposed DPCLs. For each of = and = , we randomly chose wo sequences of 200,000 scores from each of he 5 risk disribuions. Here, we se N = 1,000,000 and implemened he same simulaion procedures as before for each sequence. The in-conrol performance is summarized in Table 2.3 and Table 2.4, respecively. The resuls show ha our mehod works quie well for even lower desired condiional false alarm raes. Table 2.3. Esimaed in-conrol performance of risk-adjused Bernoulli CUSUM chars wih DPCLs ( = ) Risk disribuion All High Risk Low Risk Surgeon 1 Surgeon 6 Sequence index ARL 0 (S.E.) SDRL Q 0.10 Q 0.25 Q 0.50 Q 0.75 Q (6.33) E E (6.33) E E (6.33) E E (6.35) E E (6.32) E E (6.34) E E (6.31) E E (6.40) E E (6.32) E E (6.34) E E-4 Geomeric / 2000 (/) / / 24

37 Risk disribuion All High Risk Low Risk Surgeon 1 Surgeon 6 Table 2.4. Esimaed in-conrol performance of risk-adjused Bernoulli CUSUM chars wih DPCLs ( = ) Sequence index ARL 0 (S.E.) SDRL Q 0.10 Q 0.25 Q 0.50 Q 0.75 Q (31.66) E E (31.97) E E (32.06) E E (31.81) E E (31.62) E E (31.64) E E (31.84) E E (31.56) E E (31.65) E E (31.69) E E-4 Geomeric / (/) / / Gombay e al. [25] designed some sequenial curailed and risk-adjused chars o conrol he overall false alarm rae over a specific horizon of paiens. Our mehod could be used o achieve he same goal since he in-conrol run lenghs resuled from DPCLs approach are approximaely geomerically disribued for any sequence of paiens. Therefore, he overall false alarm rae for a sequence of T paiens can be approximaed by 1 1 overall T. Then for a specified, one could se he condiional false alarm rae o be overall. For overall T insance, if T = 100 and we desired = 0.01, he condiional false alarm rae could be se as overall Our simulaion resuls showed ha he average overall false alarm rae for he firs T = 100 paiens of all 10 sequences wih = was (S.E. = overall 1.644E -4 ) which is close o

38 To demonsrae he applicaion of he proposed DPCLs, we ploed some resuls of he firs sequence from he risk disribuion Surgeon 1 wih he condiional false alarm rae se as in Figures 2.1 and 2.2. Figure 2.1 gives he resuls for he firs 50 paiens. Figure 2.1(a) shows ha he surgical failure raes corresponding o he 50 paiens risk scores vary considerably. Figure 2.1(b) shows he DPCLs obained using our proposed procedure. There are no conrol limis for he firs wo observaions. The reason is ha he approximaed condiional false alarm rae canno be se o be less han or equal o. Thus, here canno be any signals a he firs wo observaions for his paricular sequence. The solid circles indicae he DPCLs for each of he paiens. We see ha he probabiliy conrol limis vary dynamically from paien o paien. Figure 2.1(c) illusraes he condiional false alarm raes = Pr C, i h (0.001) which are always conrolled o be less han or equal o = Similar resuls are shown in Figure 2.2 which provides he resuls for he firs 1,000 paiens in he same sequence. 26

39 (a) Esimaed surgical failure rae p (b) DPCLs (c) = Pr{C,i > h ()} Figure 2.1. (a) Esimaed surgical failure rae p, (b) DPCLs, and (c) = Pr{C,i > h ()} of he firs 50 paiens in he firs sequence from risk disribuion Surgeon 1 (= 0.001) 27

40 (a) Esimaed surgical failure rae p (b) DPCLs (c) = Pr{C,i > h ()} Figure 2.2. (a) Esimaed surgical failure rae p, (b) DPCLs, and (c) = Pr{C,i > h ()} of he firs 1000 paiens in he firs sequence from risk disribuion Surgeon 1 (= 0.001) 28

41 We also compared he DPCLs of wo sequences from wo risk disribuions ( High Risk and Low Risk ) wih he larges difference in mean Parsonne scores in Figure 2.3. The condiional false alarm rae for boh sequences was se o be The plo shows ha differen risk disribuions can lead o quie differen ses of conrol limis. Using our approach we were able o obain similar in-conrol ARLs ( and ) and average condiional false alarm raes of E-4 and E-4, respecively. Figure 2.3. DPCLs comparison of wo sequences from wo differen risk disribuions ( High Risk and Low Risk ) (= 0.001) If he disribuions of risk scores changes over ime, as i would for a new surgeon ha gains more experience, hen our mehod will resul in appropriae limis. This is illusraed in Figure 2.4 where he populaion shifs from he lower risk paiens o he higher risk paiens afer paien 500. Here was se as and he in-conrol ARL was esimaed o be (S.E. = 3.16), very close o 1/. 29

42 Figure 2.4. DPCLs where he risk disribuion shifed from Low Risk o High Risk afer he 500 h paien (= 0.001) The simulaion resuls show ha our mehod has consisen performance for any paien mix. By using he proposed DPCLs, we are able o obain a desired in-conrol run lengh disribuion close o he geomeric disribuion for any paien sequence from any risk disribuion Comparison of consan conrol limi and DPCLs From he previous sudy of Tian e al. [21], he upper CUSUM char in-conrol ARL for he risk disribuion of all scores from 2,218 paiens is 7,400.1 applying he consan conrol limi h + = 4.5 under he assumpion ha he paien populaion is known. To compare he consan conrol limi wih our proposed DPCLs, we se he false alarm rae a 1/ = o obain he same in-conrol ARL. The resuls are shown in Figure 2.5. The dashed line indicaes he consan conrol limi h + = 4.5, while he solid line racks he DPCLs for a paricular sequence of scores chosen from he risk disribuion. The wo ypes of conrol limis resul in very similar in- 30

43 conrol ARL performance for he risk-adjused Bernoulli CUSUM char. The advanage of our mehod is ha we are able o obain he same in-conrol ARL and mainain he condiional false alarm raes a he specified level wihou requiring any informaion or assumpions abou he paien populaions. Also, wih our approach, i is possible o signal sooner for process changes ha occur near he sar of he monioring process. Wih he DPCLs one could signal a he hird paien whereas wih he consan conrol limi a signal could no be given unil he eighh paien. Figure 2.5 would lead one o believe ha seady-sae ou-of-conrol performance would be quie similar. Figure 2.5. Comparison of consan conrol limi (dashed line) and DPCLs (solid line) for comparable in-conrol ARLs (= ) In addiion, we compared in Figure 2.6 he condiional false alarm raes for he wo chars shown in Figure 2.5. I is quie clear ha he condiional false alarm raes of he risk-adjused CUSUM char wih DPCLs (darker line) were conrolled a a specified level = 1/ = while he ones of he char wih he consan conrol limi (ligher line) varied considerably. 31

44 Figure 2.6. Comparison of he condiional false alarm raes (FARs) for he DPCLs conrol char (darker line) and he consan limi char (ligher line) shown in Figure 2.5 (= ) 4.3. Esimaed ou-of-conrol performance We also examined he ou-of-conrol run lengh performance of he risk-adjused Bernoulli CUSUM char wih DPCLs compared o he performance of he char wih consan conrol limis under he assumpion ha he paien risk disribuion is known. Ou-of-conrol performance comparisons are a lile complicaed, however, by he fac ha he wo ypes of chars are based on conrolling differen aspecs of in-conrol performance. We used he wo chars compared in Secion 4.2 for illusraion and sudied heir ou-ofconrol performance. Since he chars were designed o deec a susained doubling of he odds raio, he shifed probabiliy of he adverse even was se o be R p p R p p p where p is he in-conrol probabiliy obained from a a equaion (3.1) for paien. We assumed he odds raio increased a paien = 1, 50, 100, 150, 32

45 200, 250, 500, 1000 and 100,000 chars for each mehod and each change poin were simulaed o esimae he ou-of-conrol ARLs. Table 2.5 presens he comparison of ou-of-conrol ARLs of he wo ypes of chars. I is clear ha boh chars have comparable ou-of-conrol performance under mos of he scenarios, excep for he earlier change poins where he risk-adjused CUSUM char wih DPCLs could deliver quicker deecion. Our resuls demonsraed he advanage of possibly quicker deecion by applying DPCLs insead of consan limis menioned in Secion 4.2. However, i is worh noing ha he main purpose of applying DPCLs is o obain he desired in-conrol run lengh disribuion and conrol he condiional false alarm raes raher han o improve he deecion abiliy of he char [22]. If he risk disribuion is unknown or misspecified hen he use of a consan conrol limi will lead o an in-conrol ARL differen from he desired value. In his case, i is no meaningful o compare ou-of-conrol performance. Table 2.5. Comparison of he esimaed ou-of-conrol ARL performance for he wo chars compared in Secion 4.2 wih differen Change poin Esimaed ou-of-conrol ARL (S.E.) RA-CUSUM wih DPCLs RA-CUSUM wih h + = (0.44) (0.41) (0.42) (0.42) (0.43) (0.42) (0.43) (0.43) (0.43) (0.42) (0.43) (0.42) (0.43) (0.42) (0.41) (0.40) 33

46 5. Discussion The risk-adjused Bernoulli CUSUM char wih consan conrol limis has become a popular ool for monioring clinical and surgical performance. However, is design requires informaion or assumpions abou paien populaions which are ofen unavailable or inaccurae. Previous sudies showed ha he effec of varying risk disribuions of paiens on he in-conrol performance of risk-adjused CUSUM chars is significan. Thus, seing fixed conrol limis for risk-adjused CUSUM chars in differen applicaions is no suiable in pracice. To overcome his disadvanage, we apply dynamic probabiliy conrol limis o risk-adjused Bernoulli CUSUM chars. By mainaining he probabiliy of a false alarm a a consan level condiional on no false alarm for previous observaions, our chars wih DPCLs give desirably consisen inconrol performance wih approximaely geomerically disribued run lenghs. The simulaion resuls illusrae ha our mehod does no rely on any informaion or assumpions abou he paien populaions or heir risk disribuions. Thus, he risk-adjused Bernoulli CUSUM char wih DPCLs is more pracical and should be applied o appropriaely monior surgical performance by hospials and healhcare praciioners. The mehod is compuaionally inensive, which will require appropriae sofware in order o be implemened. Sofware o updae he DPCLs online for risk-adjused Bernoulli CUSUM chars is being developed for pracical use in applicaions. Less compuaionally inensive algorihms o deermine DPCLs should be sudied as well. For example, a Markov chain model was used o compue he DPCLs for he EWMA chars sudied by Shen e al. [22]. Finally, our approach could be easily adaped o design he risk-adjused CUSUM mehod of Tang e al. [27] where here can be more han wo possible oucomes for each paien. 34

47 References 1. Rogers CA, Reeves BC, Capuo M, Ganesh JS, Bosner RS, Angelini GD. Conrol char mehods for monioring cardiac surgical failure and heir inerpreaion. The Journal of Thoracic and Cardiovascular Surgery 2004; 128(6): Woodall WH, Fogel SL, Seiner SH. The monioring and improvemen of surgical oucome qualiy. To appear in Journal of Qualiy Technology Seiner SH, Cook RJ, Farewell VT, Treasure T. Monioring surgical performance using riskadjused cumulaive sum chars. Biosaisics 2000; 1(4): Seiner SH, Cook RJ, Farewell VT. Risk-adjused monioring of binary surgical oucomes. Medical Decision Making 2001; 21(3): Grigg O, Farewell V. An overview of risk-adjused chars. Journal of he Royal Saisical Sociey Series A 2004; 167(3): Seiner SH. Risk-adjused monioring of oucomes in healh care. Chaper 14 in Saisics in Acion: A Canadian Oulook, edied by J. F. Lawless, Chapman and Hall/CRC, 2014; Beiles CB, Moron AP. Cumulaive sum conrol chars for assessing performance in arerial surgery. ANZ Journal of Surgery 2004; 74(3): Harris JR, Forbes, TL, Seiner SH, Lawlor K, Derose G, Harris KA. Risk-adjused analysis of early moraliy afer rupured abdominal aoric aneurysm repair. Journal of Vascular Surgery 2005; 42(3):

48 9. Novick RJ, Fox SA, Si LW, Forbes TL, Seiner S. Direc comparison of risk-adjused and non risk-adjused CUSUM analyses of coronary arery bypass surgery oucomes. The Journal of Thoracic and Cardiovascular Surgery 2006; 132(2): Moore R, Nuley M, Cina CS, Moamedi M, Faris P, Abuznadah W. Improved survival afer inroducion of an emergency endovascular herapy proocol for rupured abdominal aoric aneurysms. Journal of Vascular Surgery 2007; 45(3): Bole A, Aylin P. Inelligen informaion: A naional sysem for monioring clinical performance. Healh Services Research 2008; 43(1p1): Moron AP, Clemens ACA, Doidge SR, Sackelroh J, Curis M, Whiby M. Surveillance of healhcare-acquired infecions in Queensland, Ausralia: Daa and lessons learned in he firs 5 years. Infecion Conrol and Hospial Epidemiology 2008; 29(8): Collins GS, Jibawi A, McCulloch P. Conrol chars mehods for monioring surgical performance: A case sudy from gasro-oesophageal surgery. European Journal of Surgical Oncology 2011; 37(6): Chen TT, Chung KP, Hu FC, Fan CM, Yang MC. The use of saisical process conrol (Risk-adjused CUSUM, risk-adjused RSPRT and CRAM wih predicion limis) for monioring he oucomes of ou-of-hospial cardiac arres paiens rescued by he EMS sysem. Journal of Evaluaion in Clinical Pracice 2011; 17(1): Sherlaw-Johnson C. A mehod for deecing runs of good and bad clinical oucomes on variable life-adjused display (VLAD) chars. Healh Care Managemen Science 2005; 8(1):

49 16. Sherlaw-Johnson C, Moron A, Robinson MB, Hall A. Real-ime monioring of coronary care moraliy: A comparison and combinaion of wo monioring ools. Inernaional Journal of Cardiology 2005; 100(2): Sherlaw-Johnson C, Wilson P, Gallivan S. The developmen and use of ools for monioring he occurrence of surgical wound infecions. Journal of he Operaional Research Sociey 2007; 58(2): Cook DA, Duke G, Har GK, Pilcher D, Mullany D. Review of he applicaion of riskadjused chars o analyze moraliy oucomes in criical care. Criical Care Resusciaion 2008; 10(3): Clinical Pracice Improvemen Cenre. VLADs for Dummies. Wiley Publishing Ausralia Py Ld: Milon, Queensland, Loke CK, Gan FF. Join monioring scheme for clinical failures and predisposed risks. Qualiy Technology and Quaniaive Managemen 2012; 9(1): Tian WM, Sun HY, Zhang X, Woodall WH. The impac of varying paien populaions on he in-conrol performance of he risk-adjused CUSUM char. To appear in Inernaional Journal for Qualiy in Healh Care Shen X, Tsung F, Zou C, Jiang W. Monioring Poisson coun daa wih probabiliy conrol limis when sample sizes are ime-varying. Naval Research Logisics 2013; 60(8): Margavio TM, Conerly MD, Woodall WH, Drake LG. Alarm raes for qualiy conrol chars. Saisics and Probabiliy Leers 1995; 24(3):

50 24. Hawkins, DM, Qiu P, Kang CW. The changepoin model for saisical process conrol. Journal of Qualiy Technology 2003; 35(4): Gombay E, Hussein AA, Seiner SH. Monioring binary oucomes using risk-adjused chars: a comparaive sudy. Saisics in Medicine 2011; 30(23): Parsonne V, Dean D, Bersein AD. A mehod of uniform sraificaion of risk for evaluaing he resuls of surgery in acquired adul hear disease. Circulaion 1989; 79(6): Tang X, Gan FF, Zhang L. Risk-adjused cumulaive sum charing procedure based on muliresponses. To appear in Journal of he American Saisical Associaion

51 Chaper 3 Dynamic probabiliy conrol limis for lower and wo-sided risk-adjused Bernoulli CUSUM chars ABSTRACT Due o is advanages of design, performance and effeciveness in reducing he effec of paiens prior risks, he risk-adjused Bernoulli cumulaive sum (CUSUM) char developed by Seiner e al. (2000) is widely applied o monior clinical and surgical oucome performance. In pracice, i is beneficial o obain evidence of improved surgical performance using he lower risk-adjused Bernoulli CUSUM chars. However, i had been shown ha he in-conrol performance of he chars wih consan conrol limis varies considerably for differen paien populaions. In our sudy, we apply he dynamic probabiliy conrol limis (DPCLs) developed for he upper riskadjused Bernoulli CUSUM chars by Zhang and Woodall (2015) o he lower and wo-sided chars and examine heir in-conrol performance. The simulaion resuls demonsrae ha he inconrol performance of he lower risk-adjused Bernoulli CUSUM chars wih DPCLs can be conrolled for differen paien populaions because hese limis are deermined for each specific sequence of paiens. In addiion, praciioners could also run upper and lower risk-adjused Bernoulli CUSUM chars wih DPCLs side by side simulaneously and obain desired in-conrol performance for he wo-sided char for any paricular sequence of paiens for a surgeon or hospial. 39

52 Keywords: average run lengh (ARL); false alarm rae; saisical process monioring; surgical oucome qualiy. 40

53 1. Inroducion I is of increasing ineres o monior he qualiy of healhcare performance, wih paricular emphasis on he qualiy of surgical oucome performance. I is imporan o deec any deerioraion of surgical performance in a imely manner in order o avoid adverse consequences. Moreover, o obain evidence of improved surgical performance is cerainly beneficial for healhcare praciioners and organizaions as well. Due o he considerable variabiliy of pre-operaive risks in differen paien populaions, he sandard Bernoulli cumulaive sum (CUSUM) char of Reynolds and Soumbos (1999) is no appropriae for monioring surgical oucome qualiy since i doesn consider he heerogeneiy among paiens. To alleviae his problem, Seiner e al. (2000) and Seiner e al. (2001) developed he risk-adjused Bernoulli CUSUM char ha adjuss for each paien s pre-operaive risk of surgical failure hrough he use of a logisic regression model and a likelihood-raio-based scoring mehod o obain he monioring saisics. The risk-adjused Bernoulli CUSUM char has been shown o be suied for seings where here is a mix of paiens wih various pre-operaive risks and herefore has been recommended by many and applied by a number of praciioners for various applicaions. For example, Grigg and Farewell (2004), Woodall (2006), Cook e al. (2008), Seiner (2014) and Woodall e al. (2015) provided he overviews of risk-adjused monioring. Beiles e al. (2004), Harris e al. (2005), Sherlaw-Johnson (2005), Sherlaw-Johnson e al. (2005), Novick e al. (2006), Moore e al. (2007), Sherlaw-Johnson e al. (2007), Bole and Aylin (2008), Moron e al. (2008), Collins e al. (2011) and Chen e al. (2011) used he riskadjused CUSUM chars o assess or monior clinical oucome performance for various applicaions. However, several researchers have brough up issues abou he effec of differen 41

54 risk disribuions on he performance of risk-adjused Bernoulli CUSUM chars (See Rogers e al. (2004), Seiner e al. (2001), Loke and Gan (2012) and Tian e al. (2015)). The in-conrol average run lenghs (ARLs) of risk-adjused CUSUM chars wih he same risk adjusmen model and consan conrol limis can vary by a facor of en for he highes and lowes risk paien populaions. Therefore, Seiner e al. (2001) recommended ha he conrol limi of he monioring procedure be adjused if he paiens mix changes subsanially. Zhang and Woodall (2015) recenly developed a simulaion-based procedure o deermine he dynamic probabiliy conrol limis (DPCLs) for risk-adjused Bernoulli CUSUM chars based on he mehod of Shen e al. (2013), who considered an applicaion involving he exponenially weighed moving average (EWMA) char o monior Poisson coun daa wih imevarying populaion sizes. The concep in non-dynamic applicaions was previously used by Margavio e al. (1995) and Hawkins e al. (2003). By mainaining he condiional false alarm rae a a consan value given ha here are no false alarms for previous observaions, one can design he risk-adjused CUSUM char wih DPCLs which resul in approximaely geomerically disribued in-conrol run lenghs wih a desired in-conrol ARL for any sequence of paiens. The simulaion resuls of applying DPCLs o upper risk-adjused Bernoulli CUSUM chars showed ha he mehod does no require any assumpions abou he paiens risk disribuion. Therefore, using DPCLs insead of consan conrol limis overcomes he major disadvanage of he risk-adjused Bernoulli CUSUM char and is more pracical o use for monioring surgical oucome qualiy. Moreover, paien populaions can change over ime and are no easy o esimae accuraely. In our sudy, we apply DPCLs o he lower risk-adjused Bernoulli CUSUM chars and examine he in-conrol performance of he lower and he wo-sided chars. The remainder of his 42

55 paper is organized as follows. The procedure o deermine he DPCLs for risk-adjused Bernoulli CUSUM chars and our simulaion seings are explained in Secion 2. This is followed by he simulaion resuls of he lower chars wih DPCLs and some examples for illusraion in Secion 3. Nex, he in-conrol performance of wo-sided risk-adjused Bernoulli CUSUM chars wih DPCLs is examined in Secion 4. Finally, we provide some discussion and conclusions in Secion 5. 43

56 2. Mehods and simulaion seings 2.1. Mehods The upper abular CUSUM saisics ( C 0 ) and lower abular CUSUM saisics ( C 0 ) can be wrien as follows, C max 0, C W C min 0, C W, = 1, 2, 3, (1) where C0 C0 0 and W and W represen he weigh assigned o he resul for he h individual for he upper and lower char, respecively. The risk-adjused CUSUM char developed by Seiner e al. (2000) is designed o monior for an odds raio change from R o 0 where R is usually se o 1 o reflec curren expeced surgical performance, and se o a value 0 R, a Ra R for deecing performance deerioraion in he upper char, and o a value 0 Ra R0 o deec process improvemen in he lower char. For he risk-adjused Bernoulli CUSUM chars, he weighs are calculaed using: W 1 p R p 1 p R p R 0 log if y 0, 1 p Ra p 0 a log if y 1. 1 p Ra p R0 (2) where p is he surgical failure rae for each paien deermined by assessing he paien s pre- operaive risk and applying a mehod such as a logisic regression model based on Parsonne scores (Parsonne e al. (1989)). We have y = 1 if paien experiences he adverse even of 44

57 ineres, such as deah or a surgical sie infecion wihin a specified ime period following surgery, and y = 0 oherwise. The char signals when C h or C h which indicaes ha here has been eiher deerioraion or improvemen in he surgical performance, respecively. The conrol limis h and h are se o yield a suiably large in-conrol ARL value when here are no changes in he odds raio of failure R. 0 Zhang and Woodall (2015) applied dynamic probabiliy conrol limis (DPCLs) o he upper risk-adjused Bernoulli CUSUM chars. The mehod of deermining DPCLs for he lower risk-adjused Bernoulli CUSUM chars is similar o he one for he upper CUSUM chars used o deec process deerioraion. The main idea is o mainain he condiional probabiliy of obaining a false alarm a a consan level from paien o paien given here are no false alarms for he previous paiens. Paricularly, for a lower risk-adjused Bernoulli CUSUM char, he DPCLs h 1 h 2 h k h,,...,,... are designed o saisfy o he exen possible he following equaions: Pr C1 h1 S1, Pr C h Ck hk, k 1,..., 1, S, for 2, 3,, (3) where S1, S 2, is he sequence of observed risk scores, such as Parsonne scores, and is he predeermined condiional false alarm rae. For each paien, we calculae he observed risk- adjused CUSUM saisic C min 0, C 1 W, where W is compued using Equaion (2) based on he observed oucome y and he failure rae p. The char signals when C h ( ). From Equaion (3), we can clearly see ha he in-conrol run lengh is approximaely 45

58 geomerically disribued wih parameer. Thus, i is expeced o find he in-conrol ARL wih use of he DPCLs, h h1 h2 h k,,...,,..., o be close o 1/. The compuaional procedure o obain he DPCLs for lower risk-adjused Bernoulli CUSUM char is oulined below. A he beginning, consider he firs paien wih some risk score S 1. Under expeced performance, he esimaed rae p can be calculaed from he risk- 1 adjusmen model. Nex, we generae N random variables Y 1,i which are independen and idenically disribued (i.i.d.) Bernoulli random variables wih probabiliy p 1, i = 1, 2,, N, where N is a sufficienly large number. Then we calculae he lower risk-adjused CUSUM C, C, C,, C N corresponding o he Y 1,i s, using equaions (1) and (2), sor he saisics 1,1 1,2 1,3 1, N CUSUM saisics in ascending order C1,(1), C1,(2), C1,(3),, C1,( N ) and ake he N N 1 smalles CUSUM saisic 1,( N ) C as he approximaed DPCL h 1. Noe ha he CUSUM saisic can only ake wo values a = 1 (i.e. he firs paien) due o he binary naure of Bernoulli variables. Noe also ha for some paiens here may no be a conrol limi such ha he approximaed condiional false alarm rae is or below. In hese cases, here is no conrol limi used and no possibiliy of he char signaling. Then, for paien 2 wih risk score S 2, we generae N i.i.d. Bernoulli random values Y 2,i wih probabiliy p, where 2 p is calculaed from our logisic regression model. Again, N 2 corresponding lower risk-adjused CUSUM saisics C 2,i, i = 1, 2,, N are compued from Equaions (1) and (2) wih randomly chosen values from a vecor of C 1,i values which are greaer han or equal o h 1. Then we sor he N C 2,i values in ascending order and ake he 46

59 N N 1 smalles CUSUM saisic 2,( N ) C as he DPCL h 2. For he nex paien, we keep he CUSUM saisics which are greaer han or equal o h 2 and repea he simulaion procedure o obain h 3. The procedure is coninued ieraively o obain he following conrol limis. The algorihmic form of he simulaion procedure o obain he conrol limi for paien ( = 1, 2, 3, ) can be summarized as follows: 1) Generae N Bernoulli random variables Y,i (i = 1, 2,, N) wih in-conrol failure rae p obained from risk-adjusmen model and N CUSUM saisics C, i min 0, C 1, j W, i (j = 1, 2,, N) using Equaions (1) and (2) accordingly, where 1, C j is randomly seleced from a vecor of CUSUM values C -1 C 1, i such ha C 1, i h 1( ). 2) Sor he N CUSUM saisics in ascending order C,(1), C,(2), C,(3),, C,( N ) and ake he N N 1 smalles CUSUM saisic,( N ) C as he approximaed DPCL h if, i,( N) Pr C C 0. 3) Calculae C min 0, C 1 W based on he surgical oucome and risk score of paien. If C h ( ), an ou-of-conrol signal is issued. Oherwise, go back o sep 1. Due o he discreeness of Bernoulli random variables, for each paien = 1, 2,, Pr C, i h ( ) (denoed by ) is always conrolled o be less han or equal o. As he conrol limi deerminaion procedure progresses, an increasing number of differen CUSUM saisic values will be generaed for each paien. Afer a number of paiens, we will observe 47

60 more Pr C, i h ( ) values close o. However, as shown in Secion 3, he observed condiional false alarm rae does no converge o as he number of paiens increases. Figure 3.1 shows an example of DPCLs obained for he firs 1000 paiens in a sequence randomly chosen from he paien populaion Surgeon 1 by applying he proposed procedure. Here he condiional false alarm rae was se as We can see ha he probabiliy conrol limis vary dynamically from paien o paien. Noe also ha for some of he paiens here are no conrol limis due o he fac ha C, i C,( N) Pr 0. Figure 3.1. DPCLs of he firs 1000 paiens in a sequence from risk disribuion Surgeon 1 (= ) for lower risk-adjused Bernoulli CUSUM char 2.2. Simulaion seings Tian e al. (2015) found ha he in-conrol ARL of he Bernoulli risk-adjused CUSUM char wih consan conrol limis can vary by a facor of wo for differen realisic paien populaions. The in-conrol ARL increases as he mean risk score of he paien populaion decreases. 48

61 The daa se of paiens used in he sudies of Seiner e al. (2000) and Zhang and Woodall (2015) was based on 6994 surgeries from a UK surgical cener over a 7-year period. In our simulaions for he lower risk-adjused Bernoulli CUSUM chars, we used he same Parsonne score sequences examined by Zhang and Woodall (2015) for he upper CUSUM chars and he same logisic regression model, logi p X, where X is he Parsonne score of paien and p is he probabiliy of deah wihin hiry days following surgery. The model was fied using he firs wo years of daa conaining 2218 paiens records which were reaed as in-conrol group. The Parsonne score sequences were randomly seleced wih replacemen from each of he following five differen Phase I paien populaions represened by differen Parsonne score disribuions defined by Tian e al. (2015): 6) All: 2,218 scores for all paiens (Mean = ), 7) High Risk: he highes 50% of he scores (Mean = ), 8) Low Risk: he lowes 50% of he scores (Mean = ), 9) Surgeon 1: 565 scores for all of surgeon 1 s paiens (Mean = ), 10) Surgeon 6: 474 scores for all of surgeon 6 s paiens (Mean = ). For each sequence of scores, we calculaed he DPCLs applying he algorihm oulined in Secion 2.1. Several condiional false alarm raes were examined. Also, N, he number of CUSUM saisics simulaed, was se o be a suiably large number. The risk-adjused CUSUM saisics were obained using Equaions (1) and (2) wih R 0 = 1 and R a = 0.5. Moreover, he observed condiional false alarm rae Pr C, i h ( ) for each paien and he proporion 49

62 of no conrol limis, P No, for each sequence were recorded as a check for accuracy of he procedure. Afer we obained he DPCLs for he specific sequence of scores, 100,000 conrol chars were simulaed o esimae he in-conrol ARL (ARL 0 ). Also, he 10 h, 25 h, 50 h, 75 h, 90 h perceniles of he run lengh (Q 0.10, Q 0.25, Q 0.50, Q 0.75, and Q 0.90 ) and sandard deviaion of he run lengh (SDRL) were esimaed in order o compare wih hose of he geomeric disribuion. 50

63 3. Resuls Similar o he sudy on upper CUSUM chars wih DPCLs, we firs se he condiional false alarm rae o be and N = 100,000. The simulaion resuls are lised in Table 3.1 for five sequences from each risk disribuion along wih he parameers of he geomeric disribuion wih p = for comparison. Table 3.1. Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = 0.005) Risk disribuion All High Risk Low Risk Surgeon 1 Surgeon 6 Sequence index ARL 0 (S.E.) SDRL Q 0.10 Q 0.25 Q 0.50 Q 0.75 Q (0.98) E (1.07) E (1.00) E (1.00) E (1.00) E (0.80) E (0.82) E (0.81) E (0.79) E (0.80) E (0.81) E (0.83) E (0.84) E (0.82) E (0.81) E (0.93) E (0.89) E (0.92) E (0.90) E (0.92) E (1.07) E (1.08) E (0.98) E (1.05) E (1.03) E Geomeric / 200 (/) / P No 51

64 Secondly, we considered a lower condiional false alarm rae of = wih N = 100,000. The resuls are summarized in Table 3.2 for five sequences from each risk disribuion and compared wih he corresponding parameers of he geomeric disribuion wih p = Table 3.2. Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = 0.001) Risk disribuion All High Risk Low Risk Surgeon 1 Surgeon 6 Sequence index ARL 0 (S.E.) SDRL Q 0.10 Q 0.25 Q 0.50 Q 0.75 Q (3.73) E (3.85) E (3.71) E (3.79) E (3.77) E (3.41) E (3.43) E (3.41) E (3.45) E (3.46) E (4.53) E (4.57) E (4.54) E (4.75) E (4.61) E (3.54) E (3.62) E (3.60) E (3.59) E (3.62) E (4.26) E (4.29) E (4.19) E (4.10) E (4.28) E Geomeric / 1000 (/) / P No 52

65 Moreover, we decreased hese condiional false alarm raes by a facor of 10 and examined he in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs. Here we se N = 1,000,000 and implemened he same simulaion procedures explained in Secion 2.1 for each sequence. The esimaed in-conrol performance is summarized in Table 3.3 and Table 3.4, for = and = , respecively. Table 3.3. Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = ) Risk disribuion All High Risk Low Risk Surgeon 1 Surgeon 6 Sequence index ARL 0 (S.E.) SDRL Q 0.10 Q 0.25 Q 0.50 Q 0.75 Q (6.65) E (6.64) E (6.48) E (6.50) E (6.74) E (6.72) E (6.60) E (6.63) E (6.92) E (6.94) E Geomeric / 2000 (/) / / P No Table 3.4. Esimaed in-conrol performance of lower risk-adjused Bernoulli CUSUM chars wih DPCLs ( = ) Risk disribuion All High Risk Low Risk Surgeon 1 Surgeon 6 Sequence index ARL 0 (S.E.) SDRL Q 0.10 Q 0.25 Q 0.50 Q 0.75 Q (34.46) E (34.74) E (33.05) E (33.39) E (42.23) E (41.82) E (33.86) E (34.27) E (36.74) E (36.72) E Geomeric / (/) / / P No 53

66 From Tables , we can see ha he in-conrol ARL performance for he lower risk-adjused Bernoulli CUSUM chars wih DPCLs is affeced by he average observed condiional false alarm rae and he proporion of no lower conrol limis P No. The run lengh disribuion ges closer o he heoreical geomeric disribuion as ges closer o and P No ges closer o 0. We have a greaer proporion of ime wih no lower limi for he larger values of. For he upper risk-adjused CUSUM char, here will be upper conrol limis for all bu he firs few paiens. Since mos paiens have small probabiliies of moraliy wihin 30 days afer surgery, i is more difficul o deec process improvemen han process deerioraion while mainaining he desired condiional false alarm rae. Having lower risk paiens leads o a greaer number of paiens for which here are no lower conrol limis. We used sequences chosen from he five risk disribuions o examine he observed condiional false alarm raes Pr C, i h ( ) more closely. Here N is se o be 1,000,000 for all scenarios. The observed condiional false alarm raes of he firs 50,000 paiens in he sequences from he five risk disribuions wih = 0.005, 0.001, , are ploed in Figures , respecively. If 0, hen here is no lower conrol limi a paien. We noice ha, in general, he proporion of no conrol limis decreases as he condiional false alarm rae decreases excep for he Low Risk disribuion. The oulying paern may be due o he fac ha N is no large enough for ha scenario. 54

67 Figure 3.2. Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion All wih = 0.005, 0.001, , Figure 3.3. Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion High Risk wih = 0.005, 0.001, ,

68 Figure 3.4. Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion Low Risk wih = 0.005, 0.001, , Figure 3.5. Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion Surgeon 1 wih = 0.005, 0.001, ,

69 Figure 3.6. Condiional false alarm raes of he firs 50,000 paiens in a chosen sequence from risk disribuion Surgeon 6 wih = 0.005, 0.001, , We furher increased N o 1,000,000 for = 0.005, and o 5,000,000 for = , o check wheher he value of N would have some effec on he performance of he DPCLs for he same sequences of paiens. The resuls are summarized in Tables Compared wih he resuls in Tables , i is clear ha larger value of N can lower he proporion of no conrol limis P No and increase he average observed condiional false alarm raes, herefore making he in-conrol run lengh disribuion closer o he geomeric disribuion wih parameer. Despie he deviaion from he heoreical geomeric disribuion, he in-conrol performance of he chars wih DPCLs for differen risk disribuions varies much less han he char wih consan conrol limis. Thus, DPCLs have advanages when designing he lower riskadjused Bernoulli CUSUM char. 57