Medical Statistics MATH Notes. (Part II)

Transcription

1 Medcal Statstcs MATH Notes (Part II) MATH Part

2 7. The Analyss of quvalence and Non-Inferorty Trals MATH Part

3 7.1 quvalence Trals Usually, the am of a clncal tral s to test whether a new treatment s better than the exstng standard treatment or a placebo. Such trals are sometmes called Superorty Trals as ther purpose s to test whether one treatment s superor to another. Some trals are desgned to establsh that two treatments are equally effectve. A new drug may have fewer sde-effects, be cheaper, or be more convenent than the current standard treatment. In such crcumstances one may wsh to establsh that the new treatment has the same ablty to treat the condton. For example some pan relef medcatons have sde effects such as causng gastrc bleedng or ulceraton. A replacement therapy mght have fewer sde effects, but one may want to check that the two drugs have the same ablty to releve pan. Studes testng ths type of hypothess are called quvalence Trals. An equvalence tral desgn may be relevant to testng non-drug nterventons. For example a tral to test whether say nurses were as effectve as a general practtoner at delverng a partcular type of care mght wsh to demonstrate that outcomes are the same. MATH Part

4 Hypotheses for quvalence Trals In a superorty tral the hypotheses used for testng for a treatment effect are H : 0 0 vs. H : 0 1. It s sometme suggested that falure to reject H : 0 0 mean that treatments are the same. Ths s ncorrect as a small study that s underpowered would lead to ths concluson by default, whereas a larger tral mght detect a small dfference. The ssue can be summed up by the statement An Absence of vdence s not vdence of Absence. Use of a statstcal test wth the hypotheses above s therefore napproprate for demonstratng equvalence. Ths suggests the hypothess for equvalence trals could be H : 0 vs. H : Unfortunately, t s never possble to show that two treatments are dentcal. Instead, one tests whether the treatment effect falls n range, say,, called the range of equvalence. Ths should be defned by consderng what range could be consdered clncally equvalent. The hypotheses are now H : 0 vs. H : 1 Rather than usng formal sgnfcance testng, statstcal analyss of equvalence trals s often based on the confdence nterval of the dfference between treatments. quvalence s establshed by demonstratng that the confdence nterval of the dfference between treatment les n the range of equvalence,. MATH Part

5 7. Analyss of quvalence Trals for a Contnuous Outcome Measures Notaton Suppose n T and nc patents have been randomly allocated to groups T and C and suppose outcome measure Y s contnuous and normally dstrbuted wth a mean T for the new treatment and mean C for the control so that the treatment effect T C. Let yt and y are the sample means and ˆ yt yc. As prevously C S S y y s ˆC T C where 1 n 1 n and the pooled sample standard devaton s s estmated by s T 1 1 C n s n s T T C C n T n C wth s T and s C beng the sample standard devatons for the two treatment groups. t s the value of the t-dstrbuton wth n n degrees of freedom havng a cumulatve probablty T C equal 1. Rejecton of the null hypothess that H : 0 vs. H : 1 the confdence nterval ˆ t S ˆ, ˆ t S ˆ 1, when s wthn the nterval,, has a Type I error less than. MATH Part

6 Proof To determne the Type 1 error we needs to estmate Pr Reject H 0 under H 0. Ths probablty depends on the value of. Snce H0 s a range of values of, PrReject H 0 wll take a range of values also. t S t S ˆ Pr Reject H ˆ ˆ ˆ ˆ 0 Pr,, The dstrbuton of ˆ S has a non-central t-dstrbuton, f 0. To smplfy the proof we wll assume the varance s known, and equal to say, so ˆ has a dstrbuton t wth z. Hence If N,, so that we can replace z z ˆ z ˆ z ˆ z ˆ z Pr Reject H ˆ ˆ 0 Pr,, z = Pr Pr, ˆ z ˆ z s the null set, hence Pr Reject H 0 If 0., z MATH Part z z Pr Reject H ˆ ˆ 0 Pr Pr z z The next step s to fnd that maxmzes Pr Reject H0 under H 0. Dfferentaton wth respect to gves d 1 z 1 z PrReject H0 d where s the densty N 0,1.

7 Snce z z Hence, t follows that z z. d 1 z 1 z PrReject H0 d z z whch s zero when. Snce, t follows that 0 z Snce PrReject H0tend to zero as tends to, 0 must be maxmum. Hence PrReject H0s monotone ncreasng for < 0 and monotone decreasng for > 0. Maxmum of the Type 1 error are therefore the boundary values When, and. z z PrReject H0 z z Smlarly, when, PrReject H. Therefore under the null hypothess H 0 : requred. 0 z, PrReject H 0, as MATH Part

8 x 7.1. An equvalence tral s carred out to test the pan relef of a new medcaton thought to have fewer sde effects than the current standard treatment. It was felt that mean pan score should not dffer by more than 5 to demonstrate equvalent pan relef wth hgher pan scores representng greater pan. Ffty subjects were randomzed to receve each treatment. Mean pan for the standard treatment s 45.1 (s.d.=0.6) and 46.3 (s.d.=19.4) for the new treatment. Test the null hypothess of non-equvalence usng a 5% sgnfcance level. MATH Part

9 By default most computer packages gve a 95% confdence ntervals, but they can gve confdence ntervals wth dfferent levels of coverage f specfed. Fgure 7.1 Stata output for a two sample t-test () 95% confdence nterval Two-sample t test wth equal varances Obs Mean Std. rr. Std. Dev. [95% Conf. Interval] x y dff dff = mean(x) - mean(y) t = Ho: dff = 0 degrees of freedom = 98 Ha: dff < 0 Ha: dff!= 0 Ha: dff > 0 Pr(T < t) = Pr( T > t ) = Pr(T > t) = () 90% confdence nterval Two-sample t test wth equal varances Obs Mean Std. rr. Std. Dev. [90% Conf. Interval] x y dff dff = mean(x) - mean(y) t = Ho: dff = 0 degrees of freedom = 98 Ha: dff < 0 Ha: dff!= 0 Ha: dff > 0 Pr(T < t) = Pr( T > t ) = Pr(T > t) = MATH Part

10 7.3 Sample Sze for quvalence Trals Consder a contnuous and normally dstrbuted outcome measure Y wth means T and C for the new and control treatment, and suppose the range of equvalence s,. Assumng 0 under the alternatve hypothess H 1 :, the sample sze requred to reject H : 0 usng a 1 s n z z / 1 equal sze groups. confdence nterval wth power per treatment group assumng two Ths formula assumes that sample sze s suffcent such that the normal approxmaton for the t-dstrbuton s vald. The power s PrReject H 0 under the alternatve hypothess H 1. From the dervaton above Pr Reject H 0 z z Snce 0 s assumed for the alternatve hypothess Power z z Snce x 1 x Hence 1, t follows that the second term z 1 z z MATH Part

11 Rearrangement gves 1 z. 1 Snce 1 z, t follows that z Hence z z z. Assumng equal sample sze nt nc n, then. n n. Therefore z z Squarng and rearrangement gves n z z / as requred x 7. Pan relef tral contnued. From the data above we have an estmate of 0. Takng the range of equvalence as (-5,5) that s =5, estmate the sample sze per arm requred to have 80% power to reject the null hypothess of non-equvalence usng a 90% confdence nterval (.e. 5% level test) assumng two equal sze groups. MATH Part

12 7.4 Non-Inferorty Trals In the tral comparng conservatve treatment wth suturng for small laceratons of the hand (Crtcal Apprasal ), the objectve was to establsh that conservatve management gave as good aesthetc outcome after 3 months. If conservatve treatment gave a better outcome, one would not be concerned and would want to reject the null hypothess as the man concern s check that conservatve treatment dd no worse. If one s only concerned to demonstrate that a new treatment s as good or better, rather than equvalent to an exstng treatment, only one bound s needed. Ths desgn s referred to as non-nferorty trals. Analyss of a non-nferorty tral can be based on a sngle-sded confdence nterval. Note a (1-) sngle-sded confdence nterval s defne by one or other lmts of the usual two sded confdence nterval, but wth coverage (1-). So the upper and lower 95% sngle-sded confdence ntervals are the upper and lower lmts of a 90% confdence nterval. The hypotheses for a non-nferorty tral are therefore H : 0 N vs. H : 1 N f >0 represents beneft to the patent or H : 0 N vs. H : 1 N f < 0 represents beneft. MATH Part

13 Suppose Y s contnuous and normally dstrbuted outcome measure wth a mean T for the new treatment and mean C for the control treatment so the treatment effect T C. Suppose also that yt and y C are the sample means and ˆ yt yc. () (Hgher score for a better outcome) Rejecton of the null hypothess H : 0 N vs. H : 1 N f the 1 %snglesded confdence nterval, gven by ˆ t S ˆ than or equal to N wll have a Type 1 error α., s greater () (Lower score for a better outcome) Rejecton of the null hypothess H : 0 N vs. H : 1 N, f the 1 % snglesded confdence nterval, gven by ˆ t S ˆ, s less than or equal to N wll have a Type 1 error α. () Hgher score for a better outcome Assumng a normal approxmaton to the t-dstrbuton and a known standard devaton,, the 1 % sngle sded lower confdence nterval for ˆ s gven by ˆ. H 0 wll be rejected provded ˆ. z Therefore N z z z Pr Reject H ˆ ˆ 0 Pr N Pr N Snce ˆ s N,,. MATH Part

14 N z N z Pr Reject H0 1 The maxmum of ths can be obtaned by dfferentaton w.r.t.. The dervatve s d 1 N z PrReject H 0 d where s the standard normal densty functon. Snce >0 for fnte values, t follows that PrReject H postve and soprreject H 0 s monotone ncreasng wth. d d Hence, the type 1 error rate has a maxmum when Settng, n (*) N N z N Pr Reject H0 z (*) 0. Hence, the type 1 error must be less than or equal to N s Result () s left as an exercse. MATH Part

15 x 7.3 Pan relef example contnued. Assumng that hgher pan scores represent more pan we would requre the upper confdence nterval to be less than 5. Are we able to show non-nferorty? MATH Part

16 7.5 Sample Sze for Parallel Group Non-Inferorty Trals Suppose hgher scores represent a better outcome for the patent. For a contnuous and normally dstrbuted wth means T and C for the new treatment and control groups. If one consders treatment to be non-nferor provded the (1-) % one-sded confdence nterval for ˆ yt yc s greater than N, the sample sze requred to demonstrate non-nferorty wth a power (1-) s n z z N per treatment group assumng 0 under the alternatve hypothess. The dervaton of the sample sze formula for non-nferorty trals s smlar to that for equvalence trals. Agan the dervaton wll assume that sample sze s suffcent for a normal approxmaton to the t- dstrbuton s reasonable so that ˆ N,. Wth hgher score beng a better outcome, one s testng H : 0 N vs. H : 1 N. From above N z Pr Reject H 0 Under the alternatve hypothess, 0. Therefore, the power N z 1 Snce 1 1 z, by takng nverses N z. MATH Part z

17 Hence N z z. Assumng equal sample szes nt nc n then. n n Substtuton gves the z z gvng N n z z as requred N The dervaton assumng lower scores are better s left as an exercse. x 7.4 Pan relef tral contnued. Calculate the sample sze requred to gve 80% power to reject the null hypothess of non-nferorty usng a sngle-sded 95% confdence nterval. 7.6 Lmtatons of quvalence and Non-nferorty Trals MATH Part

18 One problem wth equvalence and non-nferorty trals s that poor desgn and sloppy mplementaton reduce the dfferences between treatment groups basng the study towards the alternatve hypothess of equvalence. It s mportant therefore that patents adhere to ther treatment n ths type of tral. We wll return to ths pont when we consder ntenton to treat analyses n the next secton. quvalence trals show that two treatments may gve the same average outcome but dfference patents may beneft from a partcular treatment. quvalence trals do not demonstrate bo-equvalence, that s patents have the same outcome whch ever treatment they receve. Comparson of Sample Sze Formulae for Parallel Group Trals and a Contnuous Outcome Sample sze per group for a contnuous outcome measure for dfference hypotheses: Superorty n z z two sded n z z one sded quvalence n z z / Non-nferorty n z z N MATH Part

19 8. Analyss wth Treatment Protocol Devatons MATH Part

20 Sometme patents n a randomsed controlled tral do not receve the treatment allocated. After consentng they or ther care provder may change ther mnd, perhaps due to the change n the patent s health. Patents may decde not to take the tablets. A patent may start a treatment but then default or change to another before recevng an adequate dose. In these stuatons the patent may be sad to be non-complant or non-adherent. These changes from the randomly allocated treatment are sometmes referred to as Treatment Protocol Devatons. If patents do not adhere to ther randomly allocated treatment, should they be ncluded n the statstcal analyses, and f so how? Analyss Strateges Where There s Non-Complance Intenton-To-Treat analyss (ITT): Patents are analysed accordng to the group to whch they were randomzed, rrespectve of whether they receved the ntended nterventon. Also called As- Randomzed Per-Protocol (PP): Patents are analysed wthn the nterventon group to whch they were randomzed after excluson of noncomplant patents. As-Treated (AT): Patents are analysed accordng to the treatment they actually receved rrespectve of the random allocaton. MATH Part

21 Table 8.1 Comparson of ntenton-to-treat wth as treated and perprotocol analyss Survval at years Randomzaton Medcal treatment Surgery Receved medcne Swtch to surgery Receved Surgery Swtched to medcne [1] [] [3] [4] Ded Alve Total Two-year mortalty n the coronary bypass surgery tral publshed by the uropean Coronary Study Group (1979) from Marubn M.G. Valsecch Analysng survval Data from Clncal Trals and Observatonal Studes, p Wley Table 8. Summary of Mortalty Rates for each Analyss Method Analyss Intenton-totreat Medcal P M Surgcal P S Treat. ffect P M - P S Per-protocol As-treated MATH Part

22 Table 8.3 Summary of Inferental Analyss usng a z-test for Proportons Analyss Treat. ffect 95% c.. p-value Sgnf. In a 5% level P M - P S test Intenton-totreat.45% -1.05% to 5.96% Per-protocol 4.9% 0.66% to 7.9% As-treated 5.40% 1.79% to 9.00% In ths tral the patents that changed from medcal treatment to surgery appear to be dfferent from those patents who changed from surgery to medcal treatment. Only /50 (4%) of those that swtched from medcne to surgery ded, whereas 6/6 (3%) of those that swtched from surgery to medcne ded, a dfference n mortalty of 19%. Ths suggests that the prognoss of these two patent subgroups were very dfferent. MATH Part

23 8.1 Comparson of ITT, PP And AT Analyses When testng H 0 : = 0 vs H 1 : 0, where there s non-complance the Intenton-to-Treat estmate ˆITT s based toward H o whereas the Per- Protocol estmate ˆPP and the As-Treated estmate ˆAT may be based ether towards or away from H o. A smple mathematcal model can be constructed to llustrate the dfference between the three estmates of treatment effect. We suppose that the patent populaton can be dvded nto three subgroups as follows: Group A - who comply wth the allocated treatment (Complers Always as randomsed) Group B - who wll always receve control treatment regardless of allocaton (Always Control Treatment) Group C- who wll receve the new treatment regardless of allocaton. (Always New Treatment) It s assumed that there are no defyers, that s patents who wll always receve the opposte of the treatment to whch they are randomly allocated. As patents enter the tral the sub-group membershp of a patent s not known or latent. Patents n each of the three complance subgroups or latent classes are lkely to have a dfferent prognoss. Consderng example 7.1 f surgery was the New Treatment and medcal was the Control, group B s patent that would always receve MATH Part

24 medcal treatment. We saw that those patents appear to have a worse prognoss. As the patents are randomly allocated the expected proportons of patents n each of the three latent classes s the same n each arm of the tral. For smplcty of presentaton t wll be assumed that the treatment effect compared to the control treatment s, n all three latent classes. The quantty s the causal effect of treatment, sometmes called by the Complance Average Causal ffect (CAC), whch s the average treatment effect n patents that comply wth the New treatment. Table 8.4 Model of expected mean outcome for each treatment and latent sub-group Latent Class Control Treatment Group New Treatment Group Probablty n Latent Class As Randomzed A + A Always Control B + B + B B Always New Treatment C + C + + C + Assumptons of Model C (=1 - A - B ) No defyers, that s patents who always receve the opposte of the allocated treatment. Proporton and characterstcs of the three latent classes complers, always control, always new treatment s the same n both arms. Ths s justfed by randomzaton. Randomzaton only effects outcome through treatment. MATH Part

25 From the table of expected means the Intenton-to-Treat estmate s ITT A B B C C A B B C C A as second and thrd terms n each bracket cancel. Hence ITT whch means ITT s based towards zero f A <1.e. f some patents do not comply wth treatment. Hence ˆITT The Per-Protocol estmate s pp A C C A B B A C A B A C C C A C A B B B A C A B C C B B A C A B C C B B 1B C C 1B C B C C B B 1B 1C PP s based by terms nvolvng B and C. Snce B and C can be ether postve or negatve ˆPP from zero. may be based ether towards or away A smlar expresson can be derved for the As-Treated estmate that also shows that t can also be based towards or away from zero dependng on the magntude of B and C. MATH Part

26 Advantages of Intenton-to-Treat The Intenton-to-Treat analyss s always based towards zero so that the effcacy of the treatment s beng under-estmated. In a superorty tral, use of ntenton-to-treat bases the statstcal analyss towards the null hypothess. If one rejects the null hypothess H o : =0 based on an ntenton-to-treat analyss, one can feel confdent that the treatment effect s larger n patents that actually take the treatment. An analyss based on ntenton-to-treat s therefore sad to be conservatve. Ths s not true for per-protocol and as-treated analyses as both can be based ether towards or away from the null hypothess. Another advantage of ntenton-to treat analyss s that randomzaton clearly defnes the groups beng compared so there s no ambguty as to how the patents should be ncluded n the analyss. In contrast, the groups beng compared n per-protocol or as-treated analyses may be less well defned. Whether a partcular patent completes treatment s often dffcult to obtan. ven f one s able to collect relable data on the treatment, the researchers needs to agree how many tablets or therapy sessons a patent has to receve before they can be consdered to have compled wth treatment, whch s an ssue for whch there may be no consensus. For ths reason an ITT analyss may therefore be easer to mplement than Per-protocol or Astreatment analyses. It s mportant that all patents are followed-up, not just those that receve treatment, for ITT analyss to be carred out. MATH Part

27 8. ffcacy and ffectveness ffcacy and ffectveness are two terms used to descrbe the ablty to produce an effect such cure a specfc llness. In clncal trals a dstncton s drawn between effcacy (also known as deal use) and effectveness (also known as typcal use). We have already seen that where there s non-complance, Intenton-to-treat underestmates the effcacy of a treatment. Intenton to Treat Analyses and ffectveness Researchers may not be just nterested n whether treatment works n patents who receve a treatment. They may want to know the overall effect of offerng a treatment. Ths s partcularly true for health polcy makers. For example n a tral of exercse for the treatment of backpan some patents may not comply. If only a small proporton of patent take the treatment, the average beneft of offerng the treatment may be small, even f t s benefcal n patents that comply. It may be mportant to know the effectveness, whch s the effect takng account of non-complance, as there are lkely to be costs assocated wth offerng the treatment to patents that do not comply. It can be argued that the ntenton-to-treat (ITT) analyss gves an estmate of treatment effect takng account of non-complance. For ths reason ITT s sometmes sad to gve an estmate of the effectveness of treatments. Ths nterpretaton of ITT assumes that the proporton of patents that comply n the tral s the same as n normal care, whch may not be true. MATH Part

28 8.3 stmatng ffcacy and the CAC estmate Suppose nstead the researcher s nterested n effcacy. Provded the assumptons below table 7.5 hold, the complance average causal effect (CAC) estmate can be obtaned. From above the ITT estmate ITT Average Causal ffect s. ITT A ˆITT T C y y for contnuous data and ˆITT p T p C for bnary data. A. Hence, the Complance One needs an estmator of A, the proporton of patents who comply wth randomzaton. Ths can be obtaned as follows: Suppose the observed proportons who receve the new treatment n the treatment and control groups are respectvely q T and q C. Consderng the control group, q c ˆ Consderng the new treatment group, q ˆ ˆ Hence A can be estmated by ˆA qt qc. C T A C Hence the Complance Average Causal ffect can be estmated by ˆ ˆ ITT q q T C. It should be noted that ths estmate assumes that there are only two treatments that patents can swtch between. Ths method does not work where one s comparng two actve treatments and some patents default to a thrd opton such as no treatment. MATH Part

29 x 8. For the bypass surgery example above the Intenton-to-Treat estmate of the treatment was.45%, 4.9% for a Per-Protocol analyss and 5.40% for an As-treated analyss. stmate the Complance Average Causal ffect,. qt qc Hence ˆ The causal effect of treatment s 3.1%, whch s smaller than the Per- Protocol (4.3%) and As-Treat estmates (5.4%). Under the assumpton we have made, one can see that the Per-Protocol and As-Treat estmates are both based away from the null. The test that the complance average causal effect (CAC) s zero s equvalence to the test that the ntenton to treat effect (ITT) s zero, that s H : 0s equvalent to 0 H : 0 0. ITT Intenton-to-Treat and quvalence and Non-nferorty Trals Applcaton of the ntenton-to-treat (ITT) analyss n an equvalence tral has problems, as t s based towards the alternatve hypothess of no dfference between treatments. An ITT analyss may therefore ncrease the probablty of acceptng the alternatve hypothess. Good complance wth treatment s therefore very mportant n both equvalence and non-nferorty trals. MATH Part

30 9. Crossover Trals MATH Part

31 9.1 Motvaton for Crossover Trals When we frst consder statstcal methods for clncal tral the noton of potental outcome was ntroduced (See secton.1) A patent had two potental outcomes, say Y T andy C. For the th patent the treatment effect was defned as Y T Y C. The expected treatment effect s therefore, Y T Y C. For many condtons a sngle course of treatment may cure some f not all patents, so they would no longer be elgble for the comparator treatments. Ths apples to most treatments for acute (short-term) condtons such as antbotc treatment for an nfecton, trauma, or surgery. For these condtons, t s only possble to measure one of the two potental outcomes n the same patent - one outcome s sad to be counter factual. As prevously dscussed, the expected treatment effect s therefore estmated as the dfference n average outcome between patents recevng the nterventon (T) and those recevng the control treatment (C). The treatment effect s estmated between groups and so the precson of the estmate of the average treatment effects depends on n the varance of the outcome Var Y. In some for chronc (long-term) dseases such as arthrts, asthma, dabetes, or hgh blood pressure, the condton s not cured by treatment. Instead symptoms may be reduced or the dsease progresson slowed by contnung treatment. It may be possble to measure both potental outcomes n the same patent, and so the MATH Part

32 treatment effect can be estmated n each patent. The average treatment effect could then be estmated wth greater precson for a gven sample sze of patents. Tral sze would then be reduced makng clncal trals easer to conduct. Suppose one s able to compare two treatments, say A and B, n the same patent. One opton would be to gve treatment A followed by treatment B. Such a desgn s potentally based, because the patent s condton may deterorate or mprove over tme rrespectve of treatment. To overcome ths, patents should be randomly allocated to receve treatments n ether order. Ths tral desgn s the called the AB-BA Crossover Desgn. In such a desgn: The tral s dvded nto two perods wth one treatment gven durng each. Patents are randomly allocated to two groups, one recevng treatment sequence A then B and the other recevng B then A. Random allocaton s mportant for two reasons: To prevent bas due to possble change n the patent over tme. To mantan concealment pror to treatment allocaton and post treatment allocaton where the tral s double blnd. MATH Part

33 Use of a Crossover desgn A crossover tral desgns may be sutable where: The condton beng treated s a chronc dsease (e.g. chronc dabetes). The condton s stable and so unlkely to change greatly from one perod to the next. The nterventon has a rapd effect. A crossover desgns are not sutable where: The condtons may resolve quckly, that s an acute condton, makng the second perod treatment unnecessary. (e.g. nfectons, trauma, and rehabltaton). Patents are lkely to wthdraw from treatment or be lost to followup. The effect of the frst treatment could plausbly contamnate the effect of second. Note for some treatments contamnaton can be prevented by havng a wash-out perod between treatments. MATH Part

34 9. Analyss of an AB-BA Crossover Desgn Suppose patents are randomly allocated to ether treatment A, followed by treatment B (Sequence AB) or treatment B then treatment A (Sequence BA). Suppose that there are nab and n BA patents n each sequence wth total sample sze N nab nba. ven though we have stated that patents should be stable, a statstcal model for a crossover desgn needs to nclude parameters called a perod effect, whch s the dfference between perod and perod 1 rrespectve of treatment order as patents health wll change over tme. Two sources of varaton can be dentfed, varaton between-patents, and varaton wthn-patent. If Y j s the response for the th subject durng perod j 1,, a model for an AB-BA crossover tral can be defne as follows: yj j Sequence AB Perod 1 yj j Sequence AB Perod yj j Sequence BA Perod 1 yj j Sequence BA Perod = mean n perod 1 for the Sequence AB. = treatment effects of B compared to A. = perod effect. = random varable for patent wth 0 and varance B MATH Part. j = random varable for patent n perod j wth j 0 and varance assumed to be normally dstrbuted, N 0,.

35 B s called the between-patent varance and s called the wthnpatent varance. Usng a Sngle Sample t-test to Analyse Crossover Trals Sometme crossover trals are analysed by usng a sngle sample t- test appled to the dfference between the two treatments, also called the pared t-test. Defne c y y 1 for AB and c y 1 y for BA, whch are the dfference n outcome between when a patent receves treatment B and treatment A. The treatment effect s then estmated by ˆ c C c wth N nab nba. The hypothess H0 N : 0 s then tested usng T C c, where Sc Sc s C wth N s C beng the sample standard devaton of the dfferences, c. If c can be assumed to be normally dstrbuted, the test statstc TC has a t-dstrbuton wth N 1 degrees of freedom. Unfortunately, ˆC can gve a based estmate of. MATH Part

36 Suppose the data generaton model for an AB-BA crossover tral defned above apples. For the treatment effect estmator ˆC c ˆ C n n AB BA where N nab nba N 0andn AB n BA effect,.. If the perod effect, ˆC wll be a based estmator of the treatment Substtuton from the model above gves c AB j j. j j. Snce j 0, c AB. Smlarly, c BA Now ˆ Hence, 1. C c N ˆ C c c. n n n n AB BA AB BA N It s rarely possble to rule-out a perod effect completely. ven where n AB and n AB are planned to be equal, the may dffer due to mbalance arsng from randomsaton, such as ncomplete blocks n block randomsaton, or patents dropout from the tral. The treatment effect estmate ˆC may therefore be based and s therefore not recommended. N N MATH Part

37 An Unbased stmator of the Treatment ffect Notaton For subject, defned y y 1, that s the dfference between perod and perod 1 rrespectve of treatment order. Let d AB AB n d AB and d BA AB and BA respectvely. BA n d BA be the sample means for sequences Suppose the data generatng model for an AB-BA crossover tral dab dba defned above apples, then ˆ s an unbased estmator of, that s ˆ d AB d BA. Substtuton from the model above gves d AB j j The terms on the RHS cancel so d AB j j. Snce j 0, d AB. Hence d n d AB AB AB AB n nab AB d Smlarly j j d BA.. MATH Part

38 d BA and dba. n BA d Hence AB d BA d AB d BA Beng the dfference of two sample means of the two sequences, dab dba statstcal nference on the estmator ˆ can be based on a two sample t-test provded the assumptons of the test are satsfed. Snce the patents n sequences AB and BA are ndependent, ths assumpton t follows that ths assumpton s satsfed by the desgn. The Varance of the Dfference d Defne Var d AB d AB and Var d BA d BA. From the model for a cross-over tral (n 9.1), for sequence AB Var d Var y y Var AB d 1 1, snce Var 1 Var 1 Var Smlarly, for sequence BA,. d BA cov, 0. 1 Hence, the varances of the two sequences are the same for ths model, that s. d d d AB BA MATH Part

39 Analyss of an AB-BA Crossover Tral usng a Two Sample t-test Hypothess Testng The hypothess H : 0vs 0 H : 0 1 can be tested usng a twosample t-test of the means of the dfferences. The test statstc T s defned as T s d d S ˆ d AB d AB BA d BA 1 1 n s n s AB d BA d n AB AB n 1 1 wheres ˆ d AB d BA sd and n n BA BA wth d d AB BA s, s be the sample AB BA varances of the dfferences for the two sequences. Under assumptons of normalty and equalty of varance d d d, the test statstc T wll have a t-dstrbuton wth n n degrees of freedom. AB AB BA BA Confdence Interval of the Treatment ffect A 1 sze confdence nterval for the treatment effect s defned by 1 1 d ˆ AB dba t n AB n BA S d AB d BA. MATH Part

40 xample Bronchodlators Crossover Tral. The data n the table below s from a two-perod AB-BA randomsed crossover tral n whch patents are treated wth two bronchodlators, salbutamol (S) and formoterol (F). The outcome s the peak expratory flow (PF). Patents were randomsed to receve F then S or S then F. Increased PF s a beneft to patents. Patent PF perod 1 PF perod Y -Y 1 F then S Drug F Drug S N FS 7 d FS s 33.0 S then F Drug S Drug F N SF 6 d SF 6.5 s 44.7 d FS d SF MATH Part

41 x 9.1 Test the null hypothess H : 0vs H : d s d FS d = SF 1 1 n s n s FS d SF d n FS FS n SF SF = ˆ 1 1 S dfs d SF sd = n n T dfs S ˆ d d FS SF d SF Degrees of freedom = n n For a 5% level test = FS FS SF SF t = n FS n SF x 9. Calculate the pont estmate and 95% confdence nterval of the treatment effect. The pont estmate of the treatment effect of salbutamol (S) compared to formoterol (F) s The 95% confdence nterval of s 1 1 d ˆ FS dsf t n n S dfs d SF MATH Part

42 Fgure 9.1 Stata Output for Bronchodlators Crossover Tral Summary statstcs: mean, sd, N by categores of: Sequence Sequence PF1 PF F then S S then F Two Sample t-test Analyss Appled to the Dfferences between Perod ( d ) Two-sample t test wth equal varances Group Obs Mean Std. rr. Std. Dev. [95% Conf. Interval] F then S S then F dff dff = mean(f then S) - mean(s then F) t = Ho: dff = 0 degrees of freedom = 11 Ha: dff < 0 Ha: dff!= 0 Ha: dff > 0 Pr(T < t) = Pr( T > t ) = Pr(T > t) = The treatment effect and ts confdence nterval are obtaned by halvng the values n the prntout. From the prnt-out dff equals mean(f then S) - mean(s then F) The treatment effect for salbutamol compared to formoterol s ˆ Concluson There s evdence that formoterol gves mproved outcome compared to salbutamol wth an ncrease n PF equal to 46.6 (95% c...9 to 70.3, p=0.001). Note that the p-values should NOT be halved!!! MATH Part

43 Fgure 9. Stata Output for the Sngle Sample t-test Appled to the Dfferences between treatments ( c ) One-sample t test Varable Obs Mean Std. rr. Std. Dev. [95% Conf. Interval] SALB-FORM mean = mean(salb-form) t = Ho: mean = 0 degrees of freedom = 1 Ha: mean < 0 Ha: mean!= 0 Ha: mean > 0 Pr(T < t) = Pr( T > t ) = Pr(T > t) = By ths method effect for salbutamol compared to formoterol ˆC Ths has a slght bas compared to the unbased method ( ˆ -46.6). It s worth notng also that the standard error for the sngle sample method s slghtly larger. For then unbased analyss S dfs d SF ˆ S whereas S ˆ , so the unbased method s also appears to be more precse. C It can be shown that the perod effect ncreases the standard error of the sngle sample methods compared to the two-sample method. MATH Part

44 9.3 Sample Sze for AB-BA Cross-over Trals Gven that the analyss of an AB-BA crossover tral should be based on a two sample t-test of the dfferences, we can estmate sample sze for a crossover tral usng the formula prevously derved n the notes for sample sze for a parallel group tral for a contnuous outcome (see secton 4.3). Consder the varance of the dfference, d. For an AB-BA crossover tral wth equal numbers randomsed to each sequence, the total sample sze N requred to have power 1 to detect a treatment effect usng a two-sded -sze test of the hypothess of superorty, H : 0vs : 0 0 1, where d s the varance of the dfferences. H s d N z z If analyss of an AB-BA crossover tral s based on a two sample t- test, the total sample sze assumng two equal sze groups can be estmated from the formula n secton 4.3 for sample sze per group. 4 Total sample sze s N z z. If s replaced d n the above formula, should be replace by, because dab dba estmates. MATH Part

45 9.4 Analyss of the Perod ffect A queston of secondary nterest s Is there a perod effect? How does one test H : 0 vs H : 0? 0 1 Consder the dfferences between treatment B and A for each subject defne by c y y 1 for AB and c y 1 y for BA. These are the dfference used n the based sngle sample method. It can be shown that c BA BA n c BA c d AB BA c BA where, suggestng ˆ c c AB AB c AB BA n AB c d AB and. The detals are an exercse. A test of the hypothess H : 0 vs 0 H : 0 1 can be carred out as a two sample t-test between the two sequences. Fgure 8.3 Stata Output for Bronchodlators Crossover Tral Two Sample t-test Analyss Appled to the Dfferences between treatments ( c ) Group Obs Mean Std. rr. Std. Dev. [95% Conf. Interval] F then S S then F dff dff = mean(f then S) - mean(s then F) t = Ho: dff = 0 degrees of freedom = 11 Ha: dff < 0 Ha: dff!= 0 Ha: dff > 0 Pr(T < t) = Pr( T > t ) = Pr(T > t) = x 9.3 stmate the perod effect from the prnt-out The perod effect MATH Part

46 9.5 The Carry-Over ffect Above, we consdered how to test for a perod effect. Ths assumed the same change for each sequence. There s the possblty that the change, after accountng for the treatment effect, dffer between the two sequences AB and BA. Ths can occur f one treatment has greater persstent nto the second perod than the other and s called the carry-over effect. One can ncorporate such a dfferental effect nto the model of an AB-BA tral by addng a term to represent the dfference n the perod effect for the two sequences, BA and AB. For each group and perod the models are now: yj j Sequence AB Perod 1 yj j Sequence AB Perod yj j Sequence BA Perod 1 yj j Sequence BA Perod Wth d y y 1, t follows that Hence d for AB and d AB n AB d for BA. dba n BA ˆ dab d BA nstead of. We have shown that the treatment effect wll be based, f there s a carry-over effect. MATH Part

47 A Flawed Statstcal Analyss nvolvng Carry-Over ffect To deal wth carry-over effect n crossover trals the followng analyss procedure has been suggested. 1. Test whether there s a carry-over effect.e. H : 0vs H : If H : 0 0 s not rejected, the analyss proceeds as descrbed above. 3. If H : 0 0 s rejected, the analyss should just be based on the perod 1 data reducng the study to a parallel group tral and the perod data s then dscarded. The followng test of for a carry-over effect has been proposed. Defne a y y 1so that a AB 1 a BA 1 Defnng the sample mean for each sequence as a AB and a BA, wth correspondng populaton means and a AB a a. Therefore BA AB BA a AB and a BA a Ths suggests the hypothess H 0 : =0 s equvalent to H 0 : whch could be tested by test. T a aba aab S ˆ a a BA AB a AB, a BA usng a two sample t- MATH Part

48 Consder now 4 4 Var a Var 1 Var Var Var 1 B 4B Therefore Var aab 4B and Var aba. nab nba Hence Var aba aab 4 B 1 1 nba nab, gvng SaBA aab 4 B 1 1. nba nab Ths contans both wthn-patent and between-patent varaton. We can therefore draw the followng concluson: Unless B s small relatve to the test statstc T a wll have low power to reject the hypothess of no carry-over effect H : 0 0 compared to a test of the treatment effect. The advantage of crossover trals s that they remove between-patent varance B, and so there s only gong to be power to detect a carryover effect where there s lttle beneft n usng a crossover desgn. To ncrease the sample sze so that a carry-over effect could be detected would remove the man advantage of the crossover desgn. Note, also that the nferental reasonng of the test s faulty as n practce one would want to show that s zero to justfy the crossover analyss. As wth an equvalence tral one would need to have H : 0 and 0 H : 0 1. For these reasons the procedure for testng for carry-over effect n a crossover tral s no longer recommended. MATH Part

49 Implcatons of Carry-Over ffect and Crossover Trals When plannng a crossover tral, t s mportant to consder whether a carry-over effect may occur, as estmaton s only unbased where t s absent. Ths assumpton has to be based on scentfc arguments regardng the way treatments work rather than statstcal tests. For example, n a partcular stuaton carry-over may not be plausble so t can be safely gnored. Alternatvely, t may be elmnated by a washout perod between the two treatments. In such crcumstances a crossover tral may be legtmately conducted and the advantages of crossover tral exploted. Unfortunately, n many stuatons a carry-over effect may be plausble and lengthy washouts perods may not be feasble as t would be unethcal for a patents to go wthout treatment. There are therefore many crcumstances where a crossover tral s not feasble, when testng treatments for chronc condtons. Ths s an example of where clncal scence s mportant for determnng the choce of desgn. In summary, f there are good scentfc reasons to beleve a carry-over effect may occur, the crossover desgn s not recommended and a parallel group desgn should be used nstead. MATH Part

50 9.6 Summary: Comparson of Parallel and Crossover Trals Parallel Group Desgn Comparson of treatments s between groups of patents. Power and sample sze depends on between-subject varaton. Crossover Desgn Comparson of treatments s wthn patents so that each patent acts as ther own control. Advantage of a Crossover Desgn compared to a Parallel Group Desgn Wthn patent estmaton of treatment effects means that varaton between patents s removed from the analyss, hence sample sze may be substantally smaller. Dsadvantages of a Crossover Desgn compared to a Parallel Group Desgn Only applcable to certan types of condton such as stable dseases. More complcated to organze. Patents wthdrawng durng the second perod mean that ther data cannot be ncluded n the statstcal analyss. Requres the assumpton of no Carry-over effect to gve unbased estmates of the treatment effect. MATH Part

51 10. Systematc Revew and Meta-Analyss MATH Part

52 10.1 Systematc Revew If a clncal tral has been properly conducted, t should provde nformaton regardng the effcacy or effectveness of a new therapy. Once a tral has been publshed, t mght be thought that t would be unethcal to undertake another tral makng the same comparson. In practce, the decson s rarely so smple. Tral results generally need to be replcated before new treatment can be wdely adopted. Clncal trals of new treatments are often repeated wthn dfferent global regons (e.g. urope, Amercas, or Asa) to assess ther generalsablty. Modfcatons to the tral desgn may be made to remove perceved bases n the desgn of earler studes or test the effect of treatment on other outcome measures. Where several trals have been carred out to compare the same treatments, the tradtonal method for assessng the evdence nvolved selectng from the readly avalable tral reports, apprasng each, before drawng conclusons n a narratve dscusson. Ths type of revew can be hghly subjectve and open to selecton bas. An alternatve s a systematc revew, n whch studes are dentfed systematcally n attempt to fnd all before combnng the results by an overall statstcal summary. Systematc revew s now an mportant component of the evaluaton of new treatments and dagnostc test procedures, and s also used to combne evdence from epdemologcal studes. Meta-analyss s the statstcal methodology for combnng data from several studes of the same queston to produce an overall summary. MATH Part

53 A systematc revew followed by a meta-analyss can brng together the results from several nconclusve or conflctng studes to gve a sngle conclusve result. It also gves greater power and precson to answer more refned research questons. For example, ndvdual trals are usually desgned to answer the queston does a treatment work on average. They rarely have suffcent power to nvestgate dfferences n the treatment effect for specfc types of patent, but ths may be possble by combnng data from several studes n a metaanalyss. A systematc revew may also enable one to nvestgate rarer outcomes, such as serous adverse events, that may not be possble n a sngle tral. For example by combnng trals of treatments for depresson t has been possble to show that some drug treatments ncreased the rsk of sucdal behavour, a result that could not be demonstrated n ndvdual trals due to lack of power for ths outcome measure. Steps n a Systematc Revew A systematc revew s smlar to a clncal tral. It nvolves several steps. 1. Defne precse objectves for the revew.. Set ncluson and excluson crtera for trals. 3. Search for trals satsfyng the ncluson crtera. 4. Assess methodologcal qualty of studes dentfed, possbly dscardng methodologcally poor studes. 5. xtract statstcal summary data or obtan raw data for each study. 6. stmate the overall treatment effect by a meta-analyss. MATH Part

54 10. Meta-analyss Meta-analyss of generally address the followng questons 1. Are the effects n the studes homogeneous? Ths s needed to justfy estmatng an overall treatment effect.. What s the overall treatment effect? 3. Do study sze, study characterstcs or methodologcal qualty correlate wth the magntude of the treatment effect? The best way to carry out a meta-analyss s to combne the raw data from ndvdual studes nto a sngle large dataset, and then carry out an analyss of all the data to estmate the overall effect. Ths s method called ndvdual patent data meta-analyss. Whlst ths s smlar to analysng a sngle large study, analyss should take account of data comng from several studes. Indvdual patent data meta-analyss s often not possble, because the orgnal raw data are no longer avalable for all studes, partcularly where some may be many years old. For ths reason, most meta-analyses use summary statstcs extracted from publshed reports. Ths method s called summary measures meta-analyss and s a specal set of methods. MATH Part

55 Fxed or Random ffect Meta-analyss Suppose there are k studes and the treatment effect estmate for the th study s ˆ. Suppose the overall treatment effect s. There are two man approaches to estmaton of. In the frst, we assume that ˆ each tral s estmatng the a common effect of treatment. Any departure of ˆ from s assumed to be smply due to samplng varaton. Ths s called Fxed ffects estmaton. The second approach s called Random ffects estmaton. Ths assumes that the studes are sampled from a larger populaton of studes. The treatment effect s then a random varable wth mean equal to the overall effect and varance v. If ˆ s the overall estmate, Var ˆ wll be larger f estmated by random effect estmaton than fxed effect estmaton due to the addtonal varance term v. In ths module we wll just descrbe methods of analyss for fxed effects estmaton. MATH Part

56 10.3 Summary Measures stmaton of the Overall ffect Suppose there are k trals comparng two treatments. Let ˆ be the estmate of the treatment effect for the j th tral and let For a contnuous outcome measure y, defne y j, Var ˆ. MATH Part v s j and nj 1,..., k; j 1, to be sample mean and varance, and the sample sze respectvely of the j th treatment n the th tral. The treatment effect of the th tral can be the mean dfference, ˆ y y 1 wth vˆ ˆ ˆ s s Var n n 1 1. If Y s bnary, one could use the rate dfference (RD), as the summary statstc for each tral. If the observed number of events s r j, the observed proportons pj rj nj P P wth ˆ 1 Var ˆ 1,..., k; j 1, ˆ. One can defne 1 1 p p p p 1 1 n 1 n Alternatvely, one mght want to estmate an overall odds rato (OR) or rate rato (RR). These are generally estmated by takng ˆ equal to the log e OR or log e RR. For log e OR, vˆ r 1 n 1 r 1 r n r 4. For log e RR, vˆ. r 1 n 1 r n vˆ demonstrated n secton.

57 The overall estmate for odd rato and rate rato are obtaned by takng the exponent of the overall log e ORand log e RR estmates. Summary Measures stmate of the Overall ffect Whchever type of summary measure s used (mean dfference, RD, log [OR] or log[rr]), an overall estmate of can be estmated by a weghted mean defned as k ˆ w ˆ w. 1 1 k Consder now the varance of the estmate for ˆ. 1 Var Var w w 1 k ˆ. ˆ k 1 Snce the studes are ndependent, Cov ˆ, ˆ j 0 Therefore, Var ˆ k 1 w ˆ Var. k w 1 Choce of Weghts and the Mnmum Varance stmate Dfferent weghts wll gve dfferent estmates of Band Var ˆ. We could weght studes equally by settng w 1, 1,..., k, but ths s rarely done as the sze of studes generally vares greatly. It can be MATH Part

58 1 shown that takng w for each gves an estmator wth Var ˆ mnmum varance, that s wth greater precson. For ths reason, nverse varance weghts are often used n meta-analyss. The weghted varance when f w 1/Var[ ]. Var ˆ k 1 w ˆ. Var wll have a mnmum k w 1 The proof uses the Lagrange Multpler method for obtanng maxma or mnma subject to a constrant. Let Var ˆ F w, w,..., w k w. Var 1 1 k k 1 w ˆ Wthout loss of generalty one can apply the constrant k w 1. 1 G w1, w,..., wk w 1. Defne k 1 Applyng the Lagrange Multpler Method one defnes,,...,,,,...,,,..., H w w w F w w w G w w w 1 k 1 k 1 k The mnmum of F subject to the constrant G s found by equatng the partal dervatves of H w, w,...,, 1 zero. Consderng the j th study w wth respect to each w to k MATH Part

59 k k H 1,,...,,. ˆ w w w k w Var w 1 wj w j 1 w j 1 Hence H w 1, w,..., w ˆ k w j Var j 0 w j gvngw ˆ Var The second dervatves of H are postve so ths must be a mnmum. Hence w 1 Var ˆ gves the estmate wth mnmum varance If ˆMV s the mnmum varance estmate then 1 1 Var ˆ MV k k 1 w ˆ 1Var 1 Substtutes 1 Var ˆ for w, nto Var ˆ gves k 1 w ˆ Var k 1 w k k ˆ w. Var ˆ 1 Var 1 MV k k k 1 k w 1 ˆ 1 1 Var 1 ˆ 1 Var ˆ 1 1 Var w as requred MATH Part 1