BIOSTATISTICS TOPIC 8: ANALYSIS OF CORRELATIONS I. SIMPLE LINEAR REGRESSION

Transcription

1 BIOSTATISTICS TOPIC 8: ANALYSIS OF CORRELATIONS I. SIMPLE LINEAR REGRESSION Gve a ma three weapos - correlato, regresso ad a pe - ad he wll use all three. Ao, 978 So far we have bee cocered wth aalyses of dffereces. Ad, dog so, we have cosdered measurg subjects o a sgle outcome varable (or two groups of subjects o oe varable). Such measuremets have yelded uvarate frequecy dstrbuto ad the aalyss s ofte referred to as uvarate aalyss. Now, we are cosderg subjects ad each subject has two measures avalable; other words, we have two varables per subject, say x ad y. Our terest ths kd of data s obvously to measure relatoshp betwee the two varables. We ca plot the value of y agast the value of x a scatter dagram ad assess whether the value of y vares systematcally wth the varato values of x. But we stll wats to have a sgle summary measure of the stregth of relatoshp betwee x ad y. I hs book "Natural Ihertace", Fracs Galto wrote: "each pecularty a ma s shared by hs ksma, but o the average, a less degree. For example, whte tall fathers would ted to have tall sos, the sos would be o the average shorter tha ther fathers, ad sos of short fathers, though havg heghts below the average for the etre populato, would ted to be taller tha ther fathers." He, the, cocluded a pheomeo called "law of uversal regresso" whch was the org of the topc we are learg rght ow. Today, the characterstcs of returg from extreme values toward the average of the full populato s well recogsed ad s termed "regresso toward the mea". We wll cosder methods for assessg the assocato betwee cotuous varables usg two methods kow as correlato aalyss ad lear regresso aalyss, whch are happeed to be some of the most popular statstcal techques medcal research.

2 I. CORRELATION ANALYSIS.. THE COVARIANCE AND COEFFICIENT OF CORRELATION I a prevous topc, we stated that f Y ad Y are depedet varables, the the varace of the sum or dfferece betwee X ad Y s equal to the varace of X plus the varace of Y, that s: var(x + Y) = var(x) + var(y) what happe f X ad Y are ot depedet? Before dscussg ths problem, we troduce the cocepts of covarace ad correlato. I elemetary trgoometry we lear that for a rght tragle, f we let the hypoteuse sde be c ad the other two sdes be a ad b, the Pythagoras' theorem states that: c = a + b ad ay tragle: c = a + b ab.cos C (Cose rule). Aalogously, f we have two radom varables X ad Y, where X may be the heght of father ad Y may be the heght of daughter, ther varace ca be estmated by: s x = = ( x x) [] respectvely. ad s = ( y y) y = Furthermore, f X ad Y are depedet, we have:

3 s + = s + s X Y X Y [] Let us ow dscuss X ad Y the cotext of geetcs. Let X be BMD of father ad Y be the BMD of daughter. It s clear that we ca fd aother expresso for the relatoshp x x by betwee X ad Y by multplyg each father's BMD from ts mea ( ) correspodg devato of hs daughter ( y y), stead of squarg the father's or daughter's devato, before summato. We refer ths quatty to as covarace betwee X ad Y ad s deoted by Cov(X, Y); that s: cov ( X, Y ) = ( x x)( y y) [3] = By defto ad aalogous to the Cose law ay tragle, we have: f X ad Y are ot depedet, the: : = σ + + Cov( X Y ) σ [4] X + Y X σy, A umber of pots eed to be oted here: (a) Varaces as defed [] are always postve sce they are derved from sums of squares, whereas, covaraces as defed [3] are derved from sum of cross-products of devatos ad so may be ether postve or egatve. (b) A postve value dcates that the devatos from the mea oe dstrbuto, say father's BMDs, are prepoderatly accompaed by devatos the other, say daughter's BMDs, the same drecto, postve or egatve. (c) A egatve covarace, o the other had, dcates that devatos the two dstrbutos are prepoderatly opposte drectos. (d) Whe the devato oe of the dstrbuto s equally lkely to be accompaed by devato of lke or opposte sg the other, the covarace, apart from errors of radom samplg, wll be zero. The mportace of covarace s ow obvous. If varato of BMD s uder geetc cotrol we would expect hgher BMD fathers geerally have hgh BMD daughters ad low 3

4 BMD fathers geerally have low BMD daughters. I other words, we should expect them to have postve covarace. Lack of geetc cotrol would produce a covarace of zero. It was by ths meas that Galto frst showed stature ma to be uder geetc cotrol. He foud that the covarace of paret ad offsprg, ad also that of pars of sblgs, was postve. The sze of the covarace relatve to some stadard gves a measure of the stregth of the assocato betwee the relatves. The stadard take s that afforded by the varaces of the two separate dstrbutos, our case, of father's BMD ad daughter's BMD. We may compare the covarace to these varaces separately ad we do ths by calculatg the regresso coeffcets whch have the forms: or Cov var Cov var ( X, Y ) ( X ) ( X, Y ) ( Y ) (regresso of daughters o father) (regresso of fathers o daughters) we ca also compare the covarace wth the two varaces at oce: ( X, Y ) ( Y ) var( Y ) Cov var Ths s called the coeffcet of correlato ad s deoted by r..e. r = ( X, Y ) ( X ).var( Y ) Cov var ( X, Y ) Cov = s x s y [5] r wll have a maxmum value of (a complete determato of daughter's BMD by father's BMD) ad mmum value of 0 (o relatoshp betwee father's ad daughter's BMDs). way: Wth some algebrac mapulato, we ca show that [5] ca be wrtte aother 4

5 r = = = ( x x)( y y) ( x x) ( y y) = = x y x = = = ( ) sxsy y [6] where s x ad s y are stadard devatos for X ad Y varable, respectvely... TEST OF HYPOTHESIS Oe obvous questo s that whether the observed coeffcet of correlato (r) s sgfcatly dfferet from zero. Uder the ull hypothess that there s o assocato the populato (r = 0), t ca be show that the statstc: t = r r has a t dstrbuto wth - df. O the other had, for a moderate or large sample sze, we ca set up a 95% cofdece terval of r by usg a theoretcal dstrbuto of r. It ca be show that the samplg dstrbuto of r s ot ormally dstrbuted. We ca, however, trasform t to a Normal dstrbuted quatty by usg the so-called Fsher's trasformato whch: + r z = l r [7] The stadard error of z s approxmately equal to: SE ( z) = [8] 3 5

6 Thus, approxmate 95% cofdece terval s: z to z Of course, we ca back-trasform the data to obta 95% cofdece terval for r (ths s left for exercse). Example : Cosder a clcal tral volvg patets presetg wth hyperlpoproteaema, basele values of the age of patets (years), total serum cholesterol (mg/ml) ad serum calcum level (mg/00ml) were recorded. Data for 8 patets are gve below: Patet Age Cholesterol (X) (Y) Mea S.D Let age be X ad cholesterol be Y, to calculate the correlato coeffcet, we eed to calculate the covarace Cov(X, Y) whch s: 6

7 Cov(X, Y) = ( x x)( y y) = = xy xy = = 0.68 The the coeffcet of correlato s: r = ( X Y ) Cov, s x s y = = To test for the sgfcace of r, we eed to covert t to the z score as gve [7]: z = + r l r = l = 0.56 wth the stadard error of z s gve [8]: SE ( z) = = = 0.58 The the t rato s 0.56 / 0.58 =.65 whch exceeds the expected value of. (wth 7 df ad 5% sgfcace level), we coclude that there s a assocato betwee age ad cholesterol ths sample of subjects. //.3. TEST FOR DIFFERENCE BETWEEN TWO COEFFICIENTS OF CORRELATION Suppose that we have two sample coeffcets of correlato r ad r whch were estmated from two ukow populato coeffcets ρ ad ρ, respectvely. Suppose further that r ad r were derved from two depedet samples of ad subjects, respectvely. To test the hypothess that ρ = ρ versus the alteratve hypothess that ρ ρ, we frstly covert these sample coeffcets to a z-score: 7

8 By theory, the statstc z z + r = l r ad + r = z l r z s dstrbuted about the mea Mea(z z ) = ρ ρ ( ) ( ) where ρ s the commo correlato coeffcet, wth varace Var(z z ) = If the samples are ot small or f ad are ot very dfferet, the statstc t = z z ca be used as a test statstc of the hypothess. 8

9 II. SIMPLE LINEAR REGRESSION ANALYSIS We are ow extedg the dea of correlato to a rather mechacal cocept called regresso aalyss. Before dog so, let us brefly recall ths dea the hstorcal cotext. As metoed earler, I 885, Fracs Galto troduced the cocept of "regresso" a study that demostrated that offsprg do ot ted toward the sze of parets, but rather toward the average as compared to the parets. The method of regresso has, however, a loger hstory. I fact, a legedary Frech mathematca by the ame of Adre Mare Legedre publshed the frst work o regresso (although he dd ot use the word) 805. Stll, the credt for dscovery of the method of least squares geerally gve to Carl Fredrch Gauss (aother legedary mathematca), who used the procedure the early part of the 9th cetury. Much used (ad perhaps overused) clche data aalyss "garbage - garbage out" ad "the results are oly as good as the data that produced them" apply the buldg of regresso models. If the data do ot reflect a tred volvg the varables, there wll be o success model developmet or drawg fereces regardg the system. Eve wth some types of relatoshp does exst, ths does ot mply that the data wll reveal t a clearly detectable fasho. May of the deas ad prcples used fttg lear models to data are best llustrated by usg smple lear regresso. These deas ca be exteded to more complex modellg techques oce the basc cocepts ecessary for model developmet, fttg ad assessmet have bee dscussed. Example (cotued): The plot of cholesterol (y-axs) versus age (x-axs) yelds the followg relatoshp: 9

10 From ths graph, we ca see that cholesterol level seems to vary systematcally wth age (whch was cofrmed earler by the correlato aalyss); moreover, the data pots seem to scatter aroud the le coects betwee two pots (0,.) ad (65, 4.5). Now, we leared earler ( Topc ) that for ay two gve pots a two-dmesoal space, we could costruct a straght le through two pots. The same prcple s appled here, although the techque of estmato s slghtly more complcated... ESTIMATES OF LINEAR REGRESSION MODEL Let the observed pars of values x ad y be ( x, y ), ( x ), y,..., ( x, ) y essece of a regresso aalyss s cocered wth relatoshps betwee a respose or depedet varable (y) ad explaatory or depedet varable (x). The smplest relatoshp s the straght le model: y = β0 + β x + ε [8]. The I ths model, β 0 ad β are ukow parameters ad are to be estmated from the observed data, ε s a radom error or departure term represetg the level of cosstecy preset repeated observatos uder smlar expermetal codtos. To proceed wth the parameter estmato, we have to make some assumptos () The value of x s fxed (ot radom); 0

11 ad o the radom error ε, we assume that ε's are: () ormally dstrbuted; () has expected value 0.e. E(ε) = 0 () costat varace σ for all levels of X; (v) ad successvely ucorrelated (statstcally depedet). Because β 0 ad β are parameters (hece, costats) ad that the value of x s fxed, we ca obta the expected value of [8] as : E(y ) = β 0 + β x [9] ad var( y ) = var(β 0 + β x + ε ) = var(ε ) = σ. [0] LEAST SQUARE ESTIMATORS To estmate β 0 ad β from a seres of data pots ( x, y ), ( x ), y,..., ( ) we use the method of least squares. Ths method estmates two costats b 0 ad b (correspodg to β 0 ad β ) so that they mmse the quatty: x,, y Q = [ y ( b + b x )] = 0 x It turs out that to mmse ths quatty, we eed to solve a system of smultaeous equatos: y = b + b x 0 xy = b x + b x 0 Ad the estmates tur out to be:

12 b = ( x x)( y y) ( x x) = cov var ( x, y) ( x) [] ad b0 = y bx [] Example (cotued): I our example, the estmates are: ( x, y) 0.68 ( x) Cov b = = = cov ad b0 = y bx = (38.83) =.089. Hece the regresso equato s: y = x That s, for ay dvdual, hs/her cholesterol s completely determed by the equato: Cholesterol = (Age) + e where e s the specfc error whch s ot accouted for by the equato (cludg measuremet error) assocated wth the subject. For stace, for subject (46 years old), hs/her expected cholesterol s: x 46 = ; whe compared wth hs/her actual value of 3.5, the resdual s e = = Smlarly, the expected cholesterol value for subject s x 6 =.45 ad s hgher tha hs/her actual level by The predcted value calculated usg the above equato, together wth the resduals (e) are tabulated the followg table.

13 I.D Observed Predcted Resdual (O) (P) (e = O - E) TEST OF HYPOTHESIS CONCERNING REGRESSION PARAMETERS. To some large extet, the terest wll le the values of slope. Iterpretato of ths parameter s meagless wthout a kowledge of ts dstrbuto. Therefore, havg calculate the estmates b ad b 0, we eed to determe the stadard error of these parameters so that we ca make fereces regardg ther sgfcace the model. Before dog ths, let us have a bref look at the sgfcace of the term e. We leared earler topc that f y s the sample mea of a varable Y, the the varace of Y s gve by ( y y). Now, the regresso case, y s actually equal = to β 0 + β x = $y. Hece, t s reasoable that the sample varace of the resduals e 3

14 should provde a estmator of σ [0]. It s from ths reasog that the ubased estmate of σ s defed as: s = = ( y yˆ ) = ( e ) [3] It ca be show that the expected values of b ad b 0 are β ad β 0 (true parameters), respectvely. Furthermore, from [3], t ca be show that the varaces of b ad b 0 are: s var ( b ) = [4] ( x x) = ad var( b ) = var( y) + ( x) ( b ) whch s: ( b ) 0 var x var 0 = s + [5] ( x x) = Oce ca go a step further by estmatg the covarace of b ad b 0 by: Cov ( b b ) ( x) s ( ) = [6], 0 b That s, b s ormally dstrbuted wth mea β ad varace gve [4], ad b 0 s ormally dstrbuted wth mea β 0 ad varace gve [5]. It follows that the test for sgfcace of b s the rato b bs t = = s s s x x whch s dstrbuted accordg to the t dstrbuto wth - df. 4

15 ad t = s b 0 ( / ) + ( x / s ) x s a test for b 0, whch s dstrbuted accordg to the t dstrbuto wth - df. Example (cotued): I our example, the estmate resdual varace s s calculated as follows: s = [(-0.475) + ( ) (0.079) ] / (8-) = We ca calculate the corrected sum of square of AGE, ( x x) from the estmate varace as: = ( x x) = s x ( - ) = (7) = =, by workg out Hece, the estmated varace of b s: var(b ) = / = SE(b ) = var( b ) = A test of hypothess of β = 0 ca be costructed as: t = b / SE(b ) = / = 0.70 whch s hghly sgfcat (p < 0.000). For the tercept we ca estmate ts varace as: 5

16 var(b 0 ) = x s + ( x x) = ( ) = = Ad the test of hypothess of β 0 = 0 ca be costructed as: t = b 0 / SE(b 0 ) =.089 / = 4.9 whch s also hghly sgfcat (p < 0.00)..3. ANALYSIS OF VARIANCE A aalyss of varace parttos the overall varato betwee the observatos Y to varato whch has bee accouted for by the regresso o X ad resdual or uexplaed varato. Thus, we ca say: Total varato = Varato explaed + Resdual about the mea by regresso model varato I ANOVA otato, we ca wrte equvaletly: SSTO = SSR + SSE or, = ( y y) = ( yˆ y) + ( y yˆ ) = = 6

17 Now, SSTO s assocated wth - df. For SSR, there are two parameters (b 0 ad b ) the model, but the costrat ( y y) = ˆ = 0 takes away df, hece t has fally df. For SSE, there are resduals (e ); however, df are lost because of two costrats o the e 's assocated wth estmatg the parameters β 0 ad β by the two ormal equatos see secto.). We ca assemble these data a ANOVA table as follows: Source df SS MS Regresso SSR = ( y y) = Resdual error - SSE = ( ) = Total - SSTO = ( y y) ˆ MSR = SSR/ y yˆ MSE = SSE / (-) = R-SQUARE From ths table t seems to be sesble to obta a "global" statstc to dcate how well the model fts the data. If we dvde the regresso sum of square (varato due to regresso model, SSR) by the total varato of Y (SSTO), we would have what statstcas called the coeffcet of determato, whch s deoted by R : : R = = = ( yˆ y) SSR SSTO ( y y) = = SSE SSTO [7] R. I fact, t ca be show that the coeffcet of correlato r defed [5] s equal to 7

18 Obvously, R s restrcted to 0 < R <. A R = 0 dcates that X ad Y are depedet (urelated), whereas a R = dcates that Y s completely determed by X. However, there are a lot of ptfalls ths statstc. A value of R = 0.75 s lkely to be vewed wth some satsfacto by expermeters. It s ofte more approprate to recogse that there s stll aother 5% of the total varato uexplaed by the model. We must ask why ths could be, ad whether a more complex model ad/or cluso of addtoal depedet varables could expla much of ths apparetly resdual varato. A large R value does ot ecessarly mea a good model. Ideed, R ca artfcally hgh whe ether the slope of the equato s large or the spread of the depedet varable s large. Also a large R ca be obtaed whe straght les are ftted to data that dsplay o-lear relatoshps. Addtoal methods for assessg the ft of a model are therefore eeded ad wll be descrbed later. F STATISTIC A assessmet of the sgfcace of the regresso (or a test of the hypothess that β = 0) s made from the rato of the regresso mea square (MSR) to the resdual mea square MSE (s ) whch s a F-rato wth ad - degrees of freedom. Ths calculato s usually exhbted a aalyss of varace table produced by most computer programs. F = MSR [8] MSE It s mportat that a hghly sgfcat F rato should ot seduce the expermeter to a belef that the straght le fts the data superbly. The F test s smply a assessmet of the exted to whch the ftted le has a slope whch s dfferet from zero. If the slope of the le s ear zero, the scatter of the data pots about the le would eed to be small order to obta a sgfcat F rato. However, a stuato wth a slope very dfferet from zero ca gve a hghly sgfcat F rato wth a cosderable scatter of pots about the le. 8

19 b bs The F test as defed [8] s actually equvalet to the t test t = = s s s The F test s therefore ca be used for testg β = 0 versus β 0 ad s ot for testg oe-sded alteratves. x x. Example (cotued): I our example, the sum of squares due to regresso le s: SSR = ( yˆ y) = = ( ) + ( ) ( ) = whch s assocated wth df, hece ts mea square s The sum of squares due to resduals s: SSE = ( ˆ ) = y y = (-0.475) + (0.3450) (0.079) =.4656 ad s assocated wth 8- = 6 df, hece ts mea square s.4656 / 6 = The F statstc s the: F = / = Hece, the ANOVA table ca be set up as follows: Source df SS MS F-test 9

20 Regresso Resdual errors Total Accordgly, the coeffcet of determato s: R = 0.49 /.96 = Ths meas that 87.75% of total varato cholesterol betwee subjects s "explaed" by the regresso equato. //.3. ANALYSIS OF RESIDUALS AND THE DIAGNOSIS OF REGRESSION MODEL A resdual s defed as the dfferece betwee the observed ad predcted y value, gve by e = y y$, the value whch s ot accouted for by the regresso equato. Hece, a examato of ths term should reveal how approprate the equato s. However, these resduals do ot have costat varace. I fact, var(e ) = (-h )s, where h s the th dagoal elemet of the matrx H whch s such that y = Hy. H s called the "hat matrx", sce t defes the trasformato that puts the "hat" o y! I vew of ths, t s preferable to work wth the stadardsed resduals. I smple lear regresso case, the stadardsed resdual r s defed as: r e = [9] MSE These stadardsed resduals have mea 0 ad varace. We ca use r to verfy assumptos of the regresso model whch we made secto.. These are: (a) are the regresso fucto s ot lear; (b) the dstrbutos of Y (cholesterol) do ot have costat varace at all level of X (age) or equvaletly the resduals do ot have costat varaces; (c) the dstrbutos of Y are ot ormal or equvaletly the resduals are ot ormal; (d) the resduals are ot depedet. 0

21 Useful graphcal methods for examg the stadard assumptos of costat varace, ormalty of the error terms ad approprateess of the ftted model clude: - A plot of resduals agast ftted values to detfy outlers, detect systematc departure from the model or detect o-costat varace; - A plot of resduals the order whch the observatos were take to detect o-depedece. - A ormal probablty plot of the resduals to detect from ormalty. - A plot of resduals agast X ca dcate whether a trasformato of the orgal X varable s ecessary, whle a plot of resduals agast X varables omtted from the model could reveal whether the y varable depeds o the omtted factors. OUTLIERS Outlers regresso are observatos that are ot well ftted by the assumed model. Such observatos wll have large resduals. A crude rule of thumb s that a observato wth a stadardsed resdual greater tha.5 absolute value s a outler ad the source of that data pot should be vestgated, f possble. More ofte tha ot, the oly evdece that somethg has goe wrog the data geeratg process s provded by the outlers themselves! A sesble way of proceedg wth the aalyss s to determe whether those values have substatal effect o the fereces to be draw from the regresso aalyss, that s, whether they are fluetal. INFLUENTIAL OBSERVATIONS Geerally speakg, t s more mportat to focus o fluetal outlers. But t s ot oly outlers that ca be fluetal. If observato s separated from the others terms of the values of the X-varables, ths observato s lkely to fluece the ftted regresso mode. Observatos separated from other ths way wll have a large value of h. We call h the leverage, A rough gude s that observatos wth h > 3p/ are fluetal, where p s the umber of beta coeffcets the model ( our example p = ).

22 There s a problem (drawback) to usg the leverage to detfy fluetal values - t does ot cota ay formato about the value of the Y varable, oly the value of the X varables. To detect a fluetal observato, a atural statstc to use s a scaled verso of ( yˆ ( j) y ) where ŷ ( j) s the ftted value for the jth observato whe the th observato s omtted from the ft. Ths leads to the so-called Cook's statstc. Fortuately, to obta the value of ths statstc, we do ot eed to carry out a regresso ft, omttg each pot term, for the statstc gve by: D = r p h ( h ) Observatos wth relatvely large values of D are defed as fluetal. Example (cotued): Calculatos of studetsed resduals ad Cook's D statstc for each observato are gve the followg table: ID Observed Predcted Std Err Std. Res Cook's D * ** * *** * **** * * ** * ** **

23 Fgure : Plot of stadardsed resduals agast predcted value of y. 3

24 .4. SOME FINAL COMMENTS (A) INTERPRETATION OF CORRELATION The followg s a extract from D Altma's commets: "Correlato coeffcets le wth the raged - to +, wth the mdpot of zero dcatg o lear assocato betwee the two varables. A very small correlato does ot ecessarly dcate that two varables are ot assocated, however. To be sure of ths, we should study a plot of the data, because t s possble that the two varables dsplay a pecular (.e. o-lear) relatoshp. For example, we should ot observe much, f ay, correlato betwee the average mdday temperature ad caledar moth because there s a cyclc patter. More commo s the stuato of a curved relatoshp betwee two varables, such as betwee brthweght ad legth of gestato. I ths case, Pearso's r wll uderestmate the assocato as t s a measure of lear assocato. The rak correlato coeffcet s better here as t assesses a more geeral way whether the varables ted to rse together (or move opposte drecto). It s surprsg how umpressve a correlato of 0.5 or eve 0.7 s whe a correlato of ths magtude s sgfcat at p<0.05 level wth a sample sze of 9 or 5 subjects. Whether these are mportat s aother matter. Feste commeted o the lack of clcal relevace of a statstcally sgfcat correlato of less tha 0. foud a sample of The problem of clcal relevace s oe that must be judged o ts merts each case, ad depeds o the cotext. For example, the same small correlato may be mportat a epdemologcal study but umportat clcally. Oe way of lookg at the correlato that helps to modfy the over-ethusasm s to calculate the R-square value, whch s the percetage of the varablty of the data that s "explaed" by the assocato betwee the two varables. So, a correlato of 0.7 mples that just 49% of the varablty may be put dow to the observed assocato. Iterpretato of assocato s ofte problematc because causato ca ot be drectly ferred. If we observe a assocato betwee two varables X ad Y, there are several possble explaatos. Excludg the possblty that t s a chace fdg, t may be because: 4

25 X causes (flueces) Y Y flueces X Both X ad Y are flueced by oe or more other varables. Correlato s ofte used as a exploratory method for vestgatg terrelatoshps amog may varables, for whch purpose t s most obvous to use hypothess tests. Although prcple, ths approach s ofte over-doe. The problem s that eve wth a modest umber of varables, the umber of coeffcets s large: 0 varables yeld (0 x 9)/ = 45 r values, ad 0 varables yelds 90 r values. O of the 0 of these wll be sgfcat at the level of 5% purely by chace, ad these ca ot be dstgushed from geue assocato. Furthermore, the magtude of correlato that s sgfcat at 5% s depedet o sample sze. I a large sample,, eve f there are several sgfcat r values of aroud 0. to 0.3, say, these are ulkely to be very useful. Whle ths way of lookg at large umbers of varables ca be helpful whe oe really has o pror hypothess, sgfcat assocato really eeds to be cofrmed aother set of data before credece ca be gve to them. Aother commo problem of terpretato occurs whe we kow that each of two varables s assocated wth a thrd varables. For example, f X s postvely correlated wth Y ad Y s postvely correlated wth Z, t s temptg to say that X ad Z must be postvely correlated. Although ths may deed be true, such a ferece s ujustfed - we ca ot say aythg about the correlato betwee X ad Z. The same s true whe oe has observed o assocato. For example, Mazess et al (984) the correlato betwee age ad heght was 0.05 ad betwee weght ad %fat was Ths does ot mply that the correlato betwee age ad %fat was also ear zero. I fact, ths correlato was Correlato ca ot be ferred from drect assocatos." (B) INTERPRETATION OF REGRESSION The varablty amog a set of observatos may be partly attrbuted to kow factors ad partly to ukow factors; the latter s ofte termed "radom varato". I lear regresso, we see how much of the varablty the respose varable ca be attrbuted to dfferet values of the predctor varable, ad the scatter ether sde of the ftted le shows uexplaed varablty. Because of ths varablty, the ftted le s oly a estmated of the relato betwee these varables the populato. As wth other 5

26 estmates (such as a sample mea) there wll be ucertaty assocated wth the estmated slope ad tercept. The cofdece tervals for the whole le ad predcto tervals for dvdual subjects show other aspect of varablty. The latter are especally useful as regresso s ofte used to make predctos about dvduals. It should be remembered that the regresso le should ot be used to make predctos for X values outsde the rage of values the observed data. Such extrapolato s ujustfed as we have o evdece about the relatoshp beyod the observed data. A statstcal model s oly a approxmato. Oe rarely beleves, for example, that the true relatoshp s exactly lear, but the lear regresso equato s take as a reasoable approxmato for the observed data. Outsde the rage of the observed data oe ca ot safely use the same equato. Thus, we should ot use the regresso equato to predct value beyod what we have observed. 6

27 III. EXERCISES. Cosder the smple regresso equato [8]. Cosder the least square resduals, gve by y y$ where =,, 3,...,. Show that (a) y$ y =. (b) ( y ˆ ) y = 0 = =. The followg data represet dastolc blood pressures take durg rest. The x values deote the legth of tme mutes sce rest bega, ad the y values deote dastolc blood pressures. x: y: (a) Costruct a scatter plot. (b) Fd the coeffcet of correlato ad the lear regresso equato of y o x. (c) Calculate 95% cofdece terval for the slope ad tercept. (d) Calculate 95% cofdece terval for the predcted value of y whe x = The followg table shows restg metabolc rate (RMR) (kcal/4 hr) ad body weght (kg) of 44 wome (Owe et al 986). Wt: RMR: Wt: RMR: Wt: RMR: Wt: RMR: (a) Perform lear regresso aalyss of RMR o body weght. (b) Exame the dstrbuto of resduals. Is the aalyss vald? 7

28 (c) Obta a 95% cofdece terval for the slope of the le. (d) Is t possble to use a dvdual's weght to predct ther RMR to wth 50 kcal/4 hr? 4. Dgox s a drug that s largely elmated uchaged the ure. I a study by Halk et al. 975, whch the authors commeted: (a) Its real clearace was correlated wth create clearace; (b) t clearace was depedet of ure flow. The authors provded data showg measuremets of these three varables from 35 patets beg treated wth dgox for cogestve heart falure. Patet Clearace Create Dgox (ml/m/.73 m ) Ur flow (ml/m)

29 (a) Do these data support statemets (a) ad (b) above? 5. Cosder the followg 4 data sets. Note that X = X = X4. Ft a lear regresso equato wth Y as a depedet varable ad X as a depedet varable. What s the most strkg feature from these data sets. Carry out a resdual plot for each data set ad commet o the result. X Y X Y X3 Y3 X4 Y