CORRELATION AND REGRESSION

Transcription

1 CHAPTER 18 CORRELATION AND REGRESSION After readng ths chapter, students wll be able to understand: LEARNING OBJECTIVES The meanng of bvarate data and technques of preparaton of bvarate dstrbuton; The concept of correlaton between two varables and quanttatve measurement of correlaton ncludng the nterpretaton of postve, negatve and zero correlaton; Concept of regresson and ts applcaton n estmaton of a varable from known set of data. UNIT OVERVIEW Bvarate Data Correlaton Analyss Margnal Dstrbuton Bvarate Frequency Dstrbuton Condtonal Dstrbuton Types of Correlaton Measures of Correlaton Postve Correlaton Negatve Correlaton Scatter Dagram Karl Person Product Moment correlaton Coeffcent Spearmar s Correlaton Coeffcent Coeffcent of Concurrent Devatons Regresson Analyss Estmaton of Regresson Analyss Meod of Least Squares Regresson Lnes Regresson equaton y on x Regresson equaton x on y

2 18. STATISTICS 18.1 INTRODUCTION In the prevous chapter, we dscussed many a statstcal measure relatng to Unvarate dstrbuton.e. dstrbuton of one varable lke heght, weght, mark, proft, wage and so on. However, there are stuatons that demand study of more than one varable smultaneously. A busnessman may be keen to know what amount of nvestment would yeld a desred level of proft or a student may want to know whether performng better n the selecton test would enhance hs or her chance of dong well n the fnal examnaton. Wth a vew to answerng ths seres of questons, we need to study more than one varable at the same tme. Correlaton Analyss and Regresson Analyss are the two analyses that are made from a multvarate dstrbuton.e. a dstrbuton of more than one varable. In partcular when there are two varables, say x and y, we study bvarate dstrbuton. We restrct our dscusson to bvarate dstrbuton only. Correlaton analyss, t may be noted, helps us to fnd an assocaton or the lack of t between the two varables x and y. Thus f x and y stand for proft and nvestment of a frm or the marks n Statstcs and Mathematcs for a group of students, then we may be nterested to know whether x and y are assocated or ndependent of each other. The extent or amount of correlaton between x and y s provded by dfferent measures of Correlaton namely Product Moment Correlaton Coeffcent or Rank Correlaton Coeffcent or Coeffcent of Concurrent Devatons. In Correlaton analyss, we must be careful about a cause and effect relaton between the varables under consderaton because there may be stuatons where x and y are related due to the nfluence of a thrd varable although no causal relatonshp exsts between the two varables. Regresson analyss, on the other hand, s concerned wth predctng the value of the dependent varable correspondng to a known value of the ndependent varable on the assumpton of a mathematcal relatonshp between the two varables and also an average relatonshp between them. 18. BIVARIATE DATA When data are collected on two varables smultaneously, they are known as bvarate data and the correspondng frequency dstrbuton, derved from t, s known as Bvarate Frequency Dstrbuton. If x and y denote marks n Maths and Stats for a group of 30 students, then the correspondng bvarate data would be (x, y ) for 1,,. 30 where (x 1, y 1 ) denotes the marks n Mathematcs and Statstcs for the student wth seral number or Roll Number 1, (x, y ), that for the student wth Roll Number and so on and lastly (x 30, y 30 ) denotes the par of marks for the student bearng Roll Number 30. As n the case of a Unvarate Dstrbuton, we need to construct the frequency dstrbuton for bvarate data. Such a dstrbuton takes nto account the classfcaton n respect of both the varables smultaneously. Usually, we make horzontal classfcaton n respect of x and vertcal classfcaton n respect of the other varable y. Such a dstrbuton s known as Bvarate Frequency Dstrbuton or Jont Frequency Dstrbuton or Two way classfcaton of the two varables x and y.

3 CORRELATION AND REGRESSION 18.3 ILLUSTRATIONS: Example 18.1: Prepare a Bvarate Frequency table for the followng data relatng to the marks n Statstcs (x) and Mathematcs (y): (15, 13), (1, 3), (, 6), (8, 3), (15, 10), (3, 9), (13, 19), (10, 11), (6, 4), (18, 14), (10, 19), (1, 8), (11, 14), (13, 16), (17, 15), (18, 18), (11, 7), (10, 14), (14, 16), (16, 15), (7, 11), (5, 1), (11, 15), (9, 4), (10, 15), (13, 1) (14, 17), (10, 11), (6, 9), (13, 17), (16, 15), (6, 4), (4, 8), (8, 11), (9, 1), (14, 11), (16, 15), (9, 10), (4, 6), (5, 7), (3, 11), (4, 16), (5, 8), (6, 9), (7, 1), (15, 6), (18, 11), (18, 19), (17, 16) (10, 14) Take mutually exclusve classfcaton for both the varables, the frst class nterval beng 0-4 for both. Soluton: From the gven data, we fnd that Range for x Range for y We take the class ntervals 0-4, 4-8, 8-1, 1-16, 16-0 for both the varables. Snce the frst par of marks s (15, 13) and 15 belongs to the fourth class nterval (1-16) for x and 13 belongs to the fourth class nterval for y, we put a stroke n the (4, 4)-th cell. We carry on gvng tally marks tll the lst s exhausted.

4 18.4 STATISTICS Table 18.1 Bvarate Frequency Dstrbuton of Marks n Statstcs and Mathematcs. X MARKS IN STATS MARKS IN MATHS Y Total 0 4 I (1) I (1) II () I (1) IIII (4) IIII (5) I (1) I (1) I (1) II () IIII (4) IIII I (6) I (1) I (1) III (3) II () IIII (5) I (1) IIII (5) III (3) 9 Total We note, from the above table, that some of the cell frequences (f j ) are zero. Startng from the above Bvarate Frequency Dstrbuton, we can obtan two types of unvarate dstrbutons whch are known as: (a) (b) Margnal dstrbuton. Condtonal dstrbuton. If we consder the dstrbuton of Statstcs marks along wth the margnal totals presented n the last column of Table 1-1, we get the margnal dstrbuton of marks n Statstcs. Smlarly, we can obtan one more margnal dstrbuton of Mathematcs marks. The followng table shows the margnal dstrbuton of marks of Statstcs. Table 18. Margnal Dstrbuton of Marks n Statstcs Marks No. of Students Total 50 We can fnd the mean and standard devaton of marks n Statstcs from Table 18.. They would be known as margnal mean and margnal SD of Statstcs marks. Smlarly, we can obtan the margnal mean and margnal SD of Mathematcs marks. Any other statstcal measure n respect of x or y can be computed n a smlar manner.

5 CORRELATION AND REGRESSION 18.5 If we want to study the dstrbuton of Statstcs Marks for a partcular group of students, say for those students who got marks between 8 to 1 n Mathematcs, we come across another unvarate dstrbuton known as condtonal dstrbuton. Table 18.3 Condtonal Dstrbuton of Marks n Statstcs for Students havng Mathematcs Marks between 8 to 1 Marks No. of Students Total 15 We may obtan the mean and SD from the above table. They would be known as condtonal mean and condtonal SD of marks of Statstcs. The same result holds for marks n Mathematcs. In partcular, f there are m classfcatons for x and n classfcatons for y, then there would be altogether (m + n) condtonal dstrbuton CORRELATION ANALYSIS Whle studyng two varables at the same tme, f t s found that the change n one varable s recprocated by a correspondng change n the other varable ether drectly or nversely, then the two varables are known to be assocated or correlated. Otherwse, the two varables are known to be dssocated or uncorrelated or ndependent. There are two types of correlaton. () Postve correlaton () Negatve correlaton If two varables move n the same drecton.e. an ncrease (or decrease) on the part of one varable ntroduces an ncrease (or decrease) on the part of the other varable, then the two varables are known to be postvely correlated. As for example, heght and weght yeld and ranfall, proft and nvestment etc. are postvely correlated. On the other hand, f the two varables move n the opposte drectons.e. an ncrease (or a decrease) on the part of one varable results a decrease (or an ncrease) on the part of the other varable, then the two varables are known to have a negatve correlaton. The prce and demand of an tem, the profts of Insurance Company and the number of clams t has to meet etc. are examples of varables havng a negatve correlaton. The two varables are known to be uncorrelated f the movement on the part of one varable does not produce any movement of the other varable n a partcular drecton. As for example, Shoesze and ntellgence are uncorrelated.

6 18.6 STATISTICS 18.4 MEASURES OF CORRELATION We consder the followng measures of correlaton: (a) Scatter dagram (b) Karl Pearson s Product moment correlaton coeffcent (c) Spearman s rank correlaton co-effcent (d) Co-effcent of concurrent devatons (a) SCATTER DIAGRAM Ths s a smple dagrammatc method to establsh correlaton between a par of varables. Unlke product moment correlaton co-effcent, whch can measure correlaton only when the varables are havng a lnear relatonshp, scatter dagram can be appled for any type of correlaton lnear as well as non-lnear.e. curvlnear. Scatter dagram can dstngush between dfferent types of correlaton although t fals to measure the extent of relatonshp between the varables. Each data pont, whch n ths case a par of values (x, y ) s represented by a pont n the rectangular axes of cordnates. The totalty of all the plotted ponts forms the scatter dagram. The pattern of the plotted ponts reveals the nature of correlaton. In case of a postve correlaton, the plotted ponts le from lower left corner to upper rght corner, n case of a negatve correlaton the plotted ponts concentrate from upper left to lower rght and n case of zero correlaton, the plotted ponts would be equally dstrbuted wthout depctng any partcular pattern. The followng fgures show dfferent types of correlaton and the one to one correspondence between scatter dagram and product moment correlaton coeffcent. FIGURE 18.1 FIGURE 18. Showng Postve Correlaton Showng Perfect Correlaton (0 < r <1) (r 1)

7 CORRELATION AND REGRESSION 18.7 FIGURE 18.3 FIGURE 18.4 Showng Negatve Showng Perfect Negatve Correlaton Correlaton ( 1 < r <0) (r 1) FIGURE 18.5 FIGURE 18.6 Showng No Showng Curvlnear Correlaton Correlaton (r 0) (r 0) (b) KARL PEARSON S PRODUCT MOMENT CORRELATION COEFFICIENT Ths s by for the best method for fndng correlaton between two varables provded the relatonshp between the two varables s lnear. Pearson s correlaton coeffcent may be defned as the rato of covarance between the two varables to the product of the standard devatons of the two varables. If the two varables are denoted by x and y and f the correspondng bvarate data are (x, y ) for 1,, 3,.., n, then the coeffcent of correlaton between x and y, due to Karl Pearson, n gven by :

8 18.8 STATISTICS r r xy Cov x, y S x S y...(18.1) where cov (x, y) x x (y y) xy x y...(18.) n n x x x S x x n n...(18.3) y y y S y y...(18.4) n n and A sngle formula for computng correlaton coeffcent s gven by r nx y x y nx x n y ( y )...(18.5) In case of a bvarate frequency dstrbuton, we have Cov(x,y) x y f,j N j x y... (18.6) S x f x o N x...(18.7) and fojy j S y N y j...(18.8) where x Md-value of the th class nterval of x.

9 CORRELATION AND REGRESSION 18.9 y j f o f oj f j Md-value of the j th class nterval of y Margnal frequency of x Margnal frequency of y frequency of the (, j) th cell N f j f o f oj Total frequency... (18.9),j PROPERTIES OF CORRELATION COEFFICIENT () () The Coeffcent of Correlaton s a unt-free measure. j Ths means that f x denotes heght of a group of students expressed n cm and y denotes ther weght expressed n kg, then the correlaton coeffcent between heght and weght would be free from any unt. The coeffcent of correlaton remans nvarant under a change of orgn and/or scale of the varables under consderaton dependng on the sgn of scale factors. Ths property states that f the orgnal par of varables x and y s changed to a new par of varables u and v by effectng a change of orgn and scale for both x and y.e. x a y c u and v b d where a and c are the orgns of x and y and b and d are the respectve scales and then we have bd r xy r b d u v...(18.10) r xy and r uv beng the coeffcent of correlaton between x and y and u and v respectvely, (18.10) establshed, numercally, the two correlaton coeffcents reman equal and they would have opposte sgns only when b and d, the two scales, dffer n sgn. () The coeffcent of correlaton always les between 1 and 1, ncludng both the lmtng values.e. 1 r 1...(18.11) Example 18.: Compute the correlaton coeffcent between x and y from the followng data n 10, xy 0, x 00, y 6 x 40 and y 50

10 18.10 STATISTICS Soluton: From the gven data, we have by applyng (18.5), r nxy x y nx x n y y (40) 10 6 (50) Thus there s a good amount of postve correlaton between the two varables x and y. Alternately As gven, x 40 x 4 n 10 y 50 y 5 n 10 Cov (x, y) xy x. y n S x x (x) n

11 CORRELATION AND REGRESSION S y y y n Thus applyng formula (18.1), we get r cov(x, y) Sx. S y As before, we draw the same concluson. Example 18.3: Fnd product moment correlaton coeffcent from the followng nformaton: x : y : Soluton: In order to fnd the covarance and the two standard devaton, we prepare the followng table: Table 18.3 Computaton of Correlaton Coeffcent x y x y x y (1) () (3) (1) x () (4) (1) (5) ()

12 18.1 STATISTICS We have 9 x y cov (x, y) xy x y n 166/ (x) x n 163 (4.8333) S y y (y) n 79 (6.50) Thus the correlaton coeffcent between x and y n gven by r cov (x, y) S s x y We fnd a hgh degree of negatve correlaton between x and y. Also, we could have appled formula (18.5) as we have done for the frst problem of computng correlaton coeffcent. Sometmes, a change of orgn reduces the computatonal labor to a great extent. Ths we are gong to do n the next problem.

13 CORRELATION AND REGRESSION Example 18.4: The followng data relate to the test scores obtaned by eght salesmen n an apttude test and ther daly sales n thousands of rupees: Salesman : Soluton: scores : Sales : Let the scores and sales be denoted by x and y respectvely. We take a, orgn of x as the average of the two extreme values.e. 54 and 70. Hence a 6 smlarly, the orgn of y s taken as b Table 18.4 Computaton of Correlaton Coeffcent Between Test Scores and Sales. Scores Sales n u v u v u v (x ) ` 1000 x 6 y 30 (1) (y ) () (3) (4) (5)(3)x(4) (6)(3) (7)(4) Total Snce correlaton coeffcent remans unchanged due to change of orgn, we have r r xy r uv n u v u v n u u n v v 8 90 ( 13) ( 14) 8 1 ( 13) 8 1 ( 14)

14 18.14 STATISTICS In some cases, there may be some confuson about selectng the par of varables for whch correlaton s wanted. Ths s explaned n the followng problem. Example 18.5: Examne whether there s any correlaton between age and blndness on the bass of the followng data: Age n years : No. of Persons (n thousands) : No. of blnd Persons : Soluton: Let us denote the md-value of age n years as x and the number of blnd persons per lakh as y. Then as before, we compute correlaton coeffcent between x and y. Table 18.5 Computaton of correlaton between age and blndness Age n Md-value No. of No. of No. of xy x y years x Persons blnd blnd per () (5) () (5) (1) () ( 000) B lakh (6) (7) (8) P (4) yb/p 1 lakh (3) (5) Total

15 CORRELATION AND REGRESSION The correlaton coeffcent between age and blndness s gven by n xy x. y r n x ( x) n y ( y) (30) (150) whch exhbts a very hgh degree of postve correlaton between age and blndness. Example 18.6: Coeffcent of correlaton between x and y for 0 tems s 0.4. The AM s and SD s of x and y are known to be 1 and 15 and 3 and 4 respectvely. Later on, t was found that the par (0, 15) was wrongly taken as (15, 0). Fnd the correct value of the correlaton coeffcent. Soluton: We are gven that n 0 and the orgnal r 0.4, x 1, y 15, S x 3 and S y 4 r cov (x, y) cov(x, y) 0.4 S S 3 4 x y Cov (x, y) 4.8 xy x y4.8 n xy xy 3696 Hence, corrected xy Also, S x 9 (x / 0) 1 9 x 3060

16 18.16 STATISTICS Smlarly, S y 16 S y y y 480 Thus corrected x n x wrong x value + correct x value Smlarly correctedy Corrected x Corrected y Thus corrected value of the correlaton coeffcent by applyng formula (18.5) (45) (95) Example 18.7: Compute the coeffcent of correlaton between marks n Statstcs and Mathematcs for the bvarate frequency dstrbuton shown n Table 18.6 Soluton: For the sake of computatonal advantage, we effect a change of orgn and scale for both the varable x and y. Defne u x a x 10 b 4 And v j y c y 10 d 4 Where x and y j denote respectvely the md-values of the x-class nterval and y-class nterval respectvely. The followng table shows the necessary calculaton on the rght top corner of each cell, the product of the cell frequency, correspondng u value and the respectve v value has been shown. They add up n a partcular row or column to provde the value of f j u v j for that partcular row or column.

17 CORRELATION AND REGRESSION Table 18.6 Computaton of Correlaton Coeffcent Between Marks of Mathematcs and Statstcs Class Interval Md-value Class Md V j f o f o u f o u f j u v j Interval -value u f oj f oj v j f oj v j f j u v j CHECK A sngle formula for computng correlaton coeffcent from bvarate frequency dstrbuton s gven by r N f u v f u f v,j j j o o j j o o oj j oj j N f u f u f v f v...(18.10) The value of r shown a good amount of postve correlaton between the marks n Statstcs and Mathematcs on the bass of the gven data.

18 18.18 STATISTICS Example 18.8: Gven that the correlaton coeffcent between x and y s 0.8, wrte down the correlaton coeffcent between u and v where () u + 3x and 4v + 16y () u 3x and 4v + 16y () u 3x and 4v 16y (v) u + 3x and 4v 16y Soluton: Usng (18.10), we fnd that r xy bd b d ruv.e. r xy r uv f b and d are of same sgn and r uv r xy when b and d are of opposte sgns, b and d beng the scales of x and y respectvely. In (), u ( ) + (-3/) x and v ( 11/4) + ( 4)y. Snce b 3/ and d 4 are of same sgn, the correlaton coeffcent between u and v would be the same as that between x and y.e. r xy 0.8 r uv In (), u ( ) + (3/)x and v ( 11/4) + ( 4)y Hence b 3/ and d 4 are of opposte sgns and we have r uv r xy 0.8 Proceedng n a smlar manner, we have r uv 0.8 and 0.8 n () and (v). (c) SPEARMAN S RANK CORRELATION COEFFICIENT When we need fndng correlaton between two qualtatve characterstcs, say, beauty and ntellgence, we take recourse to usng rank correlaton coeffcent. Rank correlaton can also be appled to fnd the level of agreement (or dsagreement) between two judges so far as assessng a qualtatve characterstc s concerned. As compared to product moment correlaton coeffcent, rank correlaton coeffcent s easer to compute, t can also be advocated to get a frst hand mpresson about the correlaton between a par of varables. Spearman s rank correlaton coeffcent s gven by r R 1 6 d n(n... (18.11) 1) where r R denotes rank correlaton coeffcent and t les between 1 and 1 nclusve of these two values. d x y represents the dfference n ranks for the -th ndvdual and n denotes the number of ndvduals. In case u ndvduals receve the same rank, we descrbe t as a ted rank of length u. In case of a ted rank, formula (18.11) s changed to

19 CORRELATION AND REGRESSION r R 1 3 tj tj 6 d + j 1... (18.1) n n 1 In ths formula, t j represents the j th te length and the summaton (t 3 j t j) extends over the lengths of all the tes for both the seres. Example 18.9: compute the coeffcent of rank correlaton between sales and advertsement expressed n thousands of rupees from the followng data: Sales : Advertsement : Soluton: Let the rank gven to sales be denoted by x and rank of advertsement be denoted by y. We note that snce the hghest sales as gven n the data, s 95, t s to be gven rank 1, the second hghest sales 90 s to be gven rank and fnally rank 8 goes to the lowest sales, namely 68. We have gven rank to the other varable advertsement n a smlar manner. Snce there are no tes, we apply formula (16.11). Table 18.7 Computaton of Rank correlaton between Sales and Advertsement. Sales Advertsement Rank for Rank for d x y d (x ) (y ) Sales (x ) Advertsement (y ) Total 0 4 j

20 18.0 STATISTICS Snce n 8 and d 4, applyng formula (18.11), we get. r R 1 6 d n(n 1) (8 1) The hgh postve value of the rank correlaton coeffcent ndcates that there s a very good amount of agreement between sales and advertsement. Example 18.10: Compute rank correlaton from the followng data relatng to ranks gven by two judges n a contest: Seral No. of Canddate : Rank by Judge A : Rank by Judge B : Soluton: We drectly apply formula (18.11) as ranks are already gven. Table 18.8 Computaton of Rank Correlaton Coeffcent between the ranks gven by Judges Seral No. Rank by A (x ) Rank by B (y ) d x y d Total 0 166

21 CORRELATION AND REGRESSION 18.1 The rank correlaton coeffcent s gven by r R 1 6 d n(n 1) (10 1) The very low value (almost 0) ndcates that there s hardly any agreement between the ranks gven by the two Judges n the contest. Example 18.11: Compute the coeffcent of rank correlaton between Eco. marks and stats. Marks as gven below: Eco Marks : Stats Marks : Soluton: Ths s a case of ted ranks as more than one student share the same mark both for Economcs and Statstcs. For Eco. the student recevng 80 marks gets rank 1 one gettng 6 marks receves rank, the student wth 60 receves rank 3, student wth 56 marks gets rank 4 and snce there are two students, each gettng 50 marks, each would be recevng a common rank, the average of the next two ranks 5 and 6.e e and lastly the last rank.. 7 goes to the student gettng the lowest Eco marks. In a smlar manner, we award ranks to the students wth stats marks. Table 18.9 Computaton of Rank Correlaton Between Eco Marks and Stats Marks wth Ted Marks Eco Mark Stats Mark Rank for Eco Rank for Stats d x y d (x ) (y ) (x ) (y ) Total

22 18. STATISTICS For Economcs mark there s one te of length and for stats mark, there are two tes of lengths and 3 respectvely. Thus t 3 j 1 t j Thus r R tj 6 d + j 1 1 n n 1 3 tj 6 ( ) 1 7(7 1) 0.15 Example 18.1: For a group of 8 students, the sum of squares of dfferences n ranks for Mathematcs and Statstcs marks was found to be 50 what s the value of rank correlaton coeffcent? Soluton: As gven n 8 and d 50. Hence the rank correlaton coeffcent between marks n Mathematcs and Statstcs s gven by 1 6 d n n 1 r R (8 1) 0.40 Example 18.13: For a number of towns, the coeffcent of rank correlaton between the people lvng below the poverty lne and ncrease of populaton s If the sum of squares of the dfferences n ranks awarded to these factors s 8.50, fnd the number of towns. Soluton: As gven r R 0.50, d d n n 1 Thus r R

23 CORRELATION AND REGRESSION n n n (n 1) 990 n (n 1) 10(10 1) n 10 as n must be a postve nteger. Example 18.14: Whle computng rank correlaton coeffcent between profts and nvestment for 10 years of a frm, the dfference n rank for a year was taken as 7 nstead of 5 by mstake and the value of rank correlaton coeffcent was computed as What would be the correct value of rank correlaton coeffcent after rectfyng the mstake? Soluton: We are gven that n 10, r R 0.80 and the wrong d 7 should be replaced by d n n 1 r R 1 6 d d 33 Corrected d Hence rectfed value of rank correlaton coeffcent (d) COEFFICIENT OF CONCURRENT DEVIATIONS A very smple and casual method of fndng correlaton when we are not serous about the magntude of the two varables s the applcaton of concurrent devatons. Ths method nvolves n attachng a postve sgn for a x-value (except the frst) f ths value s more than the prevous value and assgnng a negatve value f ths value s less than the prevous value. Ths s done for the y-seres as well. The devaton n the x-value and the correspondng y-value s known to be concurrent f both the devatons have the same sgn.

24 18.4 STATISTICS Denotng the number of concurrent devaton by c and total number of devatons as m (whch must be one less than the number of pars of x and y values), the coeffcent of concurrent devaton s gven by r C + c m...(18.13) m If (c m) >0, then we take the postve sgn both nsde and outsde the radcal sgn and f (c m) <0, we are to consder the negatve sgn both nsde and outsde the radcal sgn. Lke Pearson s correlaton coeffcent and Spearman s rank correlaton coeffcent, the coeffcent of concurrent devatons also les between 1 and 1, both nclusve. Example 18.15: Fnd the coeffcent of concurrent devatons from the followng data. Year : Prce : Demand : Soluton: Table Computaton of Coeffcent of Concurrent Devatons. Year Prce Sgn of Demand Sgn of Product of devaton devaton from devaton from the the prevous (ab) prevous fgure (b) fgure (a) In ths case, m number of pars of devatons 7 c No. of postve sgns n the product of devaton column Number of concurrent devatons

25 CORRELATION AND REGRESSION 18.5 Thus r C ± ± c m m 4 7 ± ± m 3 ± ± c m 3 (Snce we take negatve sgn both nsde and outsde of the radcal sgn) m 7 Thus there s a negatve correlaton between prce and demand REGRESSION ANALYSIS In regresson analyss, we are concerned wth the estmaton of one varable for a gven value of another varable (or for a gven set of values of a number of varables) on the bass of an average mathematcal relatonshp between the two varables (or a number of varables). Regresson analyss plays a very mportant role n the feld of every human actvty. A busnessman may be keen to know what would be hs estmated proft for a gven level of nvestment on the bass of the past records. Smlarly, an outgong student may lke to know her chance of gettng a frst class n the fnal Unversty Examnaton on the bass of her performance n the college selecton test. When there are two varables x and y and f y s nfluenced by x.e. f y depends on x, then we get a smple lnear regresson or smple regresson. y s known as dependent varable or regresson or explaned varable and x s known as ndependent varable or predctor or explanator. In the prevous examples snce proft depends on nvestment or performance n the Unversty Examnaton s dependent on the performance n the college selecton test, proft or performance n the Unversty Examnaton s the dependent varable and nvestment or performance n the selecton test s the In-dependent varable. In case of a smple regresson model f y depends on x, then the regresson lne of y on x n gven by y a + bx (18.14) Here a and b are two constants and they are also known as regresson parameters. Furthermore, b s also known as the regresson coeffcent of y on x and s also denoted by b yx. We may defne

26 18.6 STATISTICS the regresson lne of y on x as the lne of best ft obtaned by the method of least squares and used for estmatng the value of the dependent varable y for a known value of the ndependent varable x. The method of least squares nvolves n mnmzng e (y y^ ) (y a bx ). (18.15) where y demotes the actual or observed value and y^ a + b x, the estmated value of y for a gven value of x, e s the dfference between the observed value and the estmated value and e s techncally known as error or resdue. Ths summaton ntends over n pars of observatons of (x, y ). The lne of regresson of y or x and the errors of estmaton are shown n the followng fgure. FIGURE 18.7 SHOWING REGRESSION LINE OF y on x AND ERRORS OF ESTIMATION Mnmsaton of (18.15) yelds the followng equatons known as Normal Equatons. y na + bx.. (18.16) x y ax + b x..... (18.17) Solvng there two equatons for b and a, we have the least squares estmates of b and a as Cov(x, y) b S r.s x.s x y Sx

27 CORRELATION AND REGRESSION 18.7 r.s S x y...(18.18) After estmatng b, estmate of a s gven by ay bx...(18.19) Substtutng the estmates of b and a n (18.14), we get y y r x x S y S x...(18.0) There may be cases when the varable x depends on y and we may take the regresson lne of x on y as x a^+ b^y Unlke the mnmzaton of vertcal dstances n the scatter dagram as shown n fgure (18.7) for obtanng the estmates of a and b, n ths case we mnmze the horzontal dstances and get the followng normal equaton n a^ and b^, the two regresson parameters : x na^ + b^y... (18.1) x y a^y + b^ y..... (18.) or solvng these equatons, we get b^ b xy cov(x, y) r.s S S y y x...(18.3) and a x - b y.... (18.4) A sngle formula for estmatng b s gven by n xy x. y b^ b yx n y ( y )...(18.5) n xy x. y Smlarly, b^ b yx n y ( y )...(18.6) The standardzed form of the regresson equaton of x on y, as n (18.0), s gven by

28 18.8 STATISTICS x x r S x y y S y... (18.7) Example 16.15: Fnd the two regresson equatons from the followng data: x: y: Hence estmate y when x s 13 and estmate also x when y s 15. Soluton: Table Computaton of Regresson Equatons x y x y x y On the bass of the above table, we have x 34 x n 6 y 56 y n 6 cov (x, y) xy x y n x S x x n

29 CORRELATION AND REGRESSION (5.6667) S y y y n 554 (9.3333) The regresson lne of y on x s gven by y a + bx cov(x, y) Where b^ S x and a y b x x Thus the estmated regresson equaton of y on x s y x When x 13, the estmated value of y s gven by ŷ The regresson lne of x on y s gven by x a^ + b^ y Where b^ cov x, y S y

30 18.30 STATISTICS and a^ x b y Thus the estmated regresson lne of x on y s x y When y 15, the estmate value of x s gven by ˆx Example 18.16: Marks of 8 students n Mathematcs and statstcs are gven as: Mathematcs: Statstcs: Fnd the regresson lnes. When marks of a student n Mathematcs are 90, what are hs most lkely marks n statstcs? Soluton: We denote the marks n Mathematcs and Statstcs by x and y respectvely. We are to fnd the regresson equaton of y on x and also of x or y. Lastly, we are to estmate y when x 90. For computaton advantage, we shft orgns of both x and y. Table 18.1 Computaton of regresson lnes Maths Stats u v u v mark (x ) mark (y ) x 74 y 76 u v

31 CORRELATION AND REGRESSION The regresson coeffcents b (or b yx ) and b (or b xy ) reman unchanged due to a shft of orgn. Applyng (18.5) and (18.6), we get n uv u. v b b yx b vu n u ( u ) 8.(71) (3).( 13) 8.(43) (3) n uv u. v and b^ b xy b uv n v ( v ) 8.(71) (3).( 13) 8.(559) ( 13) Also a^ y b x (595) (595) and a^ x b y The regresson lne of y on x s y x and the regresson lne of x on y s x y

32 18.3 STATISTICS For x 90, the most lkely value of y s ŷ x Example 18.17: The followng data relate to the mean and SD of the prces of two shares n a stock Exchange: Share Mean (n `) SD (n `) Company A Company B Coeffcent of correlaton between the share prces 0.48 Fnd the most lkely prce of share A correspondng to a prce of ` 60 of share B and also the most lkely prce of share B for a prce of ` 50 of share A. Soluton: Denotng the share prces of Company A and B respectvely by x and y, we are gven that x ` 44, y ` 58 S x ` 5.60, S y ` 6.30 and r 0.48 The regresson lne of y on x s gven by y a + bx Where b S r S y x a y bx ` ( ) ` 34.4 Thus the regresson lne of y on x.e. the regresson lne of prce of share B on that of share A s gven by y ` ( x) When x ` 50, ` ( )

33 CORRELATION AND REGRESSION ` 61.4 Agan the regresson lne of x on y s gven by x a^ + b^y The estmated prce of share B for a prce of ` 50 of share A s ` 61.4 Where b^ S r S x y a^ x b y ` ( ) ` 19.5 Hence the regresson lne of x on y.e. the regresson lne of prce of share A on that of share B n gven by x ` ( y) When y ` 60, ˆx ` ( ) ` Example 18.18: The followng data relate the expendture or advertsement n thousands of rupees and the correspondng sales n lakhs of rupees. Expendture on Ad : Sales : Fnd an approprate regresson equaton. Soluton: Snce sales (y) depend on advertsement (x), the approprate regresson equaton s of y on x.e. of sales on advertsement. We have, on the bass of the gven data, n 5, x y xy x b n y x y n x x

34 18.34 STATISTICS a y bx Thus, the regresson lne of y or x.e. the regresson lne of sales on advertsement s gven by y x 18.6 PROPERTIES OF REGRESSION LINES We consder the followng mportant propertes of regresson lnes: () The regresson coeffcents reman unchanged due to a shft of orgn but change due to a shft of scale. Ths property states that f the orgnal par of varables s (x, y) and f they are changed to the par (u, v) where xa yc u and v p q b yx q b vu p. (18.8) and bxy p b uv q (18.9) () The two lnes of regresson ntersect at the pont x,y, where x and y are the varables under consderaton. Accordng to ths property, the pont of ntersecton of the regresson lne of y on x and the regresson lne of x on y s x,y.e. the soluton of the smultaneous equatons n x and y.

35 CORRELATION AND REGRESSION () The coeffcent of correlaton between two varables x and y n the smple geometrc mean of the two regresson coeffcents. The sgn of the correlaton coeffcent would be the common sgn of the two regresson coeffcents. Ths property says that f the two regresson coeffcents are denoted by b yx (b) and b xy (b ) then the coeffcent of correlaton s gven by r ± b b.. (18.30) yx xy If both the regresson coeffcents are negatve, r would be negatve and f both are postve, r would assume a postve value. Example 18.19: If the relatonshp between two varables x and u s u + 3x 10 and between two other varables y and v s y + 5v 5, and the regresson coeffcent of y on x s known as 0.80, what would be the regresson coeffcent of v on u? Soluton: u + 3x 10 x10/3 u 1/3 and y + 5v 5 From v y5/ 5/ (16.8), we have q b yx b p vu or, 5/ 0.80 b 1/ b vu vu 8 b vu Example 18.0: For the varables x and y, the regresson equatons are gven as 7x 3y 18 0 and 4x y 11 0 () Fnd the arthmetc means of x and y. () Identfy the regresson equaton of y on x.

36 18.36 STATISTICS () Compute the correlaton coeffcent between x and y. (v) Gven the varance of x s 9, fnd the SD of y. Soluton: () Snce the two lnes of regresson ntersect at the pont (x, y), replacng x and y by x and y respectvely n the gven regresson equatons, we get 7 x 3y 180 and 4 x y 110 Solvng these two equatons, we get x 3 and y 1 Thus the arthmetc means of x and y are gven by 3 and 1 respectvely. () Let us assume that 7x 3y 18 0 represents the regresson lne of y on x and 4x y 11 0 represents the regresson lne of x on y. Now 7x 3y y 6 + x 3 7 b yx 3 Agan 4x y x + y b xy Thus r b yx b xy < 1 1 Snce r 1 r 1, our assumptons are correct. Thus, 7x 3y 18 0 truly represents the regresson lne of y on x. () Snce r 7 1

37 CORRELATION AND REGRESSION r 7 1 (We take the sgn of r as postve snce both the regresson coeffcents are postve) (v) b yx S r S y x 7 3 S y ( S x 9 as gven) S y PROBABLE ERROR The correlaton coeffcent calculated from the sample of n pars of value from large populaton. It s possble to determne the lmts of the correlaton coeffcent of populaton and whch coeffcent of correlatonof correlaton of the populaton wll le from the knowledge of sample correlaton coeffcent. Probable Error s a method of obtanng correlaton coeffcent of populaton. It s defned as: 1 r P.E N Where r Correlaton coeffcent fromn pars of sample observatons PE 3 SE When SE Standard Error of correlaton coeffcent 1 r S.E N The lmt of the correlaton coeffcent s gven by p r ± P.E Where p Correlaton coeffcent of the populaton The followng are the assumpton whle probable Errors are sgnfcant. () If r< PE there s no evdence of correlaton () If the value of r s more than 6 tmes of the probable error, then the presence of correlaton coeffcent s certan () Snce r les between -1 and +1 (-1 < r < 1) the probable error s never negatve.

38 18.38 STATISTICS Note: The formula PE s valued onlyf (1) The sample chooses to fnd r s a sample random sample () the populaton s normal. Example 18.1: Compute the Probable Error assumng the correlaton coeffcent of 0.8 from a sampleof 5 pars of tems. Soluton: r 0.8,n 5 P.E Example 18.: If r 0.7 ; and n 64 fnd out the probable error of the coeffcent of correlatonand determne the lmts for the populaton correlaton coeffcent: Soluton: r 0.7 ; n (0.7) Probable Error (P.E.) (0.6745) ( ) Lmts for the populaton correlaton coeffcent (0.7 ± 0.043).e. (0.743, 0.657) 18.8 REVIEW OF CORRELATION AND REGRESSION ANALYSIS So far we have dscussed the dfferent measures of correlaton and also how to ft regresson lnes applyng the method of Least Squares. It s obvous that we take recourse to correlaton analyss when we are keen to know whether two varables under study are assocated or correlated and f correlated, what s the strength of correlaton. The best measure of correlaton s provded by Pearson s correlaton coeffcent. However, one severe lmtaton of ths correlaton coeffcent, as we have already dscussed, s that t s applcable only n case of a lnear relatonshp between the two varables. If two varables x and y are ndependent or uncorrelated then obvously the correlaton coeffcent between x and y s zero. However, the converse of ths statement s not necessarly true.e. f the correlaton coeffcent, due to Pearson, between two varables comes out to be zero, then we cannot conclude that the two varables are ndependent. All that we can conclude s that no lnear relatonshp exsts between the two varables. Ths, however, does not rule out the exstence of some non lnear relatonshp between the two varables. For example, f we consder the followng pars of values on two varables x and y. (, 4), ( 1, 1), (0, 0), (1, 1) and (, 4), then cov (x, y) ( + 4) + ( 1+1) + (0 0) + (1 1) + ( 4) 0 as x 0

39 CORRELATION AND REGRESSION Thus r xy 0 Ths does not mean that x and y are ndependent. In fact the relatonshp between x and y s y x. Thus t s always wser to draw a scatter dagram before reachng concluson about the exstence of correlaton between a par of varables. There are some cases when we may fnd a correlaton between two varables although the two varables are not causally related. Ths s due to the exstence of a thrd varable whch s related to both the varables under consderaton. Such a correlaton s known as spurous correlaton or non-sense correlaton. As an example, there could be a postve correlaton between producton of rce and that of ron n Inda for the last twenty years due to the effect of a thrd varable tme on both these varables. It s necessary to elmnate the nfluence of the thrd varable before computng correlaton between the two orgnal varables. Correlaton coeffcent measurng a lnear relatonshp between the two varables ndcates the amount of varaton of one varable accounted for by the other varable. A better measure for ths purpose s provded by the square of the correlaton coeffcent, Known as coeffcent of determnaton. Ths can be nterpreted as the rato between the explaned varance to total varance.e. Explaned varance r Total varance Thus a value of 0.6 for r ndcates that (0.6) 100% or 36 per cent of the varaton has been accounted for by the factor under consderaton and the remanng 64 per cent varaton s due to other factors. The coeffcent of non-determnaton s gven by (1 r ) and can be nterpreted as the rato of unexplaned varance to the total varance. Coeffcent of non-determnaton (1 r ) Regresson analyss, as we have already seen, s concerned wth establshng a functonal relatonshp between two varables and usng ths relatonshp for makng future projecton. Ths can be appled, unlke correlaton for any type of relatonshp lnear as well as curvlnear. The two lnes of regresson concde.e. become dentcal when r 1 or 1 or n other words, there s a perfect negatve or postve correlaton between the two varables under dscusson. If r 0 Regresson lnes are perpendcular to each other. SUMMARY? The change n one varable s recprocated by a correspondng change n the other varable ether drectly or nversely, then the two varables are known to be assocated or correlated. There are two types of correlaton. () Postve correlaton () Negatve correlaton? We consder the followng measures of correlaton:

40 18.40 STATISTICS (a) Scatter dagram: Ths s a smple dagrammatc method to establsh correlaton between a par of varables. (b) Karl Pearson s Product moment correlaton coeffcent: Cov(x,y) r r xy S S x y A sngle formula for computng correlaton coeffcent s gven by r n xy x y n x x n y () The Coeffcent of Correlaton s a unt-free measure. () The coeffcent of correlaton remans nvarant under a change of orgn and/or scale of the varables under consderaton dependng on the sgn of scale factors. () The coeffcent of correlaton always les between 1 and 1, ncludng both the lmtng values.e. 1 < r < + 1 (c) Spearman s rank correlaton co-effcent: Spearman s rank correlaton coeffcent s gven by 6 d r R 1 n(n, where rr denotes rank correlaton coeffcent and t les between 1) 1 and 1 nclusve of these two values.d x y represents the dfference n ranks for the -th ndvdual and n denotes the number of ndvduals. In case u ndvduals receve the same rank, we descrbe t as a ted rank of length u. In case of a ted rank, r R 1 6 d n(n 1) j tj 3 t 1 j In ths formula, t j represents the j th te length and the summaton extends over the lengths of all the tes for both the seres. (d) Co-effcent of concurrent devatons: The coeffcent of concurrent devaton s gven by r C c m m If (c m) >0, then we take the postve sgn both nsde and outsde the radcal sgn and f (c m) <0, we are to consder the negatve sgn both nsde and outsde the radcal sgn.

41 CORRELATION AND REGRESSION In regresson analyss, we are concerned wth the estmaton of one varable for gven value of another varable (or for a gven set of values of a number of varables) on the bass of an average mathematcal relatonshp between the two varables (or a number of varables). In case of a smple regresson model f y depends on x, then the regresson lne of y on x n gven by y a + b, here a and b are two constants and they are also known as regresson parameters. Furthermore, b s also known as the regresson coeffcent of y on x and s also denoted by b yx The method of least squares s solvng the equatons of regresson lnes The normal equatons are y na + bx x y ax + bx Solvng the normal equatons b cov(x y ) x S r.s.s x y Sx The regresson coeffcents reman unchanged due to a shft of orgn but change due to a shft of scale. Ths property states that f the orgnal par of varables s (x, y) and f they are changed to the par (u, v) where x a y c u and v p q b yx p b q b p vu q and bxy uv The two lnes of regresson ntersect at the pont, where x and y are the varables under consderaton. Accordng to ths property, the pont of ntersecton of the regresson lne of y on x and the regresson lne of x on y s.e. the soluton of the smultaneous equatons n x and y. The coeffcent of correlaton between two varables x and y n the smple geometrc mean of the two regresson coeffcents. The sgn of the correlaton coeffcent would be the common sgn of the two regresson coeffcents. r b b yx xy Correlaton coeffcent measurng a lnear relatonshp between the two varables ndcates the amount of varaton of one varable accounted for by the other varable. A better measure for ths purpose s provded by the square of the correlaton coeffcent, Known

42 18.4 STATISTICS as coeffcent of determnaton. Ths can be nterpreted as the rato between the explaned varance to total varance.e. Explaned varance r Total varance The coeffcent of non-determnaton s gven by (1 r ) and can be nterpreted as the rato of unexplaned varance to the total varance. The two lnes of regresson concde.e. become dentcal when r 1 or 1 or n other words, there s a perfect negatve or postve correlaton between the two varables under dscusson. If r 0 Regresson lnes are perpendcular to each other. EXERCISE Set A Wrte the correct answers. Each queston carres 1 mark. 1. Bvarate Data are the data collected for (a) Two varables (b) More than two varables (c) Two varables at the same pont of tme (d) Two varables at dfferent ponts of tme.. For a bvarate frequency table havng (p + q) classfcaton the total number of cells s (a) p (b) p + q (c) q (d) pq 3. Some of the cell frequences n a bvarate frequency table may be (a) Negatve (b) Zero (c) a or b (d) Non of these 4. For a p x q bvarate frequency table, the maxmum number of margnal dstrbutons s (a) p (b) p + q (c) 1 (d) 5. For a p x q classfcaton of bvarate data, the maxmum number of condtonal dstrbutons s (a) p (b) p + q (c) pq (d) p or q 6. Correlaton analyss ams at (a) Predctng one varable for a gven value of the other varable (b) Establshng relaton between two varables

43 CORRELATION AND REGRESSION (c) Measurng the extent of relaton between two varables (d) Both (b) and (c). 7. Regresson analyss s concerned wth (a) Establshng a mathematcal relatonshp between two varables (b) Measurng the extent of assocaton between two varables (c) Predctng the value of the dependent varable for a gven value of the ndependent varable (d) Both (a) and (c). 8. What s spurous correlaton? (a) It s a bad relaton between two varables. (b) It s very low correlaton between two varables. (c) It s the correlaton between two varables havng no causal relaton. (d) It s a negatve correlaton. 9. Scatter dagram s consdered for measurng (a) Lnear relatonshp between two varables (b) Curvlnear relatonshp between two varables (c) Nether (a) nor (b) (d) Both (a) and (b). 10. If the plotted ponts n a scatter dagram le from upper left to lower rght, then the correlaton s (a) Postve (b) Zero (c) Negatve (d) None of these. 11. If the plotted ponts n a scatter dagram are evenly dstrbuted, then the correlaton s (a) Zero (b) Negatve (c) Postve (d) (a) or (b). 11. If all the plotted ponts n a scatter dagram le on a sngle lne, then the correlaton s (a) Perfect postve (b) Perfect negatve (c) Both (a) and (b) (d) Ether (a) or (b). 13. The correlaton between shoe-sze and ntellgence s (a) Zero (b) Postve (c) Negatve (d) None of these. 14. The correlaton between the speed of an automoble and the dstance travelled by t after applyng the brakes s (a) Negatve (b) Zero (c) Postve (d) None of these.

44 18.44 STATISTICS 15. Scatter dagram helps us to (a) Fnd the nature correlaton between two varables (b) Compute the extent of correlaton between two varables (c) Obtan the mathematcal relatonshp between two varables (d) Both (a) and (c). 16. Pearson s correlaton coeffcent s used for fndng (a) Correlaton for any type of relaton (b) Correlaton for lnear relaton only (c) Correlaton for curvlnear relaton only (d) Both (b) and (c). 17. Product moment correlaton coeffcent s consdered for (a) Fndng the nature of correlaton (b) Fndng the amount of correlaton (c) Both (a) and (b) (d) Ether (a) and (b). 18. If the value of correlaton coeffcent s postve, then the ponts n a scatter dagram tend to cluster (a) From lower left corner to upper rght corner (b) From lower left corner to lower rght corner (c) From lower rght corner to upper left corner (d) From lower rght corner to upper rght corner. 19. When r 1, all the ponts n a scatter dagram would le (a) On a straght lne drected from lower left to upper rght (b) On a straght lne drected from upper left to lower rght (c) On a straght lne (d) Both (a) and (b). 0. Product moment correlaton coeffcent may be defned as the rato of (a) The product of standard devatons of the two varables to the covarance between them (b) The covarance between the varables to the product of the varances of them (c) The covarance between the varables to the product of ther standard devatons (d) Ether (b) or (c). 1. The covarance between two varables s (a) Strctly postve (b) Strctly negatve (c) Always 0 (d) Ether postve or negatve or zero.. The coeffcent of correlaton between two varables