(Intentional blank page) Please remove this page and make both-sided copy from the next page.

Transcription

1 (Intentonal blank page) Please remove ths page and make both-sded copy from the next page.

2 Statstcal Data Analyss by Excel for Impact Evaluaton (Advanced Course) Text Hstogram, average & standard devaton Dependent t-test (Before-after t-test) Independent t-test (Two-groups t-test) Verson 1. March 03, 013 Ryo SASAKI, Ph.D.

3 Table of Contents About the Author Preparaton of Excel: Installaton of Analyss Tool 4 th sesson: Regresson analyss wth cross-secton data 1 5 th sesson: Regresson analyss wth tme-seres data 14 < Current Postons > <About the Author> Ryo SASAKI, Ph.D. Senor Researcher Adjunct Professor, Graduate School of Socal Desgn Adjunct Professor, Global Collaboraton Center < Academc Background > Ph.D. n Evaluaton, The Evaluaton Center, Western Mchgan Unversty (008) M.P.A n Publc Polcy Analyss, Robert F. Wagner Graduate School of Socal Servce, New York Unversty (1996) B.A. n Law and Poltcs, Department of Law, Sant Paul s (Rkkyo) Unversty (1991) <Contact nformaton> Emal: sasakryo (a) gmal.com Homepage:

4 <Advanced course> 4th sesson: Regresson analyss wth cross-secton secton data How much was the REAL effect of Your nterventon? : Let s estmate the effect by comparng wth your rval groups The government of cty E n Kathmandu ntroduced pay-trash-bag polcy n order to reduce the volume of trash and garbage. (named PaTrash ). Ths polcy requests the collecton trucks to collect only the bags that are pad n advance. Other ctes are watchng how much ths polcy get successful at Cty E. Then, the volume of trash collected are recorded at major ctes n Kathmandu. Can we say the volume of trash has been decreased by ntroducton of ths new polcy? <Necessary condtons> - The data of outcome ndcators (n ths case, Volume of trash ), the date of nterventon ndcators(n ths case, PaTrash ), and other explanatory varables (n ths case, Populaton ) are necessary. - No strong correlaton between explanatory varables s necessary. 1

5 Input of data 1st column.name of town nd column Populaton (,000people) 3 rd column Introducton of PaTrash (1=Yes, 0=No) 4 th column Volume of trash (tones)

6 Operaton of Excel Select Tool > Data Analyss Select Regresson analyss > OK 3

7 Select Y area. You can see $D$1~$D$1 Select X area. In ths case, select B1 ~ D3. You can see $B$1:$C$1 Also, check Label. Excel understand the frst row ndcates the names. 4

8 Obtan the analyss result You can get the followng output (The orgnal sheet locates behnd ths output sheet You should check the four tems. 1 R Square (R ): How much changes the obtaned equaton explans. In ths case, 87.63% s explaned by the obtaned equaton. F:It ndcates the statstcal sgnfcance of the whole equaton. If ts p-value (see Sgnfcance F) <0.05, t s statstcally sgnfcant. In ths case, p-value s It s less than P-value of coeffcents:it ndcates the statstcal sgnfcance of each varable. If p-value <0.05, t s statstcally sgnfcant. In ths case, all two varables, as well as the cut-off, are judged as statstcally sgnfcant. 4 Coeffcents: It ndcates how much each varable can explan the change n the outcome varable (n ths case Volume of trash ). In ths case, volume of trash ncreases by f 1,000 people ncreases. Also the volume of trash decreases by 1, tones by ntroducton of PaTrash. Volume of trash= (populaton) ( PaTrash ) 5

9 Result of Analyss In ths case, t can be concluded that the volume of trash decreased by ntroducton of the pay-trash-bag polcy ( PaTrash ). The volume decreased by ths polcy s estmated as 1,914 tones. Snce the volume of trash n the case that ths polcy be not ntroduced s calculated as 3,433 tones, about 56% of reducton was realzed. ( (7.) (0) = 3,433 ) By the way, the strong correlaton between volume of trash and populaton s observed and t s confrmed statstcally sgnfcant. Thus, t s expected that the ncrease of the volume of trash along wth the ncrease of populaton s expected. (Note) A scatter plots lke the followng page would help readers understandng. It s also recommended to add a regresson lne (that should be drawn by hand on the chart). 6

10 It s estmated that the new polcy ( PaTrash ) made the volume of trash decreased at town E by 1,914 tones. 7

11 Advanced Study Vsual explanaton for understandng the Multple Regresson Ths s the lne chart drawn from the orgnal data. Y Y s explaned by X1. Only X1 (Populaton) s used. X1 Y Y s explaned by X1 & X. X (PaTrash) s added. X1 X X : 0 (No) 1 (Yes) =>The lne drops. If I can use a 3-dmentonal space, I can show ths graph correctly because I can control 3 axes (Y, X1 & X). However, snce a paper s -dmentnal space, I showed t by the above charts. 8

12 [ 1 ] Calculaton laton by Excel Chart Functon Snce the hand calculaton of multple regresson (= a regresson usng more than one explanatory varable) s very dffcult, let s conduct smple regresson (= a regresson usng only one explanatory varable). set. We wll examne the followng hypothess usng the followng data The test score of natonal language (X) wll affect the test score of Englsh (Y). Y = b0 + b1x No. X Y Open you Excel data. Select the XY area. 9

13 Then, clck Chart Wzard. You can see the followng dalogue box. Clck XY Scatter. Then, clck next. 10

14 Then you can see the followng prevew chart. Please clck Next and then Next. Fnally please clck Fnsh. You can obtan the followng scatter plot chart. Please clck one of the dots. If you do t, the color of the dots wll turn from blue to yellow (or, anyway, another color rather than blue). 11

15 Select Chart and then, select Add Trendlne. Select Lnear (Actually, Lnear has already been selected as default). Clck Opton tab. Gve checks to Dsplay equaton on chart and Dsplay R-squared value on chart. 1

16 You get the regresson lne wth ts regresson equaton and R-square (R ). Thus, the regresson equaton obtaned s. Y = X It ndcates 1 pont ncrease n the test score of natonal language wll realze ponts ncrease n the test score of Englsh. 13

17 [ ] Calculaton by Excel s Data Analyss Select Data Analyss. Then, select Regresson. Follow the nstructons as you learned n ths practce. You can get the followng result. You can construct the regresson equaton from the table. Also you can get the calculaton results of R-square, F and ts p-value, coeffcents, and ther t-statstcs and p-values. Thus, the regresson equaton obtaned s. Y = X 14

18 [ 3 ] Hand Calculaton 1. Correlaton Coeffcent 1 Calculaton of correlaton coeffcent between X and Y *X Y = The average of X and Y, respectvely Covarance between X and Y = ( Xno.1 X) ( Yno.1 Y) + ( Xno. X) ( Yno. Y) ( Xno. last X) ( Yno. ( n 1) last Y) Correlato n_ X _ Y ( Cova. rance_ betweenxan dy) = ( Std_ X () Std_ Y) In formal equaton COV( X, Y)= n ~1 ( x x) ( Y ( n 1) Y) r( X, Y)= COV( X, Y) s X s Y Degree of correlaton (assocaton) 0.8 r Strong correlaton (assocaton) between X and Y 0.6 r <0.8 Moderate correlaton (assocaton) between X and Y 0.4 r <0.6 Week correlaton (assocaton) between X and Y r <0.4 Almost no correlaton (assocaton) between X and Y 1 Pearson product-moment correlaton coeffcent 15

19 Practce 1:Calculaton of Correlaton Coeffcent No. X ( X X) Y ( Y Y) ( x x) ( Y Y) Average Average of X of Y ( ) ( Y ) Std of X Std of Y Total= 4 Cov (X,Y)= 5 ( s X ) ( s Y ) Correlaton Coeffcent between X and Y= 6 r( X, Y)= COV( X, Y) s X s Y 16

20 Practce 1:Calculaton of Correlaton Coeffcent (Answers) Practce 1: Calculaton of Correlaton Coeffcent (Answers) No X 1( X X) y ( Y Y) ( x x) ( Y Y) Average of X X Std of X s X Average of Y Y 1.13 合計 = Std of Y Cov(X,Y)= s Y Correlaton Coeffcent between X and Y =

21 . Calculaton of Smple regresson, R, F, and coeffcents Y = b + 0 b1x *The formal expresson s Yˆ b0 + b1x Cut-off Coeffcent estmaton of Y) = (Yˆ s the y-value x-value Calculaton coeffcen t Stdof ( b 1 ) = correlaton_ coeffcent Stdof s The formal expresson s Y b 1 = r sx Y X cut off( b b 0 ) = (averageofy) (averageofx) ( 1 ) 0 = Y X b The formal expresson s 1 b 18

22 Practce :Calculaton of smple regresson (1) Collect the followng nformaton that you have already calculated. (From the prevous practce) correlaton coeffcent r = Average of X = Average Y = Std of X = X s, Std of Y = s. Y Coeffcent b 1 = Cut-off b 0 = = = Regresson equaton Y = () Please wrte the graph (straght lne) n the graph n the prevous page. (3) Please calculate the expected value of 1. 19

23 Practce : : Calculaton of smple regresson (Answers) Practce : Calculaton of smple regresson (Answers) (From the prevous practce) Correlaton coeffcent r = averge of X X = averge of Y Y = 1.13 Std of X = 7.51 Std of Y = s X s Y Coeffcent b = = Cut-off b = = Regresson equaton Y = X () Please wrte the graph (straght lne) n the graph n the prevous (3) Please calculate the expected value of 1 (x=19) *19= 36.6 Agan, thus, the regresson equaton obtaned s. Y = X (Note) The results of hand calculaton s slghtly dfferent from the calculaton results made by Excel s Data analyss functon. However, they would be dentcal (the same) f we use more decmal ponts n our hand calculaton. 0

24 Practce 3: F-statstcs statstcs, R and Adjusted R y= x No. Actual X Actual Y Expected Y Yˆ Expected Y- Average Y Y Y Expected Y - Average Y ˆ ˆ ( Y Y) Actual Y - Expected Y Y ˆ Y Actual Y - Expected Y Y ˆ) ( Y Y Total Total n= p= = = p =FDIST(F,p-1,n-p) = = = = F value= n = 1 n = 1 <Formal formula> ( Yˆ Y) ( Y Yˆ ) df df R E R n ( Yˆ Y) = 1 = n n ( Y Yˆ ) + ( Yˆ = 1 = 1 Y) R /( p 1) F = (1 R )/( n p) 1

25 Practce 3: F-statstcs statstcs, R and Adjusted R (Answers) y= x No. Actual X Actual Y Expected Y Expected Y- Average Y Expected Y - Average Y Actual Y - Expected Y Actual Y - Expected Y Y ˆ) Y 1.13 Total Total n= 15 p= Yˆ ˆ ˆ Y Y ( Y Y) Y ˆ Y ( Y = = p =FDIST(F,p-1,n-p) = = = (69.3%) = (66.9%) <Formal formula> F value= n = 1 n = 1 ( Yˆ Y) ( Y Yˆ ) df df R E R n ( Yˆ Y) = 1 = n n ( Y Yˆ ) + ( Yˆ = 1 = 1 Y) R /( p 1) F = (1 R )/( n p)

26 Advanced study What s the meanng of R and F-value? The followng s the mage of a numerator and a denomnator of R and F. The formula of R s In short, R s the rato of Explaned part and the total part (= Explaned part + Unexplaned part). Roughly sayng, f ths rato s more than 0.60 (60%), we mght be able to say the regresson equaton explans the stuaton well. The formula of F s In short, F-statstcs s the rato of Explaned part and Unexplaned part. (wth some adjustment by n and p). Roughly sayng, f ths rato s more than 4 tmes, the assocated p-value s automatcally less than 0.05 (5%). And then, we conclude the regresson equaton as a whole s statstcally sgnfcant. (Note) The dots of actual Y also exst below the regresson lne n the above mage. For smplfcaton, I draw only dots above the lne. 3

27 Practce 4: : t-value t and p-value of b1 and b0 No. Actual X X Actual X - Actual X - Average X Average X. ( X X) ( X X) Actual Y - Expected Y Y ˆ) ( Y Total Already Calculated n= p= = = p =TDIST(t, n-p, ) = = = p =TDIST(t, n-p, ) = <Formal formula> 4

28 Practce 4: : t-value t and p-value of b1 and b0 (Answers) No. Actual X Actual X - Average X Actual X - Average X. ( X X) ( X X) X Actual Y - Expected Y Y ˆ) Total n= 15 p= ( Y Already Calculated = = p =TDIST(t, n-p, ) = = = p =TDIST(t, n-p, ) = <Formal formula> 5

29 (Reference) Please confrm your calculaton results usng the followng result by Excel Data Analyss tool. SUMMARY OUTPUT Regresson Statstcs Multple R R Square Adjusted R Standard E Observato 15 ANOVA df SS MS F Sgnfcance F Regresson Resdual Total CoeffcentsStandard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0% Intercept X Varable If you apply the actual X value to the obtaned equaton (Y= X), you can get the followng dstrbuton of the expected values. 6

30 [ 3 ] Hand Calculaton of Trash TrashBag Bag case The hand calculaton of ths case s as follows. However, the calculaton s very complcated, ths exercse wll not be conducted n the class. The persons who are nterested n ths calculaton can try t at home. Calculaton of Regresson Equaton (1)Let's calcuate the followngs. The number of X can be more than. No Y X1 X X1² X² X1X X1Y XY Cty A Cty B Cty C Cty D Cty E Cty F Cty G Cty H Cty I Cty J Cty K n ΣY ΣX1 ΣX ΣX1² ΣX² ΣX1X ΣX1Y ΣXY By the way, a matrx X means the followng shape. 1 X11 X1 X= 1 X1 X : : : 1 X1n Xn If you change column and rows, you can get X` X`= X11 X1 X1n X1 X Xn Also, a matrx Y s as follows. Y1 Y= Y : Yn ()Calculate X`X. (Actually, just copy the values from the very frst table.) n ΣX1 ΣX X'X= ΣX1 ΣX1² ΣX1X = ΣX ΣX1X ΣX² (3) Calculate (X'X)ˉ¹. (Actually, just copy the values from the Annex (= the next sheet).) ˉ¹ (X'X)ˉ¹= = (4)Calculate X`Y. (Actually, just copy the values from the very frst table.) ΣY X`Y= ΣX1Y = ΣXY (5) Calculate b=(x`x)ˉ¹ X`Y b 0 (X`X)ˉ¹ X`Y= * = = b b (6) The followng regresson equaton s obtaned. E (Y) = X X X X 7

31 Annex:Calculaton of (X'X)ˉ¹ (X'X)ˉ¹ s the formula to satsfy the followng relatonshp. n ΣX1 ΣX X'X= ΣX1 ΣX1² ΣX1X ΣX ΣX1X ΣX² a b c (X'X)ˉ¹= d e f g h Thus, we can make the followng equatons. an+bσx1+cσx=1 aσx1+bσx1²+cσx1x=0 aσx+bσx1x+cσx²=0 # of equatons s three and # of unknown X s three (a,b,c). thus, we can solve them. dn+eσx1+fσx=0 dσx1+eσx1²+fσx1x=1 dσx+eσx1x+fσx²=0 # of equatons s three and #r of unknown X s three (d,e,f). thus, we can solve them. gn+hσx1+σx=0 gσx1+hσx1²+σx1x=0 gσx+hσx1x+hσx²=1 # of equatons s three and #r of unknown X s three (g,h,). thus, we can solve them ˉ¹ (X'X)ˉ¹= Let's conduct the actual calculaton X'X= a b c (X'X)ˉ¹= d e f g h Smataneous equatons 1 a*11+b*115.9+c*1=1 a*115.9+b* c*7.=0 a*1+b*7.+c*1=0 Smataneous equatons d*11+e*115.9+f*1=0 d*115.9+e* f*7.=1 d*1+e*7.+f*1=0 Smataneous equatons3 g*11+h*115.9+*1=0 g*115.9+h* *7.=0 g*1+h*7.+*1=1 By solvng 1. a= b= c= By solvng. d= e= f= By solvng 3. g= h= = The followng results are obtaned. a b c (X'X)ˉ¹= d e f g h

32 Calculaton of F -value. (1) Calculate Y`Y. () Calculate (1/n) Y`1 1' Y. Y`Y=ΣY², and thus (1/n) Y`1 1' Y = No Y Y² Cty A ( Cty B Y ) = = Cty C n 11 Cty D Cty E Cty F Cty G Cty H Cty I Cty J Cty K n ΣY ΣY² Y`Y= (3) Calculate SST, SSE, SSR. SST =Y`Y- (1/n) Y`1 1' Y = = ( Y ) n SSE =Y`Y-b`X`Y = * = = SSR =b`x`y- (1/n) Y`1 1' Y = = (4) Make an ANOVA table. Source SS df MS F P(F) Regresson p-1= E-05 (FINV(F,df1,df) =FDIST(G111,,1) Error n-p= Total n-1=10 * "p" s counted as "3" (b 0,b 1,b ) Calculaton of R and Adjusted R R² =SSR SSR/SST SST= Adjusted R² R = 1-( -(n-1/n-p)*( )*(1-R²)= )= =1-(10/8)*( )=

33 Calculaton of t-values t of X1, X X t b1 = n = 1 ( Y Yˆ ) n = 1 b 1 ( X X) 1 ( n p) (1)Calcuate the followngs. No Y X1 X (X1-X)² (X-X)² E(Y) (Y-E(Y)) Cty A Cty B Cty C Cty D Cty E Cty F Cty G Cty H Cty I Cty J Cty K n ΣY ΣX1 ΣX Sxx1 Sxx Σ(Y-E(Y)) p Y X1 X ()Calcuate the t-values. Calculaton of tb1 t Calculaton of tb t Σ(Y-E(Y))^/(n-p)= Σ(Y-E(Y))^/(n-p)= (Σ(Y-E(Y))^/(n-p))/Sxx 1 = (Σ(Y-E(Y))^/(n-p))/Sxx = Eb 1 =SQRT((Σ(Y-E(Y))^/(n-p))/Sxx 1)= SEb 1 =SQRT((Σ(Y-E(Y))^/(n-p))/Sxx )= t b 1= b 1 /SE b 1= t b = b /SE b = p=tdist(tb1,n-p,)= p=tdist(tb,n-p,)= The values from Excel's output are The values from Excel's output are tb1=6.317, p= Thus, almost the same.) tb= p= Thus, almost the same.) 30

34 5th sesson: Regresson analyss wth tme-seres data How much was the REAL effect of your nterventon? : Let s calculate the effect from the past trend. The Kathmandu-based retal chan (named Shop Gold ) ntroduced the new sales event (named FrstSaturdy ). It s held n early mornng on frst Saturday of every month. Then, the sales are recorded durng a half year. Can we say the sales has been ncreased by ntroducton of ths new campagn? <Necessary condtons> - The data of outcome ndcators (n ths case sales ), the date of nterventon ndcators(n ths case new sales event ), and other explanatory varables are necessary - No strong correlaton between explanatory varables s necessary. 31

35 [ 1 ] Excel Calculaton Input of data 1st column.year/month/week nd column No. of week (Sequental number of month) 3 rd column Weather(1=Ran 0=Fne/Cloudy) 4 th column FrstSaturday (1=Yes, 0=No) 5 th column Sales volume (US$ thousands) 3

36 Operaton of Excel Select Tool > Data Analyss Select Regresson > OK Can we say new campagn really effectve? 33

37 You can see the followng box. Select Y area. You can see $E$1:$E$7. 34

38 Select X area. In ths case, select B1-D7. You can see $B$1:$D$7 Also, check Label. Excel understand the frst row ndcates the names. 35

39 Obtan the analyss result You can get the followng output (The orgnal sheet locates behnd ths output sheet You should check the four tems. 1 R Square (R ): How much changes the obtaned equaton explans. In ths case, 80.3% s explaned by the obtaned equaton. F:It ndcates the statstcal sgnfcance of the whole equaton. If ts p-value (see Sgnfcance F) <0.05, t s statstcally sgnfcant. In ths case, the p-value s 6.0E-08. It s and t s less than P-value of coeffcents:it ndcates the statstcal sgnfcance of each varable. If p <0.05, t s statstcally sgnfcant. In ths case, all three varables are judged as statstcally sgnfcant. 4 Coeffcents: It ndcates how much each varable can explan the change n the outcome varable (n ths case Sales volume ). Sales volume= (No. of week) (Weather) ( FrstSaturday) 36

40 Results of analyss In ths case, t s judged that the new sales event ( FrstSaturdy ) ncrease the sales volume. Due to conduct FrstSaturday, the sales volume ncrease by US$49.90 thousands. It s also judged that weather affect the sales volume. In addton, the trend of slght ncrease s observed. Even though t s not sure the exact reason, the slght upward trend of the whole natonal economy mght affect here. By the way, the sales volume of Shop Gold can be almost explaned by the followng equaton, and ths can be utlzed for predcton of the future sales volume (e.g., the volume of one year later). Sales volume= (No. of week) (Weather) ( FrstSaturday) (Unt: US$ thousands) 37

41 It s advsable to add the followng graph. (But the ndcaton of FrstSaturday and the one of ran were added by hand). It s estmated that the sale volume was ncreased by the new sales event FrstSaturday. 38

42 Advanced Study Vsual explanaton for understandng the Multple Regresson Ths s the lne chart drawn from the orgnal data. (Sales) (No of Data) Y Y s explaned by X1. Only X1 (No. of week) s used. (Sales) (No of Week) X1 No. of week ndcates the effect of the general trend of economc growth. Y 700 Y s explaned by X1 & X X (Weather) s added. (Sales) (No of Week) X1 X X : 0 (fne/cloudy) 1 (ran) =>The lne drops at several ponts. Y (Sales) (No of Week) X1 X X3 Y s explaned by X1, X X & X3. X3 (FrstSaturday) s added. X3 :0 (FrstSaturday s not) 1 (FrstSaturday s held) => The lne goes up at several ponts. 39

43 ..If I can use a 4-dmentonal space, I can show ths graph correctly because I can control 4 axes (Y, X1, X & X3). However, snce a paper s -dmentnal space, I showed t by these charts. [ ] Addtonal practces The addtonal problems wll be dstrbuted n the tranng sesson. Also you can brng your own data set (your sales volume, etc.) 40