Usng Multvarate Rank Sum Tests to Evaluate Effectveness of Computer Applcatons n Teachng Busness Statstcs by Yeong-Tzay Su, Professor Department of Mathematcs Kaohsung Normal Unversty Kaohsung, TAIWAN 80264 Chy-Ly (Kathleen) Lang, Assstant Professor Department of Communty Development and Appled Economcs The Unversty of Vermont Burlngton, Vermont Selected Paper presented n the Amercan Agrcultural Economcs Assocaton Annual Meetng, Nashvlle, Tennessee, August 8-11, 1999. Copyrght 1999 by Yeong-Tzay Su and Chy-ly (Kathleen) Lang. All rghts reserved. Readers may make verbatm copes of ths document for non-commercal purposes by any means, provded that ths copyrght notce appears on all such copes.
ABSTRACT Arguments about usng computer facltes to classroom teachng have ganed a lot of attenton over tme. Usng the computer facltes wll be helpful to demonstrate real world applcatons, whle poor data or napproprate case studes mght reduce the confdence n applyng computer programs n classroom teachng. In ths artcle we examne the results of usng computer facltes to teach Busness statstcs to a group of Management students sampled from Krannert School of Management, Purdue Unversty. Ths study shows that students are attracted to the nteractve computer programs desgned for the Busness Statstcs course, and students are more motvated to attend classes when usng computer facltes are appled n teachng. Furthermore, computer programs help students to understand confusng topcs (such as the Central Lmt Theorem), and students feel that teachng them to use computer facltes really mproves students ablty to apply smlar programs n analyzng real world problems.
INTRODUCTION Snce 1980 computer facltes have been popularly appled to classroom teachng n order to mprove the qualty of teachng and learnng, especally for those classes nvolvng statstcal concepts. At the very begnnng of the computer era, many people had doubtes n the effectveness of applyng computer facltes n classroom teachng on campus. Prevous study showed that more and more computer facltes and programs had been adapted n classroom teachng, especally for teachng statstcs (Evans, 1973). A prevous survey concluded that more than 86% of the schools have used computer facltes n teachng statstcs n the M.B.A. programs (Rose, Machak and Spvey, 1998). Recently the Department of Educaton n Tawan set a sde specal fundng to support the unverstes to develop programs for computer applcatons n teachng every course ncludng statstcs. Even though computer applcatons become more and more popular n dfferent schools n dfferent countres, the effectveness of the computer applcatons stll puzzle most of the teachers. There s lmted nformaton on the evaluaton of the effectveness of the computer programs appled n classroom teachng. Teachers usually ask: How well do the students learn from applyng computer programs n classroom teachng? Do computer programs really mprove the qualty of learnng and teachng? How sgnfcant the mpacts are on students who attend the courses? Ths artcle presented the results of the case study whch examned the effectveness of applyng computer facltes to teach Busness Statstcs to several groups of undergraduate Management students (ncludng Junors and Senors) sampled from Krannert School of Management at Purdue Unversty. The followng sectons wll descrbe the data of the case study, methodology appled to ths study to analyze the data, results from the study, and fnally the mplcatons and concludng remarks. Page -1-
DISCUSSION OF THE DATA The data of ths study was collected from the teachng evaluaton of the same nstructor who taught Busness Statstcs every semester between 1994 and 1996. Ths nstructor exposed three groups of the students to dfferent frequences of the computer applcatons: never use computer programs, moderately appled computer programs, or frequently appled computer programs. These three groups of the students were randomly assgned to each secton taught by the same nstructor when they regstered for the Busness Statstcs course n the begnnng of each semester. Then the nstructor randomly chose sectons and decded the frequency to apply computer programs n teachng each secton. The nstructor kept the same teachng style, the same homework assgnments, the same nstructon procedures, and the same examples demonstrated n the class. The only varable n teachng each secton was the frequency of the computer applcatons n classroom. By the end of each semester, students were asked to evaluate the nstructor as well as the course. The complete teachng evaluaton contaned twenty questons, each queston had fve possble answers: strongly agree, agree, uncertan, dsagree, and strongly dsagree. Each answer had fve dfferent grades: strongly agree (SA 5), agree (AG 4), uncertan (UC 3), dsagree (DA 2), and strongly dsagree (SD 1). We can rank the students evaluaton towards the nstructor or the course by students preference, namely SA > AG > UC > DA > SD. There were seven questons n the evaluaton form whch were drectly or ndrectly related to the usage of the computer facltes, and they were: 1. Ths nstructor stmulated nterest n the course, 2. Explanaton of the materal was clear and to the pont, Page -2-
3. Ths nstructor used meanngful examples and applcatons, 4. Overall, ths course was very useful to me and my career, 5. My nstructor motvated me to do my best work, 6. My nstructor explaned dffcult materal clearly, 7. Overall, ths course s among the best I have ever taken. Totally 202 students took the evaluaton. Among these 202 students, 36 students were from the class n whch computer programs were never appled (n 36, group 1), 60 students 1 were from the classes n whch computer programs were moderately appled (n 60, group 2), 2 and 106 students were from the classes n whch computer programs were frequently appled (n 3 106, group 3). Table 1 to Table 3 summarzed the number of the students who chose to answer strongly agree, agree, uncertan, dsagree, or strongly dsagree for each queston. For example, 36 students were n group 1 (computer ads were never used), and 5 students answered strongly agree, 14 students answered agree, 10 students answered uncertan, 5 students answered dsagree, and 2 students answered strongly dsagree for queston 1. Table 1. Computer ads were never used. SA AG UC DA SD 1 5 14 10 5 2 2 8 16 5 7 0 3 10 15 8 2 1 4 3 12 13 5 3 5 9 11 8 6 2 6 6 17 8 3 2 7 2 9 14 8 3 n 36 1 Page -3-
Table 2. Computer ads were moderately used. SA AG UC DA SD 1 16 33 9 2 0 2 19 31 7 3 0 3 23 29 8 0 0 4 14 20 19 6 0 n 2 60 5 17 25 11 6 0 6 16 37 6 1 0 7 5 19 29 7 0 Table 3. Computer ads were frequently used. SA AG UC DA SD 1 48 50 7 1 0 2 52 45 7 2 0 3 56 45 3 2 0 4 24 42 31 6 3 n 106 3 5 46 41 18 1 0 6 54 41 10 1 0 7 29 36 29 8 3 METHODOLOGY - THE MULTIVARIATE RANK SUM TEST Classc Ch-Square test would not be approprate for testng the varatons of the students answers n ths study, due to dfferent number of observatons n each groups. A Multvarate Rank Sum Test had been developed to test the varablty of the students answers between three groups. Assume for each 1,2,3,{Y j, j 1,..., n } are dentcally and ndependently dstrbuted (..d) random varables wth P{Y j k} p k > 0, where k 1,, 5 k 1 5 p k 1 123,, Page -4-
Consder the testng hypothess wth at least one nequalty strct: H o : ~ p ~ p ~ p H : ~ p ~ p ~ 1 p ~ pl ( ~ p, ~ 2 3 versus p,, ~ 1 1 2 3 1 2 p5) ~ p ~ p p p k 12,, 345,, l l' k ' k k ' k ' k ' k ' Set N n + n + n and replace each Y by r, ts rank n the overall sample, use average ranks for 1 2 3 j j tes. Let M k # { j: Y j k}, and m m R k n j 1 r k j m m k k Then R R n rj m k ' < k k ' + ( 1+ m k ) / 2 f Yj k and R m + m ( 1+ m ) / 2 5 k 1 l< k l k k Under the null hypothess the Y s are..d. random varables. j On the other hand, f the alternatve hypothess s true, than R tends to have a larger value 3 and R tends to have a small value. An easy decson rule for testng hypothess s set by T R + 1 2 R and reject H f and only f T s large. Snce the sample szes n, n, and n are all large, a large 3 o 1 2 3 sample approxmaton s approprate. We wll splt the alternatve hypothess nto three parts - ~ p < ~ p ~ p < ~ p ~ p < ~ p 1 2 2 3 1 3 and compare the correspondng par of rank averages. Page -5-
If H s true, then t s easy to see [cf. Kruskal 1952] that ER n ( N + 1) / 2 and o ER ( N + 1) / 2, where VarR and Cov( R, R ) wth 5 n N n N n N n ( )( + 1) ( ) γ 12 12 N ( N 1) ' nn ' ( N + 1) nn ' + γ 12 12 N ( N 1) γ m ( m 1)( m + 1) k 1 Moreover, the random varables k k k R n ( N + 1) / 2 2 ( N N γ ) / 12 are approxmately multnormal wth zero mean and covarance matrx whose.j term s n δ j j 2 N nn N Random varables X and Y are multnormal wth zero mean, unt varances, and correlaton coeffcent ' f and only f they have the jont densty 1 2πσ 1 ρ 2 2 e 2 2 2 2 ( x 2ρxy+ y )/ 2σ ( 1 ρ ) Let Z X-Y then the jont densty of Z and Y s gven by Page -6-
The p.d.f. of Z s 1 g ZY ( z, y) 2 2 2πσ 1 ρ e 1 2 2 2πσ 1 ρ e f ( z) g zy ( z, y) dy 1 e 2πσ 1 ρ 1 e 2πσ 1 ρ 2 2 2 2 (( z+ y ) 2ρ( z+ y ) y+ y )/ 2σ ( 1 ρ ) 2 2 2 2 2 2 ( y z/ 2) / σ ( 1+ ρ ) + ( 1 ( 1 ρ)/ 2) z / 2σ ( 1 ρ ) ( 2 ( 1 ρ)) z / 4σ ( 1 ρ ) 2 2 z / 4σ ( 1 ρ) 2 2 2 2 Therefore Z s normally dstrbuted wth zero mean and varance 2 ) (1 - '). For 1 j 3, set Z and Z ' j j R ( N + 1) / 2 R ( N + 1) / 2 n ( N n ) / N n ( N n ) / N j Z ' j 2 2( 1+ nn / N ) j j Then f H o s true, by the above dscusson each Z j s approxmately normally dstrbuted wth zero mean and unt varance. So we reject H f Z > z for the testng hypothess (1) wth level. o 13 However, nstead of testng the hypothess (1), we test the alternatve hypothess ~ p < ~ p ~ p < ~ p ~ p < ~ p 1 2 2 3 1 3 separately. Alternatvely we compare the p-value ~ p < ~ p p P {Z > Z } wth and conclude f P <. j o j j j THE TESTING RESULTS AND CONCLUSIONS Wth the sgnfcance level 0.01, the testng results, p-values, and the decsons are Page -7-
summarzed n Table 4 to Table 10. Table 4. Instructor stmulates nterest n the course. _ Computer use SD DA UC AG SA Never 2 5 10 14 5 Moderate 0 2 9 33 16 Frequent 0 1 7 50 48 Testng results and conclusons. Testng statstcs Value p-value Reject Ho Z12 2.210267.0136 No Z23 3.011938.00115 Yes Z13 5.311483.00000 Yes Table 5. Explanaton of the materal was clear and to the pont Computer use SD DA UC AG SA Never 0 7 5 16 8 Moderate 0 3 7 31 19 Frequent 0 2 7 45 52 Testng results and conclusons. Testng statstcs Value p-value Reject Ho Z 1.286386.09965 No 12 Z 2.618263.00442 Yes 23 Z 3.986411.00005 Yes 13 Table 6. Ths nstructor used meanngful examples and applcatons. Computer use SD DA UC AG SA Never 1 2 8 15 10 Moderate 0 0 8 29 23 Frequent 0 2 3 45 56 Testng results and conclusons. Testng statstcs Value p-value Reject Ho Z 1.353227 0.08743 No 12 Z 2.333013 0.0098 Yes 23 Z 3.745691 0.0001 Yes 13 Page -8-
Table 7. Overall, ths course was very useful to me and my career. Computer use SD DA UC AG SA Never 3 5 13 12 3 Moderate 0 6 20 20 14 Frequent 3 6 31 42 24 Testng results and conclusons. Testng statstcs Value p-value Reject Ho Z 1.906184.0283 No 12 Z 0.686453.2473 No 23 Z 2.512481.0059 Yes 13 Table 8. My nstructor motvates me to do my best work. Computer use SD DA UC AG SA Never 2 6 8 11 9 Moderate 0 6 12 25 17 Frequent 0 1 18 41 46 Testng results and conclusons. Testng statstcs Value p-value Reject Ho Z 0.824744.2047 No 12 Z 2.563889.00517 Yes 23 Z 3.496861.0002 Yes 13 Table 9. My nstructor explans dffcult materal clearly. Computer use SD DA UC AG SA Never 2 3 8 17 6 Moderate 0 1 6 37 16 Frequent 0 1 10 41 54 Testng results and conclusons. Testng statstcs Value p-value Reject Ho Z 1.712704.0434 No 12 Z 2.892640.0019 Yes 23 Z 4.676796.0000 Yes 13 Page -9-
Table 10. Overall, ths course s among the best I have ever taken. Computer use SD DA UC AG SA Never 3 8 14 9 2 Moderate 0 7 29 19 5 Frequent 3 8 30 36 29 Testng results and conclusons. Testng statstcs Value p-value Reject Ho Z 1.056293.1457 No 12 Z 3.021289.0013 Yes 23 Z 4.199651.0000 Yes 13 After comparng the answers for all seven questons, there was no sgnfcant varablty between the students n group 1 and students n group 2. However students revealed sgnfcant varablty between group 2 and group 3, as well as between group 1 and group 3. From the above tables, all p-values of Z are greater than 0.01. Ths means that there s no sgnfcant 12 mprovement n learnng or responses for students exposed to moderate usage of the computer programs, comparng to the students who had never been exposed to computer programs. The results were qute dfferent when comparng students n group 2 (moderately use computer programs ) to students n group 3 (frequently use computer programs), and also when comparng students n group1 (never use computer programs) to students n group 3. To compare the varablty n responses between group 2 and group 3, all p-values expect the one for the 4 th queston (overall ths course was very helpful to me and my career) s smaller than 0.01. Ths means that students who have been frequently exposed to computer programs tend to be more nterested n the course contents comparng to students who have been exposed to moderate usage of the computer programs. Students exposed to frequent computer usage also fell that they understand the materals better, and they agree that they have been motvated to do ther best Page -10-
work. Generally speakng students exposed to frequent computer usage perform better n the course, comparng to students exposed to moderate computer usage. Comparng students n the class nvolvng frequent computer usage to students n the class never ntroducng computer programs, all p-values for Z are small than 0.01. Ths provdes a 13 very strong support to the concluson: frequent usage of computer programs really mproves the teachng and learnng qualty n Busness Statstcs courses. SUMMARY AND IMPLICATIONS Over years teachers have been debatng about the effectveness of the computer applcatons n classroom teachng. For some courses as Statstcs, computer programs could be very helpful to explan dffcult contents such as regresson analyss, Central Lmt Theorem, and probablty dstrbutons. Students wll be beneft from the demonstraton of the computer programs n classroom, and they wll be more attracted and more nterested n learnng Statstcs. They wll also be used to the applcatons of the computer programs, so that they wll be able to apply smlar computer programs n analyzng real world problems. However to apply computer programs frequently n the classroom teachng may or may not help students get good grades. Whether students really learn more or perform better through out the semester, ths can not be answered easly from ths study. Further research need to be focused on evaluatng the relatonshp between students overall performance and the frequency of the computer applcatons n classroom teachng. Page -11-
REFERENCES [1] Evans, D.A. (1973). The nfluence of computers on the teachng of statstcs. J.R. Statstcs Soc. A 136(2), 153-190. [2] Hollander, M. and Wolfe, D.A. (1973). Nonparametrc statstcal methods. John Wly & Sons, New York. [3} Kruskal, W.H. (1952). A nonparametrc test for several sample problems. Ann. Math. Statstcs 23, 525-540. [4] Mead, R. and Stern, R.D. (1973). The use of a computer n the teachng of statstcs. J.R. Statstcs Soc. A. 136(2), 153-190. [5] Rose, E.L., Machak, J.A. and Spvey, W.D. (1988). A survey of the teachng of statstcs n M.B.A. programs. Journal of Busness and Economcs Statstcs 6(2), 273-282. Page -12-