Confidence Intervals for the Squared Multiple Semipartial Correlation Coefficient. James Algina. University of Florida. H. J.

Transcription

1 Eet Size Conidene Inteval 1 Conidene Intevals o the Squaed Multiple Semipatial Coelation Coeiient by James Algina Univesity o Floida H. J. Keselman Univesity o Manitoba all D. Penield Univesity o Miami

2 Eet Size Conidene Inteval Abstat The squaed multiple semipatial oelation oeiient is the inease in the squaed multiple oelation oeiient that ous when two o moe peditos ae added to a multiple egession model. Coveage pobability was investigated o two vaiations o eah o thee methods o setting onidene intevals o the population squaed multiple semipatial oelation oeiient. esults indiated that the poedue that povides oveage pobability in the.95,.975 inteval o a 95% onidene inteval depends pimaily on the numbe o added peditos. Guidelines o seleting a poedue ae pesented.

3 Eet Size Conidene Inteval 3 Coeiient Conidene Intevals o the Squaed Multiple Semipatial Coelation A ommonly used eet size (ES) in multiple egession analysis is the inease in when one independent vaiable X j is added to the model. This ES, whih is alled the squaed semipatial oelation oeiient, is oten symbolized by, measues the stength o elationship between X j the dependent vaiable Y, ontolling o the othe independent vaiables in the model. The oeiient also be used when seveal vaiables ae added to the model. In this ontext, an is alled the squaed multiple semipatial oelation oeiient (Pedhazu, 1997) o the semipatial (Cohen, Cohen, West, Aiken, 003) measues the stength o assoiation between Y the added independent vaiables, ontolling o the othe independent vaiables in the model. Hedges Olkin (1981) pesented methods o alulating the asymptoti sampling ovaiane matix o ommonality omponents (See also Mood, 1969, 1971). These esults an be used to onstut a onidene inteval (CI) o, the population ES estimated by. Olkin Finn (1995) pesented a method equivalent to the Hedges Olkin method, illustated the new method o the ase in whih thee is one independent vaiable in the model in addition to X j (i.e., o the ase o two independent vaiables). Al Ga (1999) simpliied the method showed how to apply it in the geneal ase o p peditos, Ga Al (1999) developed a ompute pogam that omputes the CI. Algina Moulde (001) ound that when the squaed semipatial oelation oeiient is o inteest, eseahes would need

4 Eet Size Conidene Inteval 4 vey lage samples sizes to ahieve adequate oveage pobability o, the population ES. Algina, Keselman, Penield (007) ound that it was possible to obtain muh bette oveage pobability, with smalle sample sizes, i peentile bootstapping methods wee used o setting CIs o the squaed semipatial oelation oeiient, athe than elying on the asymptoti intevals. The pupose o the pesent pape was to investigate whethe asymptoti o peentile bootstap intevals would esult in adequate oveage pobability o when a squaed multiple semipatial oelation oeiient is o inteest. Method Coveage pobability was estimated o the asymptoti two peentile bootstap CIs. Speiially, simulation was used to estimate oveage pobability o ombinations o p, k, n,,, whee n is the sample size, is the population squaed multiple oelation oeiient o a model with p peditos (the ull model), is the population squaed multiple oelation oeiient o a model with k peditos (the edued model) that ae a pope subset o the p peditos. In the onditions we planned to investigate, the numbe o peditos in the ull model anged om p 3 to p 9 in steps o. Ate eviewing the esults we added onditions with p 8 peditos in the ull model. The dieene in the numbe o peditos in the ull edued models i.e., p k anged om to p 1 in steps o 1. The squaed multiple oelation oeiient o the peditos in the edued model anged om.00 to.50 in steps o.10. The squaed multiple oelation oeiient o the ull model anged om to.10 in steps o.01 om.10 to.0 in steps o.05.

5 Eet Size Conidene Inteval 5 Sample size anged om 50 to 00 in steps o 50. The peditos the dependent vaiable wee distibuted as a multivaiate nomal distibution. Eah o the 3,760 ombinations o p, k, n,, was epliated 5000 times in ode to estimate oveage pobability. Fo eah epliation, six 95% CIs wee onstuted: two CIs was onstuted by using two vaiations o (a) the asymptoti method; (b) bootstapping ; () bootstapping, the,, dieene in oeted values o, whee p n p 1, 1, k n k 1, 1. Fo eah CI, the popotion o the epliations that ontained estimated the pobability o oveage. vaiane o Unde multivaiate nomality o the peditos iteion, the asymptoti is 4 (1 ) n (1) (Stuat, Od, & Anold, 1999). The asymptoti vaiane o is obtained by substituting o in the subsipting. Aoding to Al Ga (1999), the asymptoti ovaiane between is [.5( )(1 / ) / ]. () n

6 Eet Size Conidene Inteval 6 The asymptoti vaiane o is, an asymptotially oet 100(1 )% CI o is z /, whee z / is a z itial value (Al & Ga, 1999). In patie, the asymptoti vaiane is estimated by substituting o, espetively. Initial esults indiated that in some onditions in whih, the asymptoti CI esulted in oveage pobabilities above.99. To addess this poblem, the lowe limit o the asymptoti CI was modiied. Speiially, i the lowe limit o the taditional asymptoti CI was less than o equal to zeo, but the F test o H : 0 was 0 signiiant, the lowe limit was set to a small value lage than zeo. In ou simulations the lowe limit was set equal to.001. To apply the taditional peentile bootstap, as desibed in Wilox (003), to the ollowing steps wee ompleted, with the ist two steps ompleted B times. 1. A sample o size n was omly seleted with eplaement om the simulated patiipants.. was alulated o the sample dawn in step1. 3. One the B values o wee obtained, they wee anked om low to high. The lowe limit o the % CI o was detemined by inding the B 1 th estimate in the ank ode, whee B indiates ounding

7 Eet Size Conidene Inteval 7 B to the neaest whole numbe; the uppe limit was detemined by inding the B B th estimate in the ank ode. In all onditions B 1000 ; thus, the bootstap lowe limit was the 6 th value the uppe limit was the 975 th value in the taditional peentile bootstap. A poblem with applying the taditional peentile bootstap to will ou when 0, that is when. Beause will inequently equal, the CI will almost neve ontain 0 the estimated oveage pobability will be nea zeo. To addess this poblem, we modiied the taditional peentile bootstap by inopoating the F test o H : 0 0. When the F test was not signiiant, the lowe limit o the CI was set equal to zeo; othewise, the lowe limit was detemined by using the taditional peentile bootstap. To apply the taditional peentile bootstap to, steps 1 to 3 wee applied with eplaing. Initial esults indiated that in some onditions in whih, this poedue esulted in oveage pobabilities above.99. To addess this poblem, the lowe limit o the bootstap CI was modiied. Speiially, i the lowe limit o the taditional CI was less than o equal to zeo, but the F test o H : 0 was 0 signiiant, the lowe limit was set to a small value lage than zeo. In ou simulations the lowe limit was set equal to.001. A Visual Basi 6.0 pogam that omputes the taditional modiied peentile bootstap CIs o is available at (UL to be added.) The multiple egession model is

8 Eet Size Conidene Inteval 8 Y 1X1 X px p. Thee is no loss in geneality i 0 /o i the vaianes o the dependent vaiable o the independent vaiables ae set equal to 1.0. Aoding to Bowne (1969, 1975), given any set o peditos that has a squaed multiple oelation oeiient o with Y, it is always possible to tansom the peditos so that (a) the independent vaiables ae mutually unoelated (b) the egession oeiients ae equal to any set o values suh that j p 1 j y. Theeoe, in the simulations (a) all vaiables had vaiane equal to one, (b) the independent vaiables wee mutually unoelated, () 1 1 0, k, k k 1 p 1 0, p. Thus, the squaed multiple oelation oeiient was o vaiables X 1 to X k o vaiables X 1 to X p. The data wee simulated by using the ollowing steps. 1. Geneate an n p matix o om vaiables. Eah o the p vaiables was nomally distibuted with mean zeo stad deviation one. All np soes wee geneated to be statistially independent. This matix is X, the matix o soes on the independent vaiables.. Geneate an n 1 veto o nomally distibuted om vaiables with mean zeo stad deviation one. All n soes wee geneated to be statistially independent to be independent o the soes in X. Multiply the geneated veto by 1. The esulting veto is ε, the veto o esiduals.

9 Eet Size Conidene Inteval 9 3. Constut the p 1 veto in whih elements 1 to k 1 ae zeo the next element is, the elements k 1 to p 1 ae zeo, the last element is. 4. Calulate the n 1 soes on the dependent vaiable by using y X ε. esults The taditional bootstap CI using was modiied by setting the lowe limit to zeo i the F test o H was not signiiant; othewise, the lowe limit was : 0 0 detemined by using the taditional peentile bootstap. Although the modiiation was designed to impove peomane when was zeo, the modiiation ould aet oveage pobability o the modiied CI when was zeo o lage. Thus, it was impotant to detemine i oveage pobability o the bootstap CI using was aeted by the modiiation when was not equal to zeo. To do this, we oused on the onditions in whih 0. When 0, o eah ombination o p, p k, n thee ae 7 ombinations o. Fo eah ombination we tabulated the numbe o times that the estimated oveage pobability was in the inteval.95,.975 o the taditional modiied vesion o the bootstap CI using. esults indiated that the modiiation did not edue the numbe o onditions in whih the inteval.95,.975 ontained the estimated oveage pobability. The asymptoti CI taditional bootstap CI using wee modiied by hanging the lowe limit to.001 when the taditional lowe limit was less o equal to zeo the F test o H : 0 0

10 Eet Size Conidene Inteval 10 was signiiant. Fo ou values o, this modiiation ould only aet the peomane o these CIs when was zeo ( in geneal ould only aet the CI i.001. Theeoe, the ollowing esults desibe peomane o the modiied CIs. Fo eah ombination o p, p k, n, we tabulated the numbe o times out o 78 possible ombinations o in whih the estimated oveage pobability was in the inteval.95,.975 o the modiied vesion o the CIs. The esults ae pesented in Table 1. Fo eah ombination, the bold value indiates the method(s) that best ontolled pobability oveage. Ties between methods ae indiated by an undelined bolded esult. Fo p k, thee wee ombinations o p n in whih the bootstap CI using peomed as well o bette than the othe CIs. Howeve, o othe values o p k, eithe the asymptoti o bootstap CI using peomed bette than the bootstap CI using. The elative peomane o the modiied asymptoti bootstap the modiied peentile bootstap using depended on p k. When p k 3, the modiied peentile bootstap tended to wok bette i p 7 ; othewise the two poedues woked about equally well. The modiied peentile bootstap tended to wok bette when p k was between 4 o 5. When p k 6, the two poedues woked about equally well, patiulaly when the sample size was at least 150. Fo p k lage than six, a value that ould only ou in ou design with 8 o 9 peditos in the ull model, the modiied asymptoti bootstap had bette ontol o oveage pobability. Inspetion o the esults suggests that o p k, the elative peomane o the modiied asymptoti CI the modiied bootstap CI using depends on.

11 Eet Size Conidene Inteval 11 The esults also suggest that lage sample sizes ae equied to ahieve ontol o oveage pobability when is small. To illustate these eets, we tabulated (Table ) the numbe o times out o the 18 possible ombinations o o.0 the numbe o times out o the 60 possible ombinations o o.03 (Table 3), that the estimated oveage pobability was in the inteval.95,.975. When.0 p k, neithe the modiied bootstap CI using o the modiied asymptoti CI was onsistently moe eetive than the othe ove all values o p. Tempoaily deining eetive as all estimated oveage pobabilities within the inteval.95,.975, the modiied bootstap CI using was eetive at a smalle size than was the asymptoti CI (see Table 3) o.03 p k. In egad to sample size equied to ahieve good ontol o oveage pobability, the ollowing omments apply to all ombinations o p p k, with the exeption o onditions in whih thee wee nine peditos in the ull model no moe than two in the edued model. When.0, the sample size equied o at least one o the methods to be eetive was 00 in some onditions. When.03, a sample size o 50 o 100 was suiient o at least one o the methods to be eetive. Disussion We investigated oveage pobability o the asymptoti CI two peentile bootstap CIs o in multiple linea egession analyses when peditos iteion wee nomally distibuted desibed the stength o assoiation o

12 Eet Size Conidene Inteval 1 seveal peditos. We also investigated modiied vesions o these CIs. In geneal, the modiied methods woked at least as well as thei unmodiied ountepats. Speiially, esults indiated that the taditional modiied bootstap CI using peomed pooly, exept when p k. Algina, et al. (007) epoted that when desibes the stength o assoiation o one pedito p k 1, using the modiied peentile bootstap with to set a CI o esulted in good oveage pobability in a wide ange o onditions. Thus, the esults, o the ase in whih p k 1, do not genealize to the ases in whih desibes the stength o assoiation o moe than one pedito p k. The taditional modiied asymptoti CI woked well in a vaiety o onditions. These esults ae ontay to esults epoted by Algina Moulde (001) who investigated the ase o p k 1 epoted that the taditional asymptoti CI tended to wok pooly in many onditions with sample sizes o 00 o less. Algina Moulde did not epot esults on the modiied asymptoti CI, but the modiiation used in this pape is designed to impove peomane when is zeo, so that it is unlikely that using the modiiation with p k 1 would oveome the poblems that Algina Moulde epoted. Unotunately, although the modiied asymptoti CI the modiied bootstap CI on woked bette than the ompetitos investigated in this study, neithe o these CIs woked well in all o the onditions we investigated. Patiulaly poblemati was the ondition in whih the numbe o peditos in the ull model was nine the numbe o peditos in the edued model was no lage than two. Deining adequate ontol o

13 Eet Size Conidene Inteval 13 oveage pobability by at least 77 o the 78 ombinations o o eah ombination o p p k, we oe the ollowing eommendations o a CI method sample size in ode to ahieve adequate ontol o oveage pobability,: (a) I p k p 7, the modiied asymptoti CI should be used with a sample size o at least 00. Fo p 8, a sample size o 150 the modiied bootstap CI using should be used; (b) I p k 3 p 7, the modiied bootstap CI using should be used with a sample size o at least 50. I p 8 the modiied asymptoti CI should be used with a sample size o at least 00; () I p k 4 o 5 p 9, the modiied bootstap CI using should be used with a sample size o at least 50; (d) I p k 6 ( theeoe p 7, the modiied bootstap CI using should be used with a sample size o at least 150; (e) I p k 7 ( theeoe p 8, the modiied asymptoti CI should be used the sample size should 00 i p 8 should be lage than 00 i p 9.

14 Eet Size Conidene Inteval 14 eeenes Al, E. F., J., & Ga,. G. (1999). Asymptoti onidene limits o the dieene between two squaed multiple oelations: A simpliied appoah. Psyhologial Methods, 4, Algina, J., Keselman, H. J., & Penield,. D. (007). Conidene intevals o an eet size measue in multiple linea egession. Eduational Psyhologial Measuement. 67, Algina, J., & Moulde, B. C. (001). Sample sizes o onidene intevals on the inease in the squaed multiple oelation oeiient. Eduational Psyhologial Measuement, 61, Bowne, M. W. (1969). Peision o pedition (eseah Bulletin 69-69). Pineton, NJ: Eduational Testing Sevie. Bowne, M. W. (1975). Peditive validity o a linea egession equation. Bitish Jounal o Mathematial & Statistial Psyhology, 8, Cohen, J, Cohen, P, West, S. G., Aiken, L. S. (003). Applied multiple egession oelation o the behavioal sienes (3 d ed.). Mahwah, NJ: Elbaum. Ga,. G., & Al, E. F., J. (1999). Asymptoti onidene limits o patial squaed patial oelations. Applied Psyhologial Measuement, 3, Hedges, L. V., & Olkin, I. (1981). The asymptoti distibution o ommonality omponents. Psyhometika, 46, Mood, A. M. (1969). Mao-analysis o the Ameian eduational system.opeations eseah, 17,

15 Eet Size Conidene Inteval 15 Mood, A. M. (1971). Patitioning vaiane in multiple egession analyses as a tool o developing leaning models. Ameian Eduational eseah Jounal, 8, Pehhazu, E. J. (1997). Multiple egession in behavioal eseah (3 d ed.). Belmont. CA: Wadswoth. Olkin, I., & Finn, J. D. (1995). Coelations edux. Psyhologial Bulletin, 188, Stuat, A., Od, J. K., & Anold, S. (1999). The advaned theoy o statistis (6 th ed., Vol. A). New Yok: Oxod Univesity Pess. Wilox,.. (003). Applying ontempoay statistial tehniques. San Diego: Aademi Pess.

16 Eet Size Conidene Inteval 16 Table 1. Numbe o Coveage Pobability Estimates (out o 78) Inside the.95,.975 Inteval o the Modiied CIs p p k Modiied Modiied Modiied Bootstap on Asymptoti Bootstap on n 50 n 100 n 150 n 00 n 50 n 100 n 150 n 00 n 50 n 100 n 150 n

17 Eet Size Conidene Inteval 17 Table 1. (Con t) Numbe o Coveage Pobability Estimates (out o 78) Inside the.95,.975 Inteval o the Modiied CIs p p k Modiied Modiied Modiied Bootstap on Asymptoti Bootstap on n 50 n 100 n 150 n 00 n 50 n 100 n 150 n 00 n 50 n 100 n 150 n

18 Eet Size Conidene Inteval 18 Table. Numbe o Coveage Pobability Estimates (out o 0) Inside the.95,.975 Inteval o Miied CIs.0 p p k Modiied Modiied Modiied Bootstap on Asymptoti Bootstap on n 50 n 100 n 150 n 00 n 50 n 100 n 150 n 00 n 50 n 100 n 150 n

19 Eet Size Conidene Inteval 19 Table. (Con t) Numbe o Coveage Pobability Estimates (out o 0) Inside the.95,.975 Inteval o Miied CIs.0 p p k Modiied Modiied Modiied Bootstap on Asymptoti Bootstap on n 50 n 100 n 150 n 00 n 50 n 100 n 150 n 00 n 50 n 100 n 150 n

20 Eet Size Conidene Inteval 0 Table 3. Numbe o Coveage Pobability Estimates (out o 60) Inside the.95,.975 Inteval o Modiied CIs.03 p p k Modiied Modiied Modiied Bootstap on Asymptoti Bootstap on n 50 n 100 n 150 n 00 n 50 n 100 n 150 n 00 n 50 n 100 n 150 n

21 Eet Size Conidene Inteval 1 Table 3. (on t) Numbe o Coveage Pobability Estimates (out o 60) Inside the.95,.975 Inteval o Modiied CIs.03 p p k Modiied Modiied Modiied Bootstap on Asymptoti Bootstap on n 50 n 100 n 150 n 00 n 50 n 100 n 150 n 00 n 50 n 100 n 150 n