Reseach Design - - Topic 7 Multiple Regession & Multiple Coelation: Two Pedictos 009 R.C. Gadne, Ph.D. Geneal Rationale and Basic Aithmetic fo two pedictos Patial and semipatial coelation Regession coefficients Relation of Multiple coelation to elations among pedictos Running SPSS Regession - - Linea Dimensionality of multiple egession
Geneal Rationale and Basic Aithmetic The ationale undelying multiple egession is that one can compute a weighted aggegate of a seies of vaiables that gives maximum pediction of a citeion. The esult is a egession equation like the following. + C + b b Raw scoe fom: Standad scoe fom: Z β Z + β Z The weights b and b (o β, β ) ae detemined such that the coelation between and (o the equivalent Z, and Z ) is as lage as possible. This will be achieved when: ( )² a minimum ( pinciple of least squaes) With algeba, it can be shown that R. ( NS )( S ) β + β
An illustation of the Pediction Model Slope of on Slope of on b b Intecept c 0 0 In the diagam, and ae shown to be othogonal (i.e., independent of each othe), but geneally the pedictos ae coelated. Thus, to ensue independence, we calculate the egession coefficients on esidualized vaiables. This involves the constucts of patial and semipatial coelation.
. Patial Coelation Plots in Standad Scoe Fom. ( )( Z Z Z Z ) ns S Z Z Z Z Z Z ( ) Z Z Z Z Z ( ) Z Z Z Z Z. Semipatial (pat) Coelation (.) Z ns ( Z Z ) Z Z 4
Standad Scoe fom: Whee: β β Regession Equations Z (.) (.) β Z + β Z Thus: Beta coefficients can be shown to equal the semipatial coelation of the citeion with a pedicto divided by the standad eo of estimate of that pedicto as pedicted by the othe pedicto(s) in standad scoe fom. 5
Raw Scoe fom: C + b + b whee b S S β and b S S β and C bs b S The Multiple Coelation Coefficient with two pedictos is: R. β + β R. + 6
Relation of Multiple Coelation to Relations Among Pedictos Othe things being equal, it can be shown that the the multiple coelation inceases as the coelation between pedictos deceases. Conside the case whee It can be shown that:.6. 5 + ± Thus:.0. 48. 9. 99 ± Thus, we can conside the values of β, β, and R. when vaies fom -.0 to.90. Applying the fomulae would poduce the following answes β -.0 -.0.00.0.40.60.80.90.84.79.600.5.476.469.556.789 β R. Z ( Z + Z ).747.646.500.96.0.9.056 -..9.87.78.75.664.65.60.607.90.870.78.70.657.65.580.564 Coelation of Z with Z + Z 7
MR Gaph of the Multiple Coelation against the coelation between the two pedictos.6,.5 -.0 > <.90 A Dot/Lines show Means 0.900 A 0.800 A 0.700 A A 0.600 A A A -0.5 0.00 0.5 0.50 0.75 As the coelation between the pedictos ( ) inceases, the multiple coelation deceases until a vey high coelation between the pedictos (but note change at the end). 8
Running SPSS Regession- - Linea Regession can be un with pedictos enteed diectly as a set, o hieachically, o indiectly with the ode detemined by a compute algoithm (i.e., fowad, backwad, o stepwise inclusion). The following example focuses on hieachical and set enty and Shows the: Data Edito, Syntax file fo the fist un, Desciptive statistics and coelation matix, Output fo thee uns:. Hieachically followed by (slide ).. Hieachically followed by (slide ).. As a set, and togethe (slide 4). 9
0
REGRESSION /DESCRIPTIVES MEAN STDDEV CORR SIG N /MISSING LISTWISE /STATISTICS COEFF OUTS R ANOVA CHANGE /CRITERIAPIN(.05) POUT(.0) /NOORIGIN /DEPENDENT /METHODENTER /METHODENTER. Coelations Desciptive Statistics Mean Std. Deviation N 8.00 7.0978 0 0.600.6 0 5.650.60 0 Peason Coelation Sig. (-tailed) N.000.75.506.75.000.75.506.75.000..000.0.000..000.0.000. 0 0 0 0 0 0 0 0 0 Note. Meng et al test indicates that.75 does not diffe significantly fom.506 (Z.7).
The following tables summaize the Change Statistics and the Regession Coefficients fo the two models. Note, in this example was enteed on the fist step and on the second step. Model Summay Model Change Statistics Adjusted Std. Eo of R Squae R R Squae R Squae the Estimate Change F Change df df Sig. F Change.75 a.55.499 5.050.55 9.908 8.000.77 b.59.474 5.50.004.5 7.77 a. Pedictos: (Constant), b. Pedictos: (Constant),, Coefficients a Model (Constant) (Constant) a. Dependent Vaiable: Unstandadized Coefficients Standadized Coefficients B Std. Eo Beta t Sig..984.804.50.006.50.4.75 4.46.000.850.96.06.008.678.54.795.40.006 -.54.69 -.09 -.68.77
These two tables show the Change Statistics and the Regession Coefficients when is enteed fist and is enteed second. Model Summay Model Change Statistics Adjusted Std. Eo of R Squae R R Squae R Squae the Estimate Change F Change df df Sig. F Change.506 a.56.4 6.90.56 6.86 8.0.77 b.59.474 5.50.7 9.858 7.006 a. Pedictos: (Constant), b. Pedictos: (Constant),, Coefficients a Model (Constant) (Constant) a. Dependent Vaiable: Unstandadized Coefficients Standadized Coefficients B Std. Eo Beta t Sig. 0.40.46 5.98.000.80.555.506.487.0.850.96.06.008 -.54.69 -.09 -.68.77.678.54.795.40.006
These two tables show the Model Summay and the Regession Coefficients when and ae enteed as a set. Model Model Summay Adjusted Std. Eo of R R Squae R Squae the Estimate.77 a.59.474 5.50 a. Pedictos: (Constant),, Model (Constant) a. Dependent Vaiable: Unstandadized Coefficients Coefficients a Standadized Coefficients B Std. Eo Beta t Sig..850.96.06.008.678.54.795.40.006 -.54.69 -.09 -.68.77 4
Impotant points to note fom the thee examples. When only one vaiable is enteed (Model in slides and ), Beta the multiple coelation (i.e., a bivaiate coelation at this stage).. Beta the unstandadized egession coefficient multiplied by the standad deviation of the pedicto divided by the standad deviation of the citeion: β bs (.678)(.6) 7.0978 S.795. In the final equation, the ode of enty doesn t make a diffeence. The multiple coelation and the egession coefficients ae identical when all pedictos ae enteed. 4. The ode of enty tells diffeent tales. In slide, entes significantly at Model, does not add significantly at Model. In slide, entes significantly at Model, and adds significantly at Model. 5
On the Dimensionality of Multiple Regession It is common pactice with multiple coelation to conclude (incoectly) that those vaiables with significant egession coefficients ae good pedictos. Thus, in ou example fo the full model, some would conclude (incoectly): is a good pedicto because β.795, t.4, p<.006 is a bad pedicto because β -.09, t -.68, ns. Close examination of the esults, howeve, yields a moe compehensive intepetation. Thus, the β s epesent the unique contibutions of each pedicto. These unique contibutions can also be assessed in tems of the impovement in pediction when a new pedicto is added. That is, the unique contibutions of and ae also defined as follows: R R R, R.59.506².796 R, R.59.75².0075 F-atios of these squaed multiple semipatial coelations 9.85 and.5 espectively, which ae the squaes of the t-tests fo the coesponding β coefficients (.4 and -.09, espectively). 6
Summaizing this on a Venn Diagam Note the segments 4,5, and 6 epesent unique contibutions of,, and x. Unique contibution of to (.796) Unique contibution of to (.0075) Unique contibution of vaiance common to and to (.59) The unique contibution the vaiance common to and is computed as: R x R R R.59.796.0075,.59 7
Undestanding Multiple Coelation Note that the sum of segments 4,5, and 6 equal the squaed multiple coelation. 7964 +.0075 +.59.59 The sum of segments 4 and 5 equal the squaed coelation of with.. 7964 +.59.5554.75² The sum of segments 5 and 6 equal the squaed coelation of with.. 0075 +.59.5604.506² 8
Undestanding the Contibutions Popotion of vaiance accounted fo uniquely by.7964.59.56 Popotion of vaiance accounted fo uniquely by.0075.59.007 Popotion of vaiance accounted fo uniquely by the vaiance common to and (i.e., x).59.59.477 Note that only the unique contibutions of and can be tested fo significance, but clealy, both and contibute to the pediction of. Focusing only on the unique contibutions gives a distoted pictue. Recall that the coelations of the pedictos with the citeion did not diffe significantly, so that thee was no evidence that one was a bette pedicto than the othe. 9