RESEARCH ESSAY COSISE PARIAL LEAS SQUARES PAH MODELIG heo K. Djksta Faculty of Economcs and Busness, Unvesty of Gonngen, ettelbosje, 9747 AE Gonngen HE EHERLADS {t.k.djksta@ug.nl} Jög Hensele Faculty of Engneeng echnology, Unvesty of wente, Denelolaan 5, 75 B Enschede HE EHERLADS {j.hensele@utwente.nl} OVA IMS, Unvesdade ova de Lsboa, 070-3 Lsbon PORUGAL {jhensele@seg.unl.pt} Appendx A In vaance-based SEM, the path coeffcents can be detemned based on the constuct scoe coelaton matx R, whch fo the model depcted n Fgue s of the followng fom ( co coelaton; el elablty: R 3 co co 3 3 3 co( ξ ( ( ( ( (, ξ el ξ el ξ co ξ, η el ξ el η ( ξ ( ( ( ( (, ξ el ξ el ξ co ξ, η el ξ el η ( ξ, η el( ξ el( η co( ξ, η el( ξ el( η Fom R, we can extact the submatx R X and the subvecto R Xy. 3 3 Unde the assumpton that the constuct scoes ae standadzed, the egesson equaton equals β β RX Xy β 3 3 det( R X 3 3 3 3 3 3 Snce the tue β equals zeo, we can explot that co(ξ, η co(ξ, ξ @ co(ξ, η and thus fnd the followng esult fo the path βˆ coeffcent as estmated by vaance-based SEM: MIS Quately Vol. 39 o. Appendces/June 05 A
Djksta & Hensele/Consstent Patal Least Squaes Path Modelng ˆ β 3 3 co co co co ( ( ξ, η el( ξ el( η co( ξ, ξ el( ξ el( ξ co( ξ, η el( ξ el( η co ( ξ, ξ ( ξ, ξ co( ξ, η el( ξ el( η co( ξ, ξ el( ξ el( ξ co( ξ, η el( ξ el( η co ( ξ, ξ ( ξ, ξ co( ξ, η el( ξ el( η ( el( ξ co ( ξ, ξ ( ξ, ξ el ( ( ( ξ co ξ, η co ξ, ξ el( ξ Appendx B A statng pont fo PLS-analyses s the so-called basc desgn, n essence a facto model. It s assumed that we have..d. column vectos of obseved scoes y, y, y 3,, y (.e., we have obsevatons. We assume the usual standadzaton n PLS of each ndcato, whch means that each vecto y n fo n,, 3,..., has zeo mean and ts elements have unt vaance. he vectos can be pattoned nto M subvectos, M $, each wth at least two components. Fo the th subvecto y n of y n, we have y n η n + ε n ( whee the loadng vecto and the vecto of measuement eos ε n have the same dmensons as y n, and the unobsevable latent vaable η n s eal-valued. 3 Fo convenence, we make the suffcent but by no means necessay assumpton that all components of all eo vectos ae mutually ndependent, and ndependent of all latent vaables. he latent vaables have also zeo mean and unt vaance. he coelaton between η n and η jn wll be denoted by ρ j. A patcula set of easy mplcatons s that the covaance matx Σ of y n can be wtten as : Ey y ρ j j jn j j ( whee the covaance matx of the measuement eos Θ s dagonal wth non-negatve dagonal elements, and we have fo the covaance between y n and y jn o keep the pape self-contaned, we collect hee the man algebac esults undelyng PLSc (fo a moe elaboate explcaton, see Djksta 00. ote that thee ae many ways to coect fo nconsstency and the one we use hee s qute possbly the smplest one. See Djksta (03 fo altenatves and extensons. Any combnaton of PLS wth one of the altenatves could legtmately be called PLSc. he use of..d. (ndependent and dentcally dstbuted sgnfes that the data ae seen as a andom sample fom a populaton. hs s fo conceptual convenence only. ogethe wth the classcal laws of lage numbes and the cental lmt theoem (Camé 946, ths entals the consstency and asymptotc nomalty popetes we need. We can make much moe geneal assumptons, but the statststcal subtletes would be a dstacton and would not lead to dffeent esults (CA-estmatos. 3 We follow exactly the specfcaton of Joeskög (969. Please note that vaables ae aanged pe ows, and obsevatons pe columns. Moeove, note that, as opposed to ou custom n the man body of the pape, t pays hee to use a subscpt fo the blocks. A MIS Quately Vol. 39 o. Appendces/June 05
Djksta & Hensele/Consstent Patal Least Squaes Path Modelng j : Ey y ρ n jn j j (3 he covaance matx of each y n wll be denoted by Σ, and the sample covaance matx by S. he sample countepats of Σ and Σ j ae denoted by S and S j, espectvely. Snce we assume that the sample data ae standadzed befoe beng analyzed, we have Sj yn yjn. n ote that the assumptons made so fa ental that the sample countepats ae consstent and jontly asymptotcally nomal estmatos of the theoetcal vaance and covaance matces (Camé 946, Chapte 8. PLS featues a numbe of teatve fxed-pont algothms, of whch we select one, the so-called Mode A algothm. hs s, n geneal, numecally the most stable algothm. 4 As a ule, t conveges fom abtay statng vectos, and t s usually vey fast. he outcome s an estmated weght vecto ŵ, wth typcal subvecto ŵ of the same dmensons as y n. Wth these weghts, sample poxes ae defned fo the latent : w y n ( w yn w Sw η n n vaables: fo η n, wth the customay nomalzaton of a unt samplng vaance, so. In Wold s PLS appoach the ηˆn s eplace the unobseved latent vaables, and loadngs and stuctual paametes ae estmated usng odnay least squaes. 5 Fo Mode A, we have fo each (% stands fo popotonal to w sgn S w j j j j C ( Hee, sgn j s the sgn of the sample coelaton between ηˆ and ηˆj, and C ( s a set of ndces of latent vaables. adtonally, C ( contans the ndces of latent vaables adjacent to ηˆ (.e., the ndces of latent vaables that appea on the othe sde of the stuctual o path equatons n whch ηˆ appeas. Clealy, ŵ s obtaned by a egesson of the ndcatos y n on the sgn-weghted sum:. hee ae ( sgn y C j η jn othe vesons (wth coelaton weghts, fo example; ths s one of the vey smplest, and t s the ognal one (see Wold, 98. hee s lttle motvaton n the PLS lteatue fo the coeffcents of S j ŵ j, 6 but the patcula choce can be shown to be elevant fo the pobablty lmts of the estmatos. he algothm takes an abtay statng vecto, and then bascally follows the sequence of egessons fo each, each tme nsetng updates when avalable (o afte each full ound; the pecse mplementaton s not mpotant. Djksta (98, 00 has shown that the PLS modes convege wth a pobablty tendng to one when the sample sze tends to nfnty, fo essentally abtay statng vectos. Moeove, the weght vectos that satsfy the fxed-pont equatons, ae locally contnuously dffeentable functons of the sample covaance matx of y.hey, as well as othe estmatos that depend smoothly on the weght vectos and S, ae theefoe jontly asymptotcally nomal. (4 Let us denote the pobablty lmt of ŵ, plm ŵ, by w. We can get t fom the equaton fo ŵ by substtuton of Σ fo S. w sgn w sgn w sgn ρ ρ w j j j j j j j j j j C ( j C( j C( j (5 Snce wj s a scala, and all tems n the sum have the vecto n common, t s clea that w. We must have w w, so w (6 4 hs s because Mode A gnoes collneaty between the obseved vaable pedctos of the poxy, whle Mode B takes account of that colllneaty and thus must fnd the nvese of the covaance matx of the pedctons, whch tself s sometmes unstable. 5 Othe estmatos lke SLS ae also possble. ( 6 In favo of the sgn-weghts, one could ague that n suffcently lage samples sgn j @ S j ŵ j s appoxmately equal to ρj jw j, whee the tem n backets measues the (postve coelaton between η j and ts poxy; see below fo esults that help justfy ths clam. So the tghte the connecton between η j, and the bette η j can be measued, the moe mpotant ηˆ j s n detemnng ŵ MIS Quately Vol. 39 o. Appendces/June 05 A3
Djksta & Hensele/Consstent Patal Least Squaes Path Modelng We conclude that PLS, Mode A, poduces estmated weght vectos that tend to vectos popotonal to the tue loadngs. One would lke to have a smple estmate fo the popotonalty facto. We popose hee as n Djksta (98, 00, 0 to defne : c w, whee the scala c s such that the off-dagonal elements of S ae epoduced as well as possble n a least squaes sense. So we mnmze the Eucldean dstance between 7 as a functon of c and obtan [ ( ] [ S dag( S ] ( ( ( ( and c w c w dag c w c w (7 ( ( w S dag S w c : w ( ww dag( ww w (8 a b b ab he use of dag means that only the off-dagonal elements of the matces ae taken nto account. So the numeato wthn backets s just wws a, and smlaly fo the denomnato. In suffcently lage samples, ĉ wll be well-defned, eal, and postve. (In all samples n ths pape and those n othe studes, ĉ attaned pope values. Its calculaton does not eque addtonal numecal optmzaton. It s staghtfowad to vefy, by eplacng S by Σ and ŵ by w, that the coecton does ts job: the matx n the denomnato equals the matx n the numeato, apat fom a facto, so c: plm c (9 ow, n patcula c w plm plm ( c w (0 It wll be useful to defne a populaton poxy η n by η n : w y n. Clealy, the squaed coelaton between a populaton poxy and ts coespondng latent vaable s R ( η, η ( w ( whch equals ( Σ ( ( + Θ ( Wth a lage numbe of hgh qualty ndcatos, ths coelaton can be close to one (: consstency at lage n PLS palance. A tvally deduced but mpotant algebac elatonshp s ( η, ηj ( Σj j ρj ( η, η ( ηj, ηj R w w R R ndcatng that the PLS poxes wll tend to undeestmate the squaed coelatons between the latent vaables. In fact, one can show that ths s tue fo multple coelatons as well (see Djksta 00. Also note that ( ηη j ( ( ( ( R, w w w c w w c so that we can estmate the (squaed qualty of the poxes consstently by (3 (4 7 hs assumes that the measuement eos wthn a block ae uncoelated, the basc desgn. If we know that some eos ae coelated o we have doubts about them, we can delete the tems fom the dffeence to be mnmzed. A4 MIS Quately Vol. 39 o. Appendces/June 05
Djksta & Hensele/Consstent Patal Least Squaes Path Modelng Moeove, wth ( ( R, : w w η η c ( j ( R, : w S w η η c j j (5 (6 we can estmate the coelatons between the latent vaables consstently (see Equaton 3 ρ : j R ( η, ηj ( η η R ( ηj ηj R,, (7 We close ths appendx wth fou obsevatons:. Standad PLS softwae fo Mode A wll poduce all the necessay, smple ngedents fo consstent estmaton.. he appoach can be and has been extended to Mode B (Djksta 98, 00, 0, but snce Mode A s numecally moe stable and faste than Mode B, t has fst poty. 3. In the man body of ths pape, we used ρ(η fo R(η, η. 4. In the man body of ths pape, we used (wthout subscpt ( hs s just η, η. R ρ A ( ( w S dag S w, : ( ww w w, w dag w, w w ( ( (8 Refeences Camé, H. 946. Mathematcal Models of Statstcs (Volume 9, Pnceton, J: Pnceton Unvesty Pess. Djksta,. K. 98. Latent Vaables n Lnea Stochastc Models: Reflectons on Maxmum Lkelhood and Patal Least Squaes Methods, Ph.D. hess, Gonngen Unvesty. (A second edton was publshed n 985 by Socometc Reseach Foundaton, Amstedam. Djksta,. K. 00. Latent Vaables and Indces: Heman Wold s Basc Desgn and Patal Least Squaes, n Handbook of Patal Least Squaes: Concepts, Methods, and Applcatons,V. E. Vnz, W. W. Chn, J. Hensele, and H. Wang (eds., ew Yok: Spnge, pp. 3-46. Djksta,. K. 0. Consstent Patal Least Squaes Estmatos fo Lnea and Polynomal Facto Models, wokng pape, Unvesty of Gonngen, Gonngen, he ethelands (http://www.ug.nl/staff/t.k.djksta/eseach. Djksta,. K. 03. A ote on How to Make PLS Consstent, wokng pape, Unvesty of Gonngen, Gonngen, he ethelands (http://www.ug.nl/staff/t.k.djksta/how-to-make-pls-consstent.pdf. Jöeskog, K. G. 969. A Geneal Appoach to Confmatoy Maxmum Lkelhood Facto Analyss, Psychometka (34:, pp. 83-0. Wold, H. O. A. 98. Soft Modelng: he Basc Desgn and Some Extensons, n Systems unde ndect obsevaton, K. G. Jöeskog and H. O. A. Wold (eds., Amstedam: oth-holland, pp. -54. MIS Quately Vol. 39 o. Appendces/June 05 A5