IMPS 8, Durham, NH Linar Non-Gaussian Structural Equation Modls Shohi Shimizu, Patrik Hoyr and Aapo Hyvarinn Osaka Univrsity, Japan Univrsity of Hlsinki, Finland
Abstract Linar Structural Equation Modling (linar SEM) Analyzs causal rlations Covarianc-basd SEM Uss covarianc structur alon for modl idntification A numbr of indistinguishabl modls Linar non-gaussian SEM Uss non-gaussian structurs for modl idntification Maks many modls distinguishabl
SEM and causal analysis SEM is oftn usd for causal analysis basd on non-primntal data Assumption: th data gnrating procss is rprsntd by a SEM modl If th assumption is rasonabl, SEM provids causal information 3
Limitations of covarianc-basd SEM Covarianc-basd SEM cannot distinguish btwn many modls Eampl 4
Linar non-gaussian SEM Many obsrvd data ar considrably non- Gaussian (Miccri, 989; Hyvarinn t al. ) Non-Gaussian structurs of data ar usful (Bntlr 983; Mooijaart 985) Non-Gaussianity distinguish btwn th two modls (Shimizu t al. 6) : 5
Indpndnt componnt analysis (ICA) Obsrvd random vctor is modld as As s i ar indpndnt and non-gaussian Zro mans and unit variancs A is a constant matri Typically squar, # variabls # indpndnt componnts Idntifiabl up to prmutation of th columns (Mooijaart 985; Comon, 994) 6
ICA stimation An altrnativ prssion of ICA (As): ~ whr W A s calld a ~, W rcovring Find such W that maimizs indpndnc of componnts of s ˆ W Many proposals (Hyvarinn t al. ) matri W ~ is stimatd up to prmutation of th rows: W PW ~ 7
ICA stimation An altrnativ prssion of ICA (As): ~ whr W A s calld a ~, W rcovring Find such W that maimizs indpndnc of componnts of s ˆ W Many proposals (Hyvarinn t al. ) matri W ~ is stimatd up to prmutation of th rows: W PW ~ 8
ICA stimation An altrnativ prssion of ICA (As): ~ whr W A s calld a ~, W rcovring Find such W that maimizs indpndnc of componnts of s ˆ W Many proposals (Hyvarinn t al. ) matri W ~ is stimatd up to prmutation of th rows: W PW ~ 9
Discovry of linar non-gaussian acyclic modls Shimizu, Hoyr, Hyvarinn and Krminn (6)
Linar non-gaussian acyclic modl (LiNGAM) Dirctd acyclic graphs (DAG) i can b arrangd in a ordr k(i) Assumptions: Linarity Etrnal influncs ar indpndnt and ar non-gaussian i i b j + k ( ij j) < k ( i) i or B +
Goal W know Data X is gnratd by W do NOT know Path cofficints: bij Ordrs k(i) Etrnal influncs: i B + What w obsrv is data X only Goal Estimat B and k(i) using data X only!
Ky ida First, rlat LiNGAM with ICA as follows: B + ( I B) A - ICA! ~ quivalntly ( I B) W Du to th prmutation indtrminacy, ICA givs: W PW ~ Can find a corrct P Th corrct prmutation is th only on that has no zros in th diagonal 3
Ky ida First, rlat LiNGAM with ICA as follows: B + ( I B) A - ICA! ~ quivalntly ( I B) W Du to th prmutation indtrminacy, ICA givs: W PW ~ Can find a corrct P Th corrct prmutation is th only on that has no zros in th diagonal 4
Ky ida First, rlat LiNGAM with ICA as follows: B + ( I B) A - ICA! ~ quivalntly ( I B) W Du to th prmutation indtrminacy, ICA givs: W PW ~ Can find a corrct P Th corrct prmutation is th only on that has no zros in th diagonal 5
Ky ida First, rlat LiNGAM with ICA as follows: B + ( I B) A - ICA! ~ quivalntly ( I B) W Du to th prmutation indtrminacy, ICA givs: W PW ~ Can find th corrct P Th corrct prmutation is th only on that has no zros in th diagonal 6
Illustrativ ampl Considr th modl:.6 443 B +.6 Goal Estimat th path dirction btwn and obsrving only and 7
Prform ICA Rlation of th LiNGAM modl with ICA: ~ W.6 443 W ~ Du to th prmutation indtrminacy, ICA might giv: ( ~ ) W PW.8 8
Prform ICA Rlation of th LiNGAM modl with ICA: ~ W.6 443 W ~ Du to th prmutation indtrminacy, ICA might giv: W ( ~ ) PW.6 9
Find th corrct P Find a prmutation of th rows of W so that it has no zros in th diagonal In th ampl.8 443.6 443 Prmut th rows W W ~
Find th corrct P Find a prmutation of th rows of W so that it has no zros in th diagonal In th ampl.8 443.6 443 Prmut th rows W W ~
Find th corrct P Find a prmutation of th rows of W so that it has no zros in th diagonal In th ampl.6 443.6 443 Prmut th rows W W ~
Find th corrct P In practic, Pˆ ma P ( ) P T W ii Havily pnalizs small absolut valus in th diagonal 3
Simulations: Estimation of B Both supr- and sub-gaussian trnal influncs tstd 5 datasts cratd for ach scattrplot B randomly gnratd at ach trial 3 3 3 - - - Numbr of variabls 5 Estimatd bij - -3-3 - - 3 3 - - -3-3 - - 3 3 - -3-3 - - 3 3 - - -3-3 - - 3 3 - -3-3 - - 3 3 - - -3-3 - - 3 3 - - - - - - -3-3 - - 3-3 -3 - - 3-3 -3 - - 3 Gnrating bij, 3, Numbr of obsrvations 4
5 Prun B () In practic, du to stimation rrors, w would gt: Nd to find which path cofficints ar actually zros +.5.65 4 43 4 4 B
Find a prmutation that givs a lowr triangular matri Th LiNGAM modl is acyclic Th matri B can b prmutd to b lowr triangular for som prmutation of variabls (Bolln, 989) First, find a simultanous prmutation of rows and columns of B that givs a lowr-triangular B In practic, find a prmutation matri Q that minimizs th sum of th lmnts in its uppr T Qˆ ma QBQ triangular part: ( ) ij Q i j 6
Find a prmutation that givs a lowr triangular matri Th LiNGAM modl is acyclic Th matri B can b prmutd to b lowr triangular for som prmutation of variabls (Bolln, 989) First, find a simultanous prmutation of rows and columns of B that givs a lowr-triangular B In practic, find a prmutation matri Q that minimizs th sum of th lmnts in its uppr T Qˆ ma QBQ triangular part: ( ) ij Q i j 7
Find a prmutation that givs a lowr triangular matri Th LiNGAM modl is acyclic Th matri B can b prmutd to b lowr triangular for som prmutation of variabls (Bolln, 989) First, find a simultanous prmutation of rows and columns of B that givs a lowr-triangular B In practic, find a prmutation matri Q that minimizs th sum of th lmnts in its uppr triangular part: ( T Qˆ min QBQ ) ij Q i j 8
9 Gt a lowr-triangular B +.5.65 +.6.5 B Applying such a simultanous prmutation of th rows and columns, w gt a prmutd B that is as lowr-triangular as possibl St th uppr-triangular lmnts to b zros T QBQ
3 Gt a lowr-triangular B +.5.65 +.65.5 B Applying such a simultanous prmutation of th rows and columns, w gt a prmutd B that is as lowr-triangular as possibl St th uppr-triangular lmnts to b zros T QBQ
3 Gt a lowr-triangular B +.5.65 +.65.5 B Applying such a simultanous prmutation of th rows and columns, w gt a prmutd B that is as lowr-triangular as possibl St th uppr-triangular lmnts to b zros T QBQ -.5
3 Gt a lowr-triangular B +.5.65 +.65.5 B Applying such a simultanous prmutation of th rows and columns, w gt a prmutd B that is as lowr-triangular as possibl St th uppr-triangular lmnts to b zros T QBQ
33 Pruning B () Onc w gt a lowr-triangular B, th modl is idntifiabl using covarianc-basd SEM Many isting mthods can b usd for pruning th rmaining path cofficints Wald tst, Bootstrapping, Modl fit Lasso-typ stimators (Tibshirani 996; Zou, 6) tc. +.65
To summariz th procdur. Estimat B ICA + finding th corrct row prmutation. Prun stimatd B. Find a row-and-column prmutation that maks stimatd B lowr triangular. Prun rmaining paths using a covarianc-basd mthod. Estimat B. Prun stimatd B 4 4 4 3 3 3 34
Summary of th rgular LiNGAM A linar acyclic modl is idntifiabl basd on non-gaussianity ICA-basd stimation works wll Confidnc intrvals (Konya t al., in progrss) Bttr pruning mthods might b dvlopd Imposing sparsnss in th ICA stag (Zhang & Chang, 6; Hayashi t al. in progrss) lik Lasso (Tibshirani 996) 35
Som tnsions
Latnt factors (Shimizu t al., 7) A non-gaussian multipl indicator modl: f Bf + d Gf + Suppos that G is idntifid, thn B is idntifid Could idntify G in a data drivn way using a ttradconstraint-basd mthod (Silva t al., 6) 37
Latnt classs (Shimizu & Hyvarinn, 8) LiNGAM modl for ach class q: B + ( I B ) ì + ì + A - ICA! q q q q q q q ICA miturs (L t al., ; Mollah t al., 6) Class :.9 Class : 6. 5 38
Unobsrvd confoundrs (Hoyr t al., in prss Can idntify and distinguish btwn mor modls.. 3. 4. 5. 6. u u u 39
Tim structurs (Hyvarinn t al., 8) Combining LiNGAM and autorgrssiv modl: k ( t) B ( t ) + ( t) In conomtrics: Structural vctor autorgrssion (Swanson & Grangr, 997) Changs ordinary AR cofficints basd on instantanous ffcts: B ( I B ) for ( : AR matri) M > M 4
Som variabls ar Gaussian (Hoyr t al., 8) Considr th modl:.6 Can idntify th path dirction if ithr of or is non-gaussian In gnral, thr ist svral quivalnt modls that ntail th sam distribution if som ar Gaussian 4
Som othr tnsions Cyclic modls (Lacrda t al., 8) Fwr quivalnt modls than covariancbasd approach Nonlinarity (Zhang & Chan, 7; Sun t al., 7) Modl fit statistics ar undr dvlopmnt Non-Gaussian structurs 4
Conclusion Us of non-gaussianity in SEM is usful for modl idntification Many obsrvd data ar considrably non-gaussian Th non-gaussian approach can b a good option 43
Most of our paprs and Matlab/Octav cod ar availabl on our wbpags Googl will find us! 44