Linear Non-Gaussian Structural Equation Models

Size: px

Start display at page:

Download "Linear Non-Gaussian Structural Equation Models"

Percival Reed
5 years ago
Views:

1 IMPS 8, Durham, NH Linar Non-Gaussian Structural Equation Modls Shohi Shimizu, Patrik Hoyr and Aapo Hyvarinn Osaka Univrsity, Japan Univrsity of Hlsinki, Finland

2 Abstract Linar Structural Equation Modling (linar SEM) Analyzs causal rlations Covarianc-basd SEM Uss covarianc structur alon for modl idntification A numbr of indistinguishabl modls Linar non-gaussian SEM Uss non-gaussian structurs for modl idntification Maks many modls distinguishabl

3 SEM and causal analysis SEM is oftn usd for causal analysis basd on non-primntal data Assumption: th data gnrating procss is rprsntd by a SEM modl If th assumption is rasonabl, SEM provids causal information 3

4 Limitations of covarianc-basd SEM Covarianc-basd SEM cannot distinguish btwn many modls Eampl 4

5 Linar non-gaussian SEM Many obsrvd data ar considrably non- Gaussian (Miccri, 989; Hyvarinn t al. ) Non-Gaussian structurs of data ar usful (Bntlr 983; Mooijaart 985) Non-Gaussianity distinguish btwn th two modls (Shimizu t al. 6) : 5

6 Indpndnt componnt analysis (ICA) Obsrvd random vctor is modld as As s i ar indpndnt and non-gaussian Zro mans and unit variancs A is a constant matri Typically squar, # variabls # indpndnt componnts Idntifiabl up to prmutation of th columns (Mooijaart 985; Comon, 994) 6

7 ICA stimation An altrnativ prssion of ICA (As): ~ whr W A s calld a ~, W rcovring Find such W that maimizs indpndnc of componnts of s ˆ W Many proposals (Hyvarinn t al. ) matri W ~ is stimatd up to prmutation of th rows: W PW ~ 7

8 ICA stimation An altrnativ prssion of ICA (As): ~ whr W A s calld a ~, W rcovring Find such W that maimizs indpndnc of componnts of s ˆ W Many proposals (Hyvarinn t al. ) matri W ~ is stimatd up to prmutation of th rows: W PW ~ 8

9 ICA stimation An altrnativ prssion of ICA (As): ~ whr W A s calld a ~, W rcovring Find such W that maimizs indpndnc of componnts of s ˆ W Many proposals (Hyvarinn t al. ) matri W ~ is stimatd up to prmutation of th rows: W PW ~ 9

10 Discovry of linar non-gaussian acyclic modls Shimizu, Hoyr, Hyvarinn and Krminn (6)

11 Linar non-gaussian acyclic modl (LiNGAM) Dirctd acyclic graphs (DAG) i can b arrangd in a ordr k(i) Assumptions: Linarity Etrnal influncs ar indpndnt and ar non-gaussian i i b j + k ( ij j) < k ( i) i or B +

12 Goal W know Data X is gnratd by W do NOT know Path cofficints: bij Ordrs k(i) Etrnal influncs: i B + What w obsrv is data X only Goal Estimat B and k(i) using data X only!

13 Ky ida First, rlat LiNGAM with ICA as follows: B + ( I B) A - ICA! ~ quivalntly ( I B) W Du to th prmutation indtrminacy, ICA givs: W PW ~ Can find a corrct P Th corrct prmutation is th only on that has no zros in th diagonal 3

14 Ky ida First, rlat LiNGAM with ICA as follows: B + ( I B) A - ICA! ~ quivalntly ( I B) W Du to th prmutation indtrminacy, ICA givs: W PW ~ Can find a corrct P Th corrct prmutation is th only on that has no zros in th diagonal 4

15 Ky ida First, rlat LiNGAM with ICA as follows: B + ( I B) A - ICA! ~ quivalntly ( I B) W Du to th prmutation indtrminacy, ICA givs: W PW ~ Can find a corrct P Th corrct prmutation is th only on that has no zros in th diagonal 5

16 Ky ida First, rlat LiNGAM with ICA as follows: B + ( I B) A - ICA! ~ quivalntly ( I B) W Du to th prmutation indtrminacy, ICA givs: W PW ~ Can find th corrct P Th corrct prmutation is th only on that has no zros in th diagonal 6

17 Illustrativ ampl Considr th modl: B +.6 Goal Estimat th path dirction btwn and obsrving only and 7

18 Prform ICA Rlation of th LiNGAM modl with ICA: ~ W W ~ Du to th prmutation indtrminacy, ICA might giv: ( ~ ) W PW.8 8

19 Prform ICA Rlation of th LiNGAM modl with ICA: ~ W W ~ Du to th prmutation indtrminacy, ICA might giv: W ( ~ ) PW.6 9

20 Find th corrct P Find a prmutation of th rows of W so that it has no zros in th diagonal In th ampl Prmut th rows W W ~

21 Find th corrct P Find a prmutation of th rows of W so that it has no zros in th diagonal In th ampl Prmut th rows W W ~

22 Find th corrct P Find a prmutation of th rows of W so that it has no zros in th diagonal In th ampl Prmut th rows W W ~

23 Find th corrct P In practic, Pˆ ma P ( ) P T W ii Havily pnalizs small absolut valus in th diagonal 3

24 Simulations: Estimation of B Both supr- and sub-gaussian trnal influncs tstd 5 datasts cratd for ach scattrplot B randomly gnratd at ach trial Numbr of variabls 5 Estimatd bij Gnrating bij, 3, Numbr of obsrvations 4

25 5 Prun B () In practic, du to stimation rrors, w would gt: Nd to find which path cofficints ar actually zros B

26 Find a prmutation that givs a lowr triangular matri Th LiNGAM modl is acyclic Th matri B can b prmutd to b lowr triangular for som prmutation of variabls (Bolln, 989) First, find a simultanous prmutation of rows and columns of B that givs a lowr-triangular B In practic, find a prmutation matri Q that minimizs th sum of th lmnts in its uppr T Qˆ ma QBQ triangular part: ( ) ij Q i j 6

27 Find a prmutation that givs a lowr triangular matri Th LiNGAM modl is acyclic Th matri B can b prmutd to b lowr triangular for som prmutation of variabls (Bolln, 989) First, find a simultanous prmutation of rows and columns of B that givs a lowr-triangular B In practic, find a prmutation matri Q that minimizs th sum of th lmnts in its uppr T Qˆ ma QBQ triangular part: ( ) ij Q i j 7

28 Find a prmutation that givs a lowr triangular matri Th LiNGAM modl is acyclic Th matri B can b prmutd to b lowr triangular for som prmutation of variabls (Bolln, 989) First, find a simultanous prmutation of rows and columns of B that givs a lowr-triangular B In practic, find a prmutation matri Q that minimizs th sum of th lmnts in its uppr triangular part: ( T Qˆ min QBQ ) ij Q i j 8

29 9 Gt a lowr-triangular B B Applying such a simultanous prmutation of th rows and columns, w gt a prmutd B that is as lowr-triangular as possibl St th uppr-triangular lmnts to b zros T QBQ

30 3 Gt a lowr-triangular B B Applying such a simultanous prmutation of th rows and columns, w gt a prmutd B that is as lowr-triangular as possibl St th uppr-triangular lmnts to b zros T QBQ

31 3 Gt a lowr-triangular B B Applying such a simultanous prmutation of th rows and columns, w gt a prmutd B that is as lowr-triangular as possibl St th uppr-triangular lmnts to b zros T QBQ -.5

32 3 Gt a lowr-triangular B B Applying such a simultanous prmutation of th rows and columns, w gt a prmutd B that is as lowr-triangular as possibl St th uppr-triangular lmnts to b zros T QBQ

33 33 Pruning B () Onc w gt a lowr-triangular B, th modl is idntifiabl using covarianc-basd SEM Many isting mthods can b usd for pruning th rmaining path cofficints Wald tst, Bootstrapping, Modl fit Lasso-typ stimators (Tibshirani 996; Zou, 6) tc. +.65

34 To summariz th procdur. Estimat B ICA + finding th corrct row prmutation. Prun stimatd B. Find a row-and-column prmutation that maks stimatd B lowr triangular. Prun rmaining paths using a covarianc-basd mthod. Estimat B. Prun stimatd B

35 Summary of th rgular LiNGAM A linar acyclic modl is idntifiabl basd on non-gaussianity ICA-basd stimation works wll Confidnc intrvals (Konya t al., in progrss) Bttr pruning mthods might b dvlopd Imposing sparsnss in th ICA stag (Zhang & Chang, 6; Hayashi t al. in progrss) lik Lasso (Tibshirani 996) 35

36 Som tnsions

37 Latnt factors (Shimizu t al., 7) A non-gaussian multipl indicator modl: f Bf + d Gf + Suppos that G is idntifid, thn B is idntifid Could idntify G in a data drivn way using a ttradconstraint-basd mthod (Silva t al., 6) 37

38 Latnt classs (Shimizu & Hyvarinn, 8) LiNGAM modl for ach class q: B + ( I B ) ì + ì + A - ICA! q q q q q q q ICA miturs (L t al., ; Mollah t al., 6) Class :.9 Class :

39 Unobsrvd confoundrs (Hoyr t al., in prss Can idntify and distinguish btwn mor modls u u u 39

40 Tim structurs (Hyvarinn t al., 8) Combining LiNGAM and autorgrssiv modl: k ( t) B ( t ) + ( t) In conomtrics: Structural vctor autorgrssion (Swanson & Grangr, 997) Changs ordinary AR cofficints basd on instantanous ffcts: B ( I B ) for ( : AR matri) M > M 4

41 Som variabls ar Gaussian (Hoyr t al., 8) Considr th modl:.6 Can idntify th path dirction if ithr of or is non-gaussian In gnral, thr ist svral quivalnt modls that ntail th sam distribution if som ar Gaussian 4

42 Som othr tnsions Cyclic modls (Lacrda t al., 8) Fwr quivalnt modls than covariancbasd approach Nonlinarity (Zhang & Chan, 7; Sun t al., 7) Modl fit statistics ar undr dvlopmnt Non-Gaussian structurs 4

43 Conclusion Us of non-gaussianity in SEM is usful for modl idntification Many obsrvd data ar considrably non-gaussian Th non-gaussian approach can b a good option 43

44 Most of our paprs and Matlab/Octav cod ar availabl on our wbpags Googl will find us! 44

Data Assimilation 1. Alan O Neill National Centre for Earth Observation UK

Data Assimilation 1. Alan O Neill National Centre for Earth Observation UK Data Assimilation 1 Alan O Nill National Cntr for Earth Obsrvation UK Plan Motivation & basic idas Univariat (scalar) data assimilation Multivariat (vctor) data assimilation 3d-Variational Mthod (& optimal