QUANT EXAMPLE ANALYSIS

Transcription

1 QUANT EXAMPLE ANALYSIS This example des nt include backgrund/intrductin sectins, theretical supprt fr hyptheses, discussin f findings, limitatins, future research directins, cnclusins, etc. This is just an example f hw ne might slice up the analysis and reprt and interpret the findings. Data Screening Univariate: - Missing Data: RD1 had ne missing value, which we imputed with the median. We used median imputatin because RD1 is an rdinal variable (was measured using a Likert scale). Tw cntrls had missing values representing 5% r less f the sample size, s we imputed the missing values fr these cntinuus (scale) variables (incme 2 missing; and custmer interactins 16 missing) using the mean f all available values. - Outliers: All variables but ne (custmer interactins) were n rdinal scales with seven r fewer intervals, thus extreme value utliers d nt exist. Fr custmer interactins, we examined a bx plt fr utliers and fund tw respndents with exceptinally high values, hwever, we had n reasn t believe these were incrrect values, and we have n theretical basis fr remving them. Thus they remain simply as high respnses. - Nrmality Once again, since nearly all ur variables are based n Likert-type scales, we have n reasn t exclude variables based n skewness unless they exhibit n variance. Thus rather than testing skewness, we fcused n kurtsis. Kurtsis greater than r less than +/ indicates ptentially prblematic kurtsis (and therefre, lack f sufficient variance). All f the burnut frm management items had brderline kurtsis issues (abs value between 1 and 2). These are fairly brderline values and we will simply flag them fr ptential future issues in subsequent analyses. BC3 and BC4 hwever, had kurtsis values arund 3.0; therefre there is insufficient variance in thse items t retain them. Accrdingly, we have drpped thse tw items. Multivariate (tested after measurement mdel): - Linearity We tested linearity by perfrming curve estimatin regressin fr all direct effects in ur mdel. The results shw that the relatinships between variables are sufficiently linear (i.e., all p-values were less than 0.05), except between Autnmy and Prductivity; hwever, n curve estimatin was significant either. Accrdingly, we have left the relatinship in ur mdel, subject t trimming during subsequent analyses. - Hmscedasticity The results f the hmscedasticity test (scatter plt f zpred n zresid) indicate that the mediatrs and SatW are hmscedastic, but Reliability is slightly mre heterskedastic. As we will be mderating by gender and jb categry, we retested reliability fr each subgrup (male, female, csr, bcr) and fund it t be hmscedastic within each. - Multicllinearity We tested the Variable Inflatin Factr fr all f the exgenus variables simultaneusly. The VIFs were all less than 2.0, indicating that the exgenus variables are all distinct. (If 1

2 yu find that they are nt all within a gd range, yu can cite O Brian 2007 wh says that high VIFs aren t necessarily a cause f alarm.) Explratry Factr Analysis We cnducted an EFA using Maximum Likelihd 1 with Prmax rtatin 2 t see if the bserved variables laded tgether as expected, were adequately crrelated, and met criteria f reliability and validity. We address each f these belw fr the final seven-factr mdel depicted in the pattern matrix belw: - Adequacy: The KMO and Bartlett s test fr sampling adequacy was significant and the cmmunalities fr each variable were sufficiently high (all abve and mst abve 0.600), thus indicating the chsen variables were adequately crrelated fr a factr analysis. Additinally, the reprduced matrix had nly 2% nn-redundant residuals greater than 0.05, further cnfirming the adequacy f the variables and 7-factr mdel. (If individual items have lw cmmunalities (like less than 0.200), yu might d yurself a favr by remving them. These items are prbably the nes that als had kurtsis issues.) - Reliability: The Crnbach s alphas fr the extracted factrs are shwn belw, alng with their labels and specificatin. All alpha s were abve 0.70 except Unsupprtive Cwrkers which was very clse at The factrs are all reflective because their indicatrs are highly crrelated and are largely interchangeable (Jarvis et al. 2003). Factr Label Crnbach s alpha Specificatin Feedback Reflective Reliability Reflective Resurce Demand Gap Reflective Learning Orientatin Reflective Autnmy Reflective Unsupprtive cwrkers Reflective Satisfactin with wrk Reflective 1 Maximum Likelihd Estimatin was chsen in rder t determine unique variance amng items and the crrelatin between factrs, and als t remain cnsistent with ur subsequent CFA. Maximum Likelihd als prvides a gdness f fit test fr the factr slutin. 2 Prmax was chsen because the dataset is quite large (n=304) and prmax can accunt fr the crrelated factrs. 2

3 - Validity: The factrs demnstrate sufficient cnvergent validity, as their ladings were all abve the recmmended minimum threshld f fr a samples size f 300 (Hair et al., 2010). The factrs als demnstrate sufficient discriminant validity, as the crrelatin matrix shws n crrelatins abve 0.700, and there are n prblematic crss-ladings. Pattern Matrix a Factr FB RL RD LO AU UC SW f1.884 f2.881 f3.861 f4.734 q2.806 q5.754 q1.712 q4.597 q3.547 rd3.895 rd4.741 rd2.698 rd1.601 l3.894 l1.831 l2.806 a1.897 a2.844 a3.753 uc2.830 uc1.633 uc3.528 sw1.908 sw3.609 sw2.472 Extractin Methd: Maximum Likelihd Estimatin. Rtatin Methd: Prmax with Kaiser Nrmalizatin. 3

4 This seven-factr mdel had a ttal variance explained f 60%, with all extracted factrs having eigenvalues abve 1.0 except ne, which was clse at Cnfirmatry Factr Analysis - Mdel Fit We remved RD3 due t pr lading. UC3 als was smewhat lw (0.58); hwever, we did nt remve it because the factr nly had three indicatrs, and a tw-indicatr factr ften results in instability. Mdificatin indices were cnsulted t determine if there was pprtunity t imprve the mdel. Accrdingly, we cvaried the errr terms between f3 and f4. The table belw indicates that the gdness f fit fr ur measurement mdel is sufficient. Metric Observed value Recmmended cmin/df Between 1 and 3 CFI >0.950 RMSEA <0.060 PCLOSE >0.050 SRMR < Validity and Reliability T test fr cnvergent validity we calculated the AVE. Fr all factrs, the AVE was abve 0.50 except fr Unsupprtive Cwrkers, which was clse at Hwever, as this factr is minimally crrelated with the ther factrs in the mdel, and because the reliability scre (0.716) was greater than 0.700, we felt this was admissible ( i.e., while it is nt especially strng internally, it is, at least, a reliable and distinct cnstruct within ur mdel). T test fr discriminant validity we cmpared the square rt f the AVE (n the diagnal in the matrix belw) t all inter-factr crrelatins. All factrs demnstrated adequate discriminant validity because the diagnal values are greater than the crrelatins. We als cmputed the cmpsite reliability fr each factr. In all cases the CR was abve the minimum threshld f 0.70, indicating we have reliability in ur factrs. (If yu experience prblems during this phase with AVE r CR, it is prbably because yu did nt have a gd EFA slutin. I wuld return t the EFA t wrk that ut first.) 4

5 CR AVE LearningO Feedback Reliability RDGap UnsCW Autnmy SatW Cmmn Methd Bias Because the data fr bth IVs and DVs was cllected using a single instrument (a survey), we cnducted a cmmn methd bias test t determine if a methd bias was affecting the results f ur measurement mdel. The test we used was the unmeasured latent factr methd recmmended by Pdsakff et al. (2003) fr studies that d nt explicitly measure a cmmn factr (as in this study). Cmparing the standardized regressin weights befre and after adding the Cmmn Latent Factr (CLF) shws that nne f the regressin weights are dramatically affected by the CLF i.e., the deltas are less than and the CR and AVE fr each cnstruct still meet minimum threshlds. Nevertheless, t err n the cnservative side, we have pted t retain the CLF fr ur structural mdel (by imputing cmpsites in AMOS while the CLF is present), and thus we have CMB-adjusted values. (Retaining the CLF is nt required if yu find n CMB.) - Invariance Tests Since we are planning n mderating the structural mdel with tw categrical variables, we cnducted cnfigural and metric invariance tests. Gender: The mdel fit f the uncnstrained measurement mdels (with grups laded separately) had adequate fit (cmin/df = 1.423; CFI 0.942), indicating that the mdel is cnfigurally invariant. After cnstraining the mdels t be equal, we fund the chi-square difference test t be nn-significant (pval>0.05); thus, ur measurement mdel meets criteria fr metric invariance acrss gender as well. [nte t students] Had it nt met the criteria fr metric invariance, yu wuld need t lk at the differences between regressin weights fr the tw grups t see which regressin weight was mst different. This might then need t be remved if pssible. If nt pssible, yu might rely n MacKenzie et al Cnstruct Measurement and Validatin Prcedures in MIS and Behaviral 5

6 Research: Integrating New and Existing Techniques, wh say that as lng as ne item per cnstruct (aside frm the cnstrained ne) is metrically invariant, then yu can prceed with further invariance tests (like multi-grup mderatin). Yu can test this using the critical ratis apprach described in the vide called: multigrup mderatin in ams made easy. Jb categry The mdel fit fr jb categry was equally gd (cmin/df = 1.356; CFI 0.952). The chi-square difference test was again nn-significant (pval>0.05). Hyptheses All hyptheses were tested while cntrlling fr Educatin, Incme, and Number f custmers handled per day. Mediatin tests were cnducted withut the presence f mderatrs. Multi-grup mderatin tests were cnducted using the full mdel, but prir t adding the interactin variables. Interactin effects were tested using the full dataset, rather than the mderated dataset. These prcedures were necessary in rder t have enugh pwer t test each set f hyptheses, and in rder t maintain theretical clarity and parsimny. [nte t students] Yu wuld f curse als prvide here sme theretical lgic fr why yu included the cntrls yu included and fr why yu expect the hypthesized relatinships t be bserved as hypthesized. Mediatin H1a. Learning Orientatin mediates the negative relatinship between Resurces demand gap and Satisfactin with wrk. H1b. Learning Orientatin mediates the negative relatinship between Resurces demand gap and Reliability. Multi-grup mderatin H2a. The psitive relatinship between Autnmy and Satisfactin with wrk will be strnger fr males than fr females. H2b. The psitive relatinship between Autnmy and Reliability will be strnger fr males than fr females. Interactin H3a. An increase in Unsupprtive Cwrkers will strengthen the negative relatinship between Resurce Demand Gap and Learning Orientatin. H3b. An increase in Unsupprtive Cwrkers will weaken the psitive relatinship between Feedback and Learning Orientatin. 6

7 Structural Mdel - Create Cmpsites frm factr scres Cmpsite variables were created using factr scres in AMOS while the CLF was present. (This is nt necessary, but ptinal. Yu may retain the full structural mdel if yu desire it just gets a bit unwieldy with interactins.) Interactin terms were created by standardizing the apprpriate variables, and then multiplying them. - Mdel Fit (f initial structural mdel after fitting i.e., nt during mderatin tests). The fitted structural mdel demnstrates adequate fit. In rder t achieve gd fit, we were required t add a direct path between resurce demand gap and satisfactin with wrk, as well as between unsupprtive cwrkers and satisfactin with wrk. We felt these additins were theretically lgical, and prbably indicate that the hypthesized mediatin is actually partial rather than full. We additinally cvaried the errr terms f the mediatrs, as we wanted t accunt fr their crrelatin withut adding theretical cmplexity t ur mdel. T remain cnsistent, 3 we cvaried the errr terms f the dependent variables. While there may exist causal relatinships between these variables, this is nt the fcus f this mdel. The actins we have taken allw us t accunt fr these ptential crrelatins withut having t explicitly therize and test them. Metric Observed value Recmmended cmin/df Between 1 and 3 CFI >0.950 RMSEA <0.060 PCLOSE >0.050 SRMR < Cntrls The cntrls did nt have a significant impact n either dependent variable, except the number f custmers handled per day had a slight negative effect n Satisfactin with wrk (standardized beta = *). 3 This issue f cnsistently applying theretical reasning when cvarying errr terms is advcated by David Kenny: He als recmmends this actin be cnsidered especially when the mdificatin indices indicate that such an actin wuld significantly reduce the chi-square. This secnd criteria was als true fr this mdel. 7

8 - Hypthesis testing Mediatin Mediatin was tested using 2000 bias crrected btstrapping resamples in AMOS. The direct and indirect effects were analyzed fr ptential partial mediatin (discvered while fitting the mdel). Just indirect effects were analyzed fr establishing full mediatin. The results are summarized in the Hyptheses Summary table belw. [nte t students] In additin t btstrapping, yu may want t fllw the Barn and Kenny apprach (direct effects tested, then add mediatr, then see if direct effects drp). Multi-grup Mderatin T test the categrical mderatin hyptheses, we prduced the critical ratis fr the differences in regressin weights between grups. Frm these critical ratis we calculated p-values t determine the significance f the difference. The results are summarized in the Hypthesis Summary table belw. Interactin T test the interactin hyptheses we first standardized the IVs and then created prduct variables. We then trimmed nn-significant interactin regressins ne at a time until nly significant paths remained. In this case, nly ne significant path remained, frm RDxUC t LO. We pltted this interactin as shwn belw. The results f the interactin tests are summarized in the Hypthesis Summary table belw. Additinally, we bserved that mdel fit was gd (cmin/df = 1.644; CFI 0.981) fr the final mderated mdel. 8

9 Hypthesis Summary Table Mediatin Evidence Supprted? H1a. Learning Orientatin mediates the negative relatinship between Resurces demand gap and Satisfactin with wrk. Direct w/ Med: -.372*** Direct w/ Med:0.237*** Indirect: -.124*** Yes: Partial Mediatin H1b. Learning Orientatin mediates the negative relatinship between Resurces demand gap and Reliability. Multi-grup mderatin H2a. The psitive relatinship between Autnmy and Satisfactin with wrk will be strnger fr males than fr females. H2b. The psitive relatinship between Autnmy and Reliability will be strnger fr males than fr females. Interactin H3a. An increase in Unsupprtive Cwrkers will strengthen the negative relatinship between Resurce Demand Gap and Learning Orientatin. H3b. An increase in Unsupprtive Cwrkers will weaken the psitive relatinship between Feedback and Learning Orientatin. Direct w/ Med: -.182*** Direct w/ Med: 0.056(ns) Indirect: -.088*** Males: 0.486*** Females: 0.267*** Zscre: -2.62*** Males: *** Females: *** Zscre: 0.545(ns) Interactin effect: * Interactin effect: 0.037(ns) Yes: Full Mediatin Yes: Strnger fr males N: N difference Yes: Strnger negative effect N: N Effect 9