A MISAPPLICATION FOR WEIGHT EVALUATION DETERMINED BY THE PRINCIPAL COMPONENT ANALYSIS

Transcription

1 Journal of heoretcal and Appled Informaton echnology 8 th February 03. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: A MISAPPLICAION FOR WEIGH EVALUAION DEERMINED BY HE PRINCIPAL COMPONEN ANALYSIS RUNCHUN HAN JIXIAN XIAO JIE YANG HONGWEI DU Hebe Unted Unversty ABSRAC Prncpal component analyss s the study of how a small number of uncorrelated prncpal components ndcate the nternal structure of multple varables. he purpose s to smplfy the data reduce the dmenson of redundances and make the new varable uncorrelated. So t s wdely utlzed n multvarate statstcal analyss. hese years however some people apply the prncpal component analyss to determne the weght of evaluaton ndex system even nclude the method n the textbook. hs paper wll clarfy the unscentfc method of valung the weght n four aspects. Key words: Prncpal Component Analyss Evaluaton System Weghts. INRODUCION Prncpal component analyss was frstly suggested by the U.S. psychologst Chares Spearman n 904. he basc dea s to measure a number of ndcators wth a few uncorrelated prncpal component ndcators (lnear combnaton of orgnal ndcators) whch can reflect the man message of orgnal observatonal ndcators. hs data processng n statstcal work s frequently used to remove redundant ndcators and get a more obectve and scentfc evaluaton however n recent years some people use PCA to select the weght of evaluaton ndex system ranges from the master's thess doctoral dssertaton to research papers and scentfc research proects PCA method s wdely used and known as the law of obectvely valung weghts. In reference [] t ntroduces the method of how to use PCA determne the weght. he reference [] sad "compared wth subectve weght method the obectve weght method s drectly based on the orgnal nformaton and obtan the weght such as component analyss factor analyss and the detals are n the reference. " Accordng to reference [] and other data above the prncpal component analyss method s to determne the weghts as follows: Suppose there are p ndcators where ( = n; = n) means the a th observaton of the th ndcator.. Frstly we use standard normal method to elmnate the mpact of dmenson namely u a a σ = where a means the average of ndcator and σ s the standard devaton of ndcator. u = u ) ( u u Secondly calculate the covarance matrx W of u u u the egenvalue p λ p of W and the correspondng λ λ n e e e normalzed egenvector p where e = e e e = ( ) p p hen usng contrbuton rate of the egenvalues to determne the prncpal components and ther number λ + λ + + λl m = mn l : 80% λ + λ + + λ p Fk = e ku + eku + + e pku p k = m hen we can get the weght λk wk = k = m m λ = 743

2 Journal of heoretcal and Appled Informaton echnology 8 th February 03. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: At last calculate the score F = w F + w F + + w F m m As we all know to carry out any evaluaton has ts specfc purpose and the ndex system s served for ths purpose whle the weghts are used to reflect the mportant level of each ndcator by evaluators to acheve an evaluaton purpose wthout such purpose the ndex system s worthless. It s dffcult to fnd out what knd of purpose and background Chares Spearman had to use prncpal component analyss to determne the weghts and how to prove such weghts s n lne wth the wshes of the evaluators at least however now people use ths method wthout suffcent demonstraton or deducton. herefore I beleve that the applcaton of the method to determne weght s lack of theoretcal bass.. WIH PRINCIPAL COMPONEN ANALYSIS MEHOD O DEERMINE HE WEIGH HE WRONG WAY In order to get enough sample space and pass KMO and Bartlett's test we randomly choose 40 schools as the samples from the report about the world unversty rankngs 008 publshed by the mes and clarfy that usng prncpal components analyss to determne the evaluaton weghts s unscentfc from four aspects. he orgnal data collected as follows: able. he Orgnal Data able RANK PEER REVIEW EMPLOYER REVIEW SCOR SAFF/SUDEN CIAIONS/SASS INERNAIONAL SAFF INERNAIONAL SUDEN RANK PEER REVIEW EMPLOYER REVIEW SCOR SAFF/SUDEN CIAIONS/SASS INERNAIONAL SAFF INERNAIONAL SUDEN For Increasng Or Decreasng he Sample It Is Precarous o Make he Prncpal Component As he Evaluaton Weghts. Based on the method from the reference [] we use SPSS for a capacty of 40 schools to determne the weght and acheve ntegrated rank. Usng prncpal component analyss by SPSS and get: able. KMO And Bartlett's est Kaser-Meyer-Olkn Measure of Samplng Adequacy. Bartlett's est of Sphercty.705 Approx. Ch- Square df 5 Sg

3 Journal of heoretcal and Appled Informaton echnology 8 th February 03. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: PEER REVIEW EMPLOYER REVIEW SCOR SAFF/SUDEN CIAIONS/SASS INERNAIONAL SAFF able 3. Component Matrx Component INERNAIONAL SUDEN Extracton Method: Prncpal Component Analyss. And able 4. otal Varance Explaned Compon ent otal Intal Egenvalues % of Varance Cumulatve % Extracton Method: Prncpal Component Analyss. From able KMO =0.705> 0.7 t shows the effect of factor (prncpal component) analyss s accredted and accordng to the probablty value (p<0.05) of Bartett s sphercty test the ndependence of the varables assumpton should be reected so the practcalty testng of factor analyss holds. If only test the prncpal components the cumulatve contrbuton rate would be 78.64% less than 80% so we take 3 nstead and the cumulatve contrbuton rate rses up to 9.4% so takng m=3. If we transform the coeffcents of able 3 Component Matrx nto orthogonal unts then: e=( ) e=( ) e3=( ) So the prncpal components are: F= u+0.969u u u4+0.57u u6 F= u u u u u u6 F3= u u u u u u6 Where u u u3 u4 u5 u6 are PEER REVIEW EMPLOYER REVIEW SCOR SAFF / SUDEN CIAIONS / SASS INERNAIONAL SAFF INERNAIONAL SUDEN standardzed data respectvely. Calculated from able 4 the weghts of the 3 prncpal components are: So the evaluaton score s: F= F+0.5F F3 able5 Symmetrc Measures Value Asymp. Std. Errora Approx. b Approx. Sg. Measure of Kappa Agreement N of Vald Cases 39 a. Not assumng the null hypothess. b. Usng the asymptotc standard error assumng the null hypothess. Accordng to the formula we get the ntegrated rank. able 9 shows the rankng result of the 40 sample schools. Now we remove one rankng observaton such as the 8th school and for the rest 39 samples KMO = smlar to 0.7 whch ndcates the effect of factor (prncpal component) analyss of s accredted and accordng to the probablty value (p<0.05) of Bartett s sphercty test the ndependence of the varables assumpton should be reected so the practcalty testng of factor analyss holds. Re-use the same method to determne the weght and the ntegrated rank at last get the ntegrated rankng samples (see able 9 k ). Among the 40 schools the ntegrated rank of the 8th school s 3. When removng t the rankng of the frst 3 schools n the rest 39-school sample 745

4 Journal of heoretcal and Appled Informaton echnology 8 th February 03. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: should be the same as n the 40-school sample whle the 33rd 34th n the 40-school sample should be the 3nd 33rd n the 39-school sample respectvely and so on. If we reset the 3nd rank as 33rd of the 39-school sample then the rankng of these two sets of samples should be the same. he feature s called rank on the ncrease or decrease of the stablty of the sample. We have seen from table 9 k and k except the th 3th 5th 8thnd 3rd school the rankng of 33 schools changed the largest dfference s 8 places the rank dfference of 4 schools s no smaller than 4. he SPSS result of Kappa consstency test on k and k s as follows n able 5. Snce p <0.00 so we can not rule out k and whch have a certan consstency. In the general vew when the Kappa Value 0.75 t shows a good consstency between the two samples; when 0.75> Kappa Value 0.4 t means the consstency s not very well; when Kappa Value <0.4 t shows the poor consstency between the two samples. [3] Because the Kappa Value = 0.3 <0.4 so the consstency s poor. hs result shows that determnng weght by the prncpal component analyss s not stable enough to ncrease or decrease the sample ths nstablty resulted n the rankngs wthout any reference to the actual value.. he Use Of Prncpal Component Analyss o Evaluate he Weght Whch Is Not Stable o he Changes Of Observatons As a rule n the evaluaton process for a postve ndcator as an evaluaton obect of observatons ncreases the evaluaton of the obect poston should move forward when the observatons decreases the evaluaton of the obect poston should be moved back evaluaton of other obects relatve rankngs should unchanged. hat s f an evaluaton obect ranked th were ranked nto the frst because the changng of ts observatons when > then the orgnal rank kth ( k <) school should now be (k +)th; when < then the orgnal rankng as kth ( <k ) school should now rank (k -)th the rest rankngs stll the same. hs feature s called the stablty of the evaluaton on observatons. On the bass of the above cases n 40- school sample change the nd school s PEER REVIEW from 95 to 0 CIAIONS / SASS pont from 63 to 30 then rank the 40 schools agan by the prncpal components method to determne weght see able 9 k 3. Here are two abromal problems: frst after the change n t ts poston ddn t decrease but rose up from 7th to st for an evaluaton method how can we beleve that the method of scence. Second even f the nd school from the 7th nto the frst one s recognzed accordng to the usual understandng the orgnal k to 6 should n turn become paragraphs to 7 other schools rank the same see able 9 k 6. Accordng to ths logc comparng k 3wth k 6 there are 33 schools have changed the order and 5 schools rank changed more than 4 places the 8th school changes the largest n 3 places. he SPSS result of Kappa consstency test on k3 and k 6 s as follows n able 6. able6 Symmetrc Measures Asymp. Std. Approx. Approx. Value Errora b Sg. Measure of Kappa Agreement N of Vald Cases 40 a. Not assumng the null hypothess. b. Usng the asymptotc standard error assumng the null hypothess. Snce p <0.00 so we can not rule out k and have a certan consstency but because Kappa Value = 0.54 <0.4 so the consstency s poor. hs shows that the use of prncpal component analyss to evaluate the weght s not stable to the changes of observatons; ths nstablty also resulted n the rankngs wthout any reference to the actual value..3 he Non-Unqueness Of he Egenvectors Whch Is Calculated From he Covarance Matrx And Causes he kng Results hat Do Not Have he Unque Soluton. For any egenvalue λ λ λp got from the covarance matrx W there are two opposte egenvectors ± e ± e ± e p where e = ( e e e p ) = p whch parwse orthogonal unt vectors. If there are k prncpal compnents we can construct k scenaros and get k dfferent evaluaton results. Under an extreme case we make e = e e = e e3 = e3 namely the dfference between the two prncpal component groups s a mnus and the result rankng s n able 9 k 4 whch s opposte to k. In front of such results whch one should be selected? Whch one s correct? k or k 4? 746

5 Journal of heoretcal and Appled Informaton echnology 8 th February 03. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: he Contrbuton Or he Characterstc Value Of Prncpal Component Is he Inherent Characterstcs Of he Data And Can Not Reflect he Sgnfcance Of he Indcator he reason to cause the analyss process unstable and unauthentc n part whch s the contrbuton rates (or egenvalue) of the PCA that reflect the nherent characterstcs of the sample data and descrbe the degree of dscrete on ther egenvector whle takng contrbuton rates as the weghts does not reflect the evaluatng standard of evaluators even not ndcate the mportance of ndcators. Suppose that X ( x x xp ) = s p- X X X dmenson varable n are n samples selected from statstcal populaton. A X w w = the p X X n w And suppose ( ) column vectors of matrx A are If V P p s the standardzed covarance matrx and the egenvalues of V P p are λ λ λ p so we can get λ 0 Γ VΓ = 0 λp Or equvalently Vγ = λγ γ = γ 0 = p = = p So γ s the correspondng egenvector of λ γ γ γ are mutually orthogonal and and p we have formula (): maxγ Vγ = λ = V γ γ γ γ = max γ Vγ = λ = V γ γ γ γ = γ γ = 0 max γ Vγ = λp = γ pvγ p γ γ = γ γ = 0 = p If a and b are constant vectors make p ( and we D X ) = V y = a X z = b can get D( y) = a Va Cov( y z) = a Vb X () Suppose G as the lnear aggregaton from w w wp y y= aw + aw + + apw p G = a a a a we set ( drecton of p = a a ) p w w wp as one of the and y s the varable or random sample of{ w }. Formulae () and () ndcate that all sample n G the sample varance on γ s maxmum and equal to λ ; on the orthogonal drecton of γ the sample varance on γ s maxmum and equal to λ ; and the rest can be deduced by analogy. he deducton above shows that the contrbuton rate or egenvalue of prncpal components are the nherent characterstcs of the data and t only reflects the dscrete degree of random sample on ts egenvactor but not denote the mportance of each ndcator. able7 Symmetrc Measures Value Asymp. Std. Errora Approx. b Approx. Sg. Measure of Kappa Agreement N of Vald Cases 40 a. Not assumng the null hypothess. b. Usng the asymptotc standard error assumng the null hypothess. In fact from ables and able 5 (the 40 student rankng part) we can observe that the scores of number 35 student are ; the ones of number 34 are and compare the two student number 34 has a better achevement than 35 but number 35 s the frst rankng. A detaled analyss of tables and 5 wll be able to know the results of the evaluaton t s very absurd t does not reflect the purpose of 747

6 Journal of heoretcal and Appled Informaton echnology 8 th February 03. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: evaluaton set by the evaluators whch has no scentfc support. able8 Symmetrc Measures Asymp. Std. Approx. Approx. Value Error a b Sg. Measure of Kappa Agreement N of Vald Cases 40 a. Not assumng the null hypothess. b. Usng the asymptotc standard error assumng the null hypothess. o sum up t s a msapplcaton to use the prncpal component analyss to determne the weght of revaluaton such method s lack of scentfc bass and can not ndcate the essentalty of the evaluaton ndcators. he examples n ths paper made KMO and Bartett s test. Accordng to the reference [] vewpont t should be authentc for factor analyss based on these sample data above the results however are totally worthless for those general samples the evaluaton result may be more nonsenscal. mes unversty rankngs were publshed by the mportance of ndcators followed by weght coeffcent the result can be obtaned by the weght n able 9 k 5. In able 9 we compare k wth k 5 k 4 wth k 5 the result can t be confdence by evaluatng the weght whch s determned by prncpal component. Contrast to k and k 5 except two schools the rankng of 38 schools changed the rank dfference of 30 schools s not smaller than 4 the 37th school has the largest dfference 34 places he SPSS result of Kappa consstency test on k and k 5 s shown as follows n able 7. Snce p = 0.33> 0.05 so k and k 5 do not have any consstency. Whle the Kappa Value = 0.06 <<0.4 t shows that the use of prncpal component analyss to determne the rankng of the weghts and the rankng of the mes have no consstency so k has no reference value. Contrast to k 4and k 5 all the rankng of 40 schools changed the rank dfference of 33 schools s no smaller than 4 the nd3rd school have the largest dfference 3 places he SPSS result of Kappa consstency test on k 4 and k 5 s shown as follows n able 8. able 9. kng able nd ex k k k3 k4 k5 k6 k remo ved Note: k the rankng based on the weghted determned by PCA. 748

7 Journal of heoretcal and Appled Informaton echnology 8 th February 03. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: k removed 8 observaton and use PCA to determne the weght and rank. k 3 change PEER REVIEW AND CIAIONS/SASS of the school wth ndex to 0 and 30 use PCA to determne the weght and rank. k 4 use e = e e = e e3 = e3 to make a new PCA rankng. k 5 the mes rankng k 7 the deal rankng based on k 3 s assumpton. k 7 based on k 3 s assumpton and the mes s weghts to rank. Snce p = 0.33> 0.05 so the sort order 4 and 5 do not have any consstency. Whle because the Kappa Value = <<0.4. hs shows that the use of prncpal component analyss to determne the weght the feature vector have taken a negatve poston vector n the mes rankng and have no consstency so there s no reference value of k 4. In summary we say that evaluaton on the weght whch s determned by the prncpal component analyss s an error there s no scentfc evdence and the method can not reflect the mportance of evaluaton as well. hs example also made KMO analyss and Bartett sphercty test accordng to the refereces [] that should be sad that the sample data n ths case call the shots for component analyss the result stll possess no value of reference. For the general problems t may brng a more absurd result of evaluaton. 3. CONCLUSION ACKNOWLEDGEMEN Hebe Socety Scence Funds:HBGL06 Strategy of Hebe Ports Servces Industry REFERENCES [] Wentong Zhang and We Dong SPSS Statstcal Analyss Advantage estbook[m] Beng Hgher Educaton Press pp.7-3. [] Junpng Ja and Xaoqun He. Statstcs[M] Beng Chna Renmn Unversty Press 006 pp.440. [3] Wentong Zhang and Je Yan SPSS Statstcal Analyss Basc estbook[m] Beng Hgher Educaton Press pp.38 As a result we draw the followng conclusons:. Weght maked by Prncpal Component Analyss s nstablty for evaluatng the change of entty thus the nstablty makes the really rank s nvald;. Eght maked by Prncpal Component Analyss s nstablty for evaluatng the change of observaton thus the nstablty makes the rank of weght s nvald 3. he postve and negatve of orthonormal vector make the expresson of Prncpal Component Analyss sn t unque that makes us cann t evaluate normally. Based on those we thnk Prncpal Component Analyss sn t a scentfc method and we wll study on the reasonable scope of the weght of Prncpal Component Analyss and study on the new way of makng weght. 749

8 Journal of heoretcal and Appled Informaton echnology 8 th February 03. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: