Multidimensional Scaling using Neurofuzzy System and Multivariate Analyses

Transcription

1 Multdmensonal Scalng usng eurofuzzy System and Multvarate Analyses Deok Hee am Engneerng and Computng Scence, Wlberforce Unversty, Wlberforce, OHIO, USA Abstract - Multdmensonal scalng s one of the mportant technques for a bg data management. In ths paper, varous statstcal analyses are compared to fnd the best-fttng method for a representaton of a hgher dmensonal data usng the reduced or smaller dmensonal data usng varous multvarate analyses wth maxmum lkelhood estmaton through the neurofuzzy systems, whch estmate the predcted output values. In addton, the estmated results are examned to fnd the best fttng technque through the comparson of the varous statstcal crtera. Keywords: data mnng, factor analyss, maxmum lkelhood estmaton, multdmensonal scalng, neurofuzzy system, prncpal component analyss Introducton In these days, many scentsts are nterested n reducng a very large data set effcently and equvalently wthout losng any sgnfcant meanng of the orgnal data set. Among the technques of the data reducton, the multdmensonal scalng (MDS) s frequently used wth applyng the statstcal methods such as prncpal component analyss, factor analyss, or clusterng analyss. The purpose of multdmensonal scalng (MDS) s a technque to acqure a vsual representaton of the pattern of smlartes n order to reduce the orgnal dmensonalty to the lower dmensonalty wth extractng essental features by dentfyng the characterstc embedded components among a set of objects or data. In order to perform the feature extracton from the hgher dmensonal space wthout losng any sgnfcant meanng of the orgnal data, statstcal procedures that uses varous transformng technques (lke orthogonal transformaton, varmax rotaton, or etc.) to convert a set of observatons of possbly correlated varables (or dmensons) nto a set of values of uncorrelated varables. In the paper, among dfferent types of statstcal methods, multvarate analyses ncludng prncpal component analyss, factor analyss, and maxmum lkelhood evaluaton are used to dscover the newly extracted structures from the exstng system wthout closely related measurement types. Moreover, n many cases, snce the examned data system cannot be expressed by a certan mathematcal expresson, neurofuzzy systems are deployed to compensate the weakness of the procedures. eurofuzzy systems can be used to compare the evaluatons between the orgnal data system and the newly reduced systems usng the varous statstcal technques to fnd out the mproved solutons. In order to show the evaluaton of the performance, the well-known data set, called Ar Fol Self ose data, s used. 2 Revew of lterature 2. Prncpal component analyss (PCA) In the varous statstcal analyss technques, PCA s the most popular technque along wth the factor analyss to dentfy or extract the most meanngful components from the unknown embedded components based upon the relatonshp of the gven varables n the orgnal mult-varables data systems. To perform prncpal component analyss (PCA) [5], the total varance of the reduced or transformed data must be consdered to dentfy the newly extracted components wthout losng any sgnfcant meanng of the orgnal data and closely correlated components between the extracted components from the orgnal varables. Iln and Rako [] and Jollffe [2] comprehensvely presented all procedures of PCA. The followng steps brefly ntroduce how to derve the steps of PCA procedures. Let X be an n dmensonal data set wth m n matrx format,.e. X {x, x 2,.x n }, where n s the number of measurement type to represent the dmenson of the data and m s the number of samples. Frst, standardze X by normalzng x, x 2,.x n, wth subtractng the mean from each measurement type. After the standardzaton of X, apply Sngular Value Decomposton (SVD) technque to calculate the newly extracted components wth the egenvectors of the covarance. Fnally, determne the dmensonalty of X wth most meanngful components by accumulatng the calculated covarance based upon the requred crteron. 2.2 Factor analyss (FA) In factor analyss [4][2], the factors are underlyng latent varables that represent the orgnal varables. If the orgnal varables y, y 2,..., y p are at least moderately correlated, the basc dmensonalty of the system s less than the orgnal dmensonalty, p. The purpose of usng factor analyss s to reduce the redundancy among the orgnal varables by usng a smaller number of newly extracted factors. Consder the basc structure from the vector, A, of raw data. Then, calculate the correlatons (or covarance) of the response vector A, denoted by Σ by applyng the basc

2 common factor analytc model. After the orgnal varables are standardzed, the basc nput to a common factor analyss s the correlaton matrx. Wth the correlaton matrx, fnd the egenvalue, λ, from the determnant equaton. Usng the egenvalues, the dagonal elements of the matrx Σ are the square root of λ f j, and 0 f j. Form an ntal factor loadngs usng Σ and Λ by multplyng each other. Then, the Varmax rotaton s appled to evaluate the rotated factor loadngs. Fnally, applyng the rotated factor loadngs, factor scores can be calculated as the projecton of an observaton on the common factors. 2.3 Maxmum-lkelhood estmaton The maxmum-lkelhood estmaton (MLE) [3][5][6] s a method of estmatng the parameters of a statstcal model whch developed by R.A. Fsher n the 920s. The probablty densty functon (pdf), f (y θ), for a random varable, y, and condtoned on a set of parameters, θ, dentfes the observed data and provdes a mathematcal descrpton of the data whch s the product of the ndvdual denstes. Ths jont densty s called lkelhood functon, whch defned as a functon of the unknown parameter vector, θ, where y s used to ndcate the collecton of sample data. Consder the random sample wth a certan condtons of the observatons. If the jont densty gves that the value of θ can make the sample data, y, most probable, ths estmaton s called as maxmum lkelhood estmate, or MLE, of the unknown parameter, θ. Let y, y 2, y be random samples from pdf, f(y θ), where Y { y, y 2, y } s the set of samples. Assumng statstcal ndependence between the dfferent samples, the pdf s f(y θ) f(y, y 2, y θ) where t s known as the lkelhood functon of θ wth respect to Y. The maxmum lkelhood method estmates θ wth the maxmum value of the lkelhood functon such as θˆ ˆ θ arg max ML θ y k f ( θ ) () y k f ( θ ) (2) wth satsfyng a necessary condton to be a maxmum of ML wth respect to θ to be zero,.e. f ( y k θ ) θ 0 Fnally, the loglkelhood functon can be defned as and L( ) ln f ( θ ) y k (3) θ (4) L( θ ) θ ln f ( yk θ ) θ f ( yk θ ) 0 f ( y θ ) θ 2.4 Varmax rotaton k The varmax rotaton was developed by Kaser (958) [8] and used as the most popular rotaton method for factor analyss. The varmax rotaton s a technque to rotate the orthogonal bass to algn wth the related coordnates n order to smplfy the nterpretaton of the partcular sub-space wthout changng the actual coordnate system. After applyng a varmax rotaton, each orgnal varable s closely assocated wth one (or a small number) of extracted factors, and each factor represents only a small number of varables. In general, the varmax rotaton searches for a rotaton (.e., a lnear combnaton) of the orgnal factors such that the varance of the loadngs s maxmzed. 2.5 eurofuzzy System A neurofuzzy system [] s a hybrd system whch s a fuzzy system appled by neural network technque that uses a learnng algorthm derved from examned and traned data to determne the developed system s characterstcs. Jang [7] ntroduced Adaptve euro-fuzzy Inference System (AFIS), whch represents a structure of a neurofuzzy system based upon Takag Sugeno fuzzy nference system usng fve dfferent layers such as nput layer, producton layer (fuzzfcaton), normalzed frng layer (nference), consequence parameters layer (defuzzfcaton), and fnalzed output layer. Fg. shows the structure of AFIS system wth fve network layers. x y A A 2 B B 2 TT TT w w 2 Layer Layer 2 Layer 3 Layer 4 Layer 5 AFIS structure Fg. Adaptve euro-fuzzy Inference System (AFIS) [7] As shown n Fg., there are fve layers of AFIS. Layer conssts of the fuzzy set generalzed the membershp functons and assocated wth the adaptve node wth a output node. Layer 2 multples the ncomng sgnals and outputs the products whch represent the frng strength of the rule. In w w 2 x y x y w f w 2 f 2 Σ f (5)

3 general, ths stage s called as a fuzzfcaton of the system. Layer 3 calculates the raton of the th rule s frng strength to the sum of all rules frng strengths called normalzed frng strengths. Layer 4 s an adaptve node wth a node functon as consequent parameters. Layer 5 computes the overall output as the summaton of all ncomng sgnals. Ths s called as defuzzfcaton of the system. The output from Layer 5 from Fg. by Jang [7] can be expressed as O, w f w f 5 (6) wth applyng the followng rulebase [2], such as Rule : IF x s A AD y s B THE f p x + q y +r Rule 2: IF x s A 2 AD y s B 2 THE f 2 p 2 x + q 2 y +r 2. w 3 Data of ar fol self nose The ar fol self nose data set s obtaned from a seres of aerodynamc and acoustc tests of two and threedmensonal arfol blade sectons conducted n an anechoc wnd tunnel. The ASA data set comprses dfferent sze ACA 002 arfols at varous wnd tunnel speeds and angles of attack. The span of the arfol and the observer poston were the same n all of the experments. For the attrbutes, the nputs are the frequency n Hertz, the angle of attack n degrees, the chord length n meters, the free-stream velocty n meters per second, and the sucton sde dsplacement thckness n meters. The only output s the scaled sound pressure level, n decbels. 4 Appled neurofuzzy systems There are fve dfferent neurofuzzy systems usng Adaptve-etwork-Based Fuzzy Inference Systems (AFIS) [7] to develop the procedures of estmatng the sucton sde dsplacement thckness wth reduced components usng prncpal component analyss (PCA) and factor analyss wth varmax rotaton or maxmum lkelhood estmaton from the fve orgnal measurements types and four reduced measurements types. The followng fgures are about the neurofuzzy system wth the fve orgnal measurements types. Fg. 2 shows the propertes of neurofuzzy system wth fve nputs and one output. As shown n Fg. 3, Gaussan Bell shape functons are used for the membershp functons for each nput and output for the neurofuzzy system. Fg. 4 shows the appled rules for the neurofuzzy system and Fg. 5 descrbes how the structure of the neurofuzzy system s developed based upon AFIS. There are fve layers to extract the fnalzed output through fuzzfcaton and defuzzfcaton procedures as shown n layer 2 to layer 4 from Fg. 5. In Fg. 6, the surface plot s presented by vsualzng the ar fol self nose data set that are too large to dsplay n numercal form and for graphng functons for the mutdmensonaltes of the ar fol self nose data set. Fg. 2 eurofuzzy nference system wth propertes ncludng three nputs and an output. Fg. 3 eurofuzzy nference system wth membershp functons

4 Fg. 4 eurofuzzy nference system wth appled rules for defuzzfcaton Fg. 5 AFIS Model Sturcutre of developed eurofuzzy nference system wth three nputs Fg. 6 Surface vewer to dsplay n the numercal form of the ar fol self nose data set 5 Analyses and results To recognze the ar fol self nose data, the scaled sound pressure level s appled as an output of each varable from the ar fol self nose data. As shown n Fg. 7, all egenvalues are presented based upon the newly transformed components from the fve orgnal measurements types. In order to decde the best-fttng reduced number of components from the fve orgnal measurements types, the accumulaton of the varances from the newly extracted components usng The Egenvalues-Greater-Than-One Rule [9], and 0.9 or above crteron for the accumulaton of the varances. There are fve dfferent technques are compared wth varous neurofuzzy systems usng the ar fol self nose data; ORG, FM, FVM, FVP, and PCA. ORG s the neurofuzzy system usng the orgnal fve components data. FM s the neurofuzzy system wth factor analyss usng maxmum lkelhood estmaton. FVM s the neurofuzzy system wth factor analyss usng varmax rotaton and maxmum lkelhood estmaton. FVP s the neurofuzzy system wth factor analyss usng varmax rotaton and prncpal components. PCA s the neurofuzzy system wth prncpal component analyss. The reduced components are also examned wth the fve orgnal measurements types usng the statstcal categores such as correlaton (COR), root means square (RMS), standard devatons (STD), mean of absolute dstance (MAD), statstcal ndex (EWI) and error rate (ERR).

5 2.0 Selecton of umber of Components Fg. 8 stand for the statstcally evaluaton wth the fve orgnal components and four newly extracted components usng neurofuzzy systems, respectvely. Egenvalue Component umber Fg. 7 The relatonshp between Components and Egenvalues TABLE Statstcal analyss between extracted components wth estmated values and the orgnal values usng neurofuzzy systems FSs COR RMS STD MAD EWI ERR ORG FM FVM FVP PCA From TABLE, only four newly extracted components are appled to estmate the scaled sound pressure level usng four nputs neurofuzzy system. In the category of the correlaton, FVP shows the best performance for the evaluaton. For the root mean square, FM evaluates the best matches. In the standard devaton, PCA s evaluaton draws the best performance. In the category of mean of absolute dstance, FM evaluates the closest pressure levels. In overall, usng the category of equally weght ndex, FVM shows the best results among the other technques. 4 5 The followng categores evaluate the performance of the neuro fuzzy systems usng reduced data models: Correlaton (CORR): Correlaton between the orgnal output and the estmated output from the neurofuzzy system usng the data from each method. Root Mean Square (RMS): Total Root Mean Square for the dstance between the orgnal output and the estmated output usng the same testng data through the neurofuzzy system. RMS n ( x n y ) where x s the estmated value and y s the orgnal output value. Standard Devaton (STD): Standard Devaton for the dstances between the orgnal output and the estmated output usng the same testng data through the neurofuzzy system. Mean of the Absolute Dstances (MAD): Mean of the absolute dstances between the orgnal output and the estmated output usng the same testng data through the neurofuzzy system Equally Weghted Index (EWI) [0]: The ndex value from the summaton of the values wth multplyng the statstcal estmaton value by ts equally weghted potental value for each feld. The value, whch s close to 0, s the better results. Error Rate (ERR): the error rate between the estmated pressure level and the orgnal pressure level through the neurofuzzy systems generated by the examned technques. 6 Concluson The presented paper descrbes how the orgnal data can be effcently dentfed by the less dmensonal data wthout losng any sgnfcant meanng wth evaluatng the system outputs wth the neurofuzzy systems. The arfol self nose data are employed and mplemented by the orgnal data and the reduced dmensonal data and evaluated by usng fve statstcal measurements n order to compare each performance. In overall, the dmensonal scalng wth applyng the factor analyss wth maxmum lkelhood estmaton shows the best performance n the equally weghted ndex and the error rate through the evaluaton usng the neurofuzzy system. 2 () Acknowledgment Fg. 8 Comparson of Statstcal Analyss usng Extracted components through neurofuzzy systems Fg. 8 plots the statstcal evaluaton usng fve dfferent cases wth the comparson based upon the suggested statstcal measurements. COR, RMS, STD, MAD, EWI, and ERR n The arfol self nose data set orgnally adapted from the collecton of the database system of UCI Machne Learnng Repostory [ Irvne, CA: Unversty of Calforna, School of Informaton and Computer Scence.

6 7 References [] A. Iln and T. Rako, Practcal Approaches to Prncpal Component Analyss n the Presence of Mssng Values, Journal of Machne Learnng Research, Vol., 200, pp [2] I.T. Jollffe, Prncpal Component Analyss, Sprnger, Second Edton, ew York, Y, [3] S. Theodords and K. Koutroumbas, Pattern Recognton, Elsever, Academc Press, San Dego, CA, 2003 [4] R. J. Rummel, Understandng Factor Analyss, The Journal of Conflct Resoluton, Vol., o. 4, Dec. 967, pp [5] In Jae Myung, Tutoral on maxmum lkelhood estmaton, Journal of Mathematcal Psychology, Vol. 47, 2003, pp [6] M. E. Tppng and C. M. Bshop, Probablstc Prncpal Component Analyss, Journal of the Royal Statstcal Socety, Seres B, Vol. 6, Part 3, 999, pp [7] J.-S.R. Jang, AFIS: Adaptve-etwork-Based Fuzzy Inference Systems, IEEE Trans. Systems, Man & Cybernetcs, Vol. 23, 993, pp [8] H. F. Kaser, The varmax crteron for analytc rotaton n factor analyss, Psychometrka Vol. 23, 958, pp [9] orman Clff, The Egenvalues-Greater-Than- One Rule and the Relablty of Components, Psychologcal Bulletn, Vol. 03, o. 2, pp , 988. [0] D. am, and H. Sngh, Materal processng for ADI data usng multvarate analyss wth neuro fuzzy systems, Proceedngs of the ISCA 9 th Internatonal Conference on Computer Applcatons n Industry and Engneerng, Las Vegas, evada, ov. 2006, pp [] Ronald Yager and Dmtar Flev, Essentals of fuzzy modelng and control, ew York, John Wley and Sons., 994. [2] Herve Abd, Lynne J. Wllams, and Domnnque Valentn, Multple factor analyss: prncpal component analyss for multtable and multblock data sets, WIREs Computatonal Statstcs, Wley Perodcals, Inc., 203.