Dirichlet Compound Multinomials Statistical Models

Transcription

1 Applied Mahemaics, 2012, 3, hp://dx.doi.org/ /am a288 Published Online December 2012 (hp://.scirp.org/journal/am) Dirichle Compound Mulinomials Saisical Models Paola Cerchiello, Paolo Giudici Deparmen of Economics and Managemen, Universiy of Pavia, Pavia, Ialy Received Augus 31, 2012; revised Sepember 31, 2012; acceped Ocober 6, 2012 ABSTRACT This conribuion deals ih a generaive approach for he analysis of exual daa. Insead of creaing heurisic rules for he represenaion of documens and ord couns, e employ a disribuion able o model ords along exs considering differen opics. In his regard, folloing Mina proposal (2003), e implemen a Dirichle Compound Mulinomial (DCM) disribuion, hen e propose an exension called sbdcm ha aes explicily ino accoun he differen laen opics ha compound he documen. e follo o alernaive approaches: on one hand he opics can be unnon, hus o be esimaed on he basis of he daa, on he oher hand opics are deermined in advance on he basis of a predefined onological schema. The o possible approaches are assessed on he basis of real daa. Keyords: Texual Daa Analysis; Mixure Models; Onology Schema; Repuaional Ris 1. Inroducion ih he rapid groh of on-line informaion, ex caegorizaion has become one of he ey echniques for handling and organizing daa in exual forma. Tex caegorizaion echniques are an essenial par of ex mining and are used o classify ne documens and o find ineresing informaion conained ihin several on-line eb sies. Since building ex classifiers by hand is difficul, ime-consuming and ofen no efficien, i is orhy o learn classifiers from experimenal daa. In his proposal e employ a generaive approach for he analysis of exual daa. In he las o decades many ineresing and poerful conribuions have been proposed. In paricular, hen coping ih he ex classificaion as, a researcher has o face he ell-non problem of polysems (muliple senses for a given ords) and synonyms (same meaning for differen ords). One of he firs effecive model able o solve hose issues is represened by Laen semanic analysis (LSA) [1]. The basic idea is o or a a semanical level by reducing he vecor space hrough Singular Value Decomposiion (SVD), producing no sparse occurrence ables ha help in discovering associaions beeen documens. In order o esablish a solid heoreical saisical frameor in his conex, in [2] a probabilisic version of LSA (plsa) has been proposed, also non as he aspec model, rooed in he family of laen class models and based on a mixure of condiionally independen mulinomial disribuions for he couple ords-documens. The inenion from he inroducion of plsa as o offer a formal saisical frameor, helping he parameer inerpreaion issue as ell. By he ay he goal as achieved only parially, in fac he mulinomial mixures, hich componens can be inerpreed as opics, offer a probabilisic jusificaion a ords bu no a documens level. In fac he laer are represened merely as lis of mixing proporions derived from mixure componens. Moreover, he mulinomial disribuion presens as many values as here are in he raining documens and herefore i learns opic mixure on hose rained documens. The exension o previously unseen documens is no appropriae since here can be ne opics. In order o overcome he asymmery beeen ords and documens and o produce a real generaive model, [3] proposed he LDA (Laen Dirichle Allocaion). The idea of such ne approach emerges from he concep of exchangeabiliy for he ords in a documen ha unfolds in he bag of ords assumpion: he order of ords in a ex is no imporan. In fac he LDA model is able o capure eiher he ords or documens exchangeabiliy unlie LSA and plsa. On he oher hand LDA is a generaive model in any sense since i posis a Dirichle disribuion over documens in he corpus, hile each opic is dran from a Mulinomial disribuion over ords. Hoever noe ha [4] in 2003 have shon ha LDA and plsa are equivalen if he laer is under a uniform Dirichle prior disribuion. Obviously LDA does no solve all he issues. The main resricion embedded in LDA approach and due o he Dirichle disribuion, is he assumpion of independence among opics. The immediae consequence as o acle he issue by inroducing he Correlaed Topic Model (CTM), as proposed in [5]. CTM inroduces correlaions among

2 2090 P. CERCHIELLO, P. GIUDICI opics by replacing he Dirichle random variable ih he logisic normal disribuion. Unlie LDA, CTM presens a clear complicaion in erms of inference and parameer esimaion since he logisic normal disribuion and he Mulinomial are no conjugae. To bypass he problem, he mos recen alernaive is represened by he Independen Facor Topic Models (IFTM) inroduced in [6]. Such proposal maes use of laen variable model approach o deec hidden correlaions among opics. The choice o explore he laen model orld allos o choose among several alernaives ranging from he ype of relaion, linear or no linear, o he ype of prior o be specified for he laen source. For sae of compleeness is imporan o menion anoher ineresing research pah focusing on he bursiness phenomenon, ha is he endency of rare ords, mosly, o appear in burs. The above menioned generaive models are no able o capure such peculiariy, ha insead is very ell modelled by he Dirichle Compund Mulinomial model (DCM). Such disribuion as inroduced by saisicians [7] and has been idely employed by oher secors lie bioinformaics [8] and language engineering [9]. An imporan conribuion in he conex of ex classificaion as brough by [10] and [11] ha profiably used DCM as a bag-of-bags-of-ords generaive process. Similarly o LDA, e have a Dirichle random variable ha generaes a Mulinomial random variable for each documen from hich ords are dran. By he ay, DCM canno be considered a opic model in a ay, since each documen derives specifically by one opic. Tha is he main reason hy [12] proposed a naural exension of he classical opic model LDA by plugging ino i he DCM disribuion and obaining he so called DCMLDA. Folloing his line of hining, e move from DCM approach and e propose an exension of he DCM, called semanicbased Dirichle Compound Mulinomial (sbdcm), ha permis o ae laen opics ino accoun. The paper is organized as follos: in Secion 2 e sho he Dirichle Compound Mulinomial (DCM) model, in Secion e propose an exension of he DCM, called semanicbased Dirichle Compound Mulinomial (sbdcm), in Secion 4 e sho ho o esimae he parameers of he differen models. Then, in Secion 5 e assess he predicive performance of he o disribuions by using seven differen classifiers. Finally e sho he differen classificaion performance according o he noledge on he opics T (non or unnon). 2. Bacground: The Dirichle Compound Mulinomial The DCM disribuion is a hierarchical model: on one hand, he Dirichle random variable is devoed o model he Mulinomial ord parameers ; on he oher hand, he Mulinomial variable models he ord coun vecors x comprising he documen. The disribuion funcion of he DCM mixure model is: d p x p x p. (1) here p x is he Mulinomial disribuion: p x n! \ x 1 x. in hich x is he ords coun vecor, x is he coun for each ord and he probabiliy of emiing a ord i ; herefore a documen is modelled as a single se of ords ( bag-of-ords ). The Dirichle disribuion p is insead parameerized as: ih p \ he Dirichle parameer vecor for ords, as consequence he hole se of ords ( bag-of-bags ) is modelled. Thus a ex (a documen in a se) is modelled as bag-of-bags-of-ords. Developing he previous inegral e obain: p n! x x x \ In Figure 1 e repor he graphical represenaion of he DCM model. From anoher poin of vie, each Mulinomial is lined o specific sub-opics and maes, for a specific documen, he emission of some ords more liely han ohers. Insead he Dirichle represens a general opic ha compounds he se of documens and hus he DCM could be also described as bag-of-scaleddocumens. The added value of he DCM approach consiss in he abiliy o handle he bursiness of a rare ord ihou inroducing heurisics [13]. In fac, if a rare ord appears once along a ex, i is much more liely o appear again. hen e consider he enire se of documens D, here each documen is independen and idenified by is coun vecor, D x,,, 1 x xn, he lielihood of he Figure 1. Hierarchical represenaion of DCM model.. (2) (3) (4)

3 P. CERCHIELLO, P. GIUDICI 2091 hole documens se (D) is N pxd p D N d 1 1 x d1 1 xd 1 here x d is he sum of he couns of each ord in he documen d-h (x d ) and x d he coun of ord -h for he documen d-h. Thus he log-lielihood is: log d11 N log log x d1 1 1 N p D x log log d (5) (6) The parameers can be esimaed by a fixed-poin ieraion scheme, as described in Secion A Semanic-Based DCM As explained in Secion 2, e have a coefficien for each ord compounding he vocabulary of he se of documens hich is called corpus. The DCM model can be seen as a bag-of-scaled-documens here he Dirichle aes ino accoun a general opic and he Mulinomial some specific sub-opics. Our aim in his conribuion is o build a frameor ha allos us o inser specifically he opics (non or unnon) ha compound he documen, ihou losing he bursiness phenomenon and he classificaion performance. Thus e inroduce a mehod o lin he coefficiens o he hypoheic opics, indicaed ih i, by means of a funcion F hich mus be posiive in since he Dirichle coefficiens are posiive. Noe ha usually dimb dima and, herefore, our proposed approach is parsimonious. Subsiuing he ne funcion ino he inegral in Equaion (1), he ne model is: p x p x p F d, e have considered as funcion F() a linear combinaion based on a marix D and he vecor. D conains informaion abou he ay of spliing among opics he observed coun vecors of he ords conained in a diagonal marix A and is a vecor of coefficien (eighs) for he opics. More specifically e assume ha: (7) 1 d11 d1 T A, D, d1 d T 1, D AD T 1 F D T Noe ha: T d d d ; ih T he number of Topics; (8) (9) d is he coefficien for ord -h used o define he degree of belonging o opic -h and by hich a porion of he coun of ord -h is assigned o ha paricular opic -h. By subsiuing his linear combinaion ino Equaion (4), e obain he same disribuion bu ih he above menioned linear combinaion for each : p x T T d x n! d 1 T T 1 x x d d 1 1 (10) This model is a modified version of he DCM, henceforh semanic-based DCM (sbdcm), and he ne loglielihood for he se of documens becomes: log p D N T T log d log x d d1 1 1 N T T * log xd d log d d11 (11) In Figure 2 e repor he graphical represenaion of he ne model here he s are subsiued by a funcion of he s. An imporan aspec of he proposed approach is represened by he number T of opics o be insered ino he sbdcm ha can be: Unnon, hus o be esimaed on he basis of he available daa.

4 2092 P. CERCHIELLO, P. GIUDICI Figure 2. Hierarchical represenaion of sbdcm model. A priori non (i.e. fixed by field expers). The firs case ill be reaed in Secion 5.1, in paricular since i is no alays possible o no in advance he number of laen opics presen in a corpora, i becomes very useful o build a saisical mehodology for discovering efficienly and consisenly a suiable number of opics. In his case he number of opics T can be considered a random variable. Thus, e use a segmenaion procedure o group he ords in order o creae groups of ords sharing common characerisics ha can be considered as laen opics. The analysis is compleed by choosing he bes number of groups and a disance marix is used o se he membership percenage (d ) of each ord o each laen opic. The second case ill be reaed in Secion 5.2 and proposes o exploi he sbdcm model by employing a priori noledge based on onological schemas ha describe he relaions among conceps ih regards o he general opics of he corpora. The onology srucure provides he se of relaions among he conceps o hich can be associaed a cerain number of ey ords, by a field exper(s). Thus, e an o use he classes of a given onology and he associaed ey ords o define in advance he number T of opics. 4. Parameers Esimaion There are several mehods o maximize he log-lielihood and o find he parameers associaed ih a DCM: simplified Neon ieraion, he Expeced Maximizaion mehod and he maximizaion of he simplified lielihood (called leave-one-ou lielihood, LOO). Among hem, he mos general and flexible algorihm is he Expeced Maximizaion (EM). The EM algorihm is an ieraive procedure able o compue he maximum-lielihood esimaes henever daa are incomplee or hey are considered complee bu no observable (laen variables) [14]. In general, considering, p X Z he join probabiliy disribuion (or probabiliy mass funcion) here X is he incomplee (bu observable) daa se and Z he missing (or laen) daa, e can calculae he complee maximum lielihood esimaion of model parameers by means of he EM algorihm. e obain an overall complexiy reducion for ha concerns he calculaion of observed maximum-lielihood (ML) esimaes. The goal is achieved by alernaing he expecaion (E-sep) of he lielihood ih he laen variables as if hey are observed and he maximizaion (M-sep) of he lielihood funcion found in he E-sep. ih he resul of he M-sep e updae he ne parameers ne ha are used again in he cycle, by saring from he E-sep, unil e reach a fixed degree of approximaion of he observed lielihood hich increases sep by sep moving oards he maximum (ha could be local) [15]. The EM can be buil in differen ays. One possibiliy is o see he EM as a loer bound maximizaion here e alernae he E-sep o calculae an approximaion of he loer bound for he log-lielihood and maximize i in he M-sep unil a saionary poin (zero gradien) is reached. If e are able o find a loer bound for he log-lielihood e can maximize i via a fixed-poin ieraion; in fac i is he same principle of considering he EM as a loer bound maximizaion. In our conex, for he DCM, he loer log p D is he folloing quaniy: bound ih xd 1 N xd d (12) his allos us o use a fixed poin ieraion hose seps are: xd 1 N xd d (13) here x d is he sum of he couns of each ord in he documen d-h x d, x d he coun of ord -h for he documen d-h and he Dirichle coefficien for ord a he -h sep. The algorihm is sopped hen a degree of approximaion is reached. The ieraion sars ih equals o he occurrence percenage of he ord -h in he corpus. The esimaed parameers, as said before, have an imporan characerisic: hey follo he bursiness phenomenon of ords. In fac he smaller is, he more bursiness effec is conained ihin a ord, as revealed in Secion 5. In he case of sbdcm by considering he ne loglielihood, Equaion (11), e can use he same loer bound employed before. The only modificaion lays on he subsiuion of he coefficiens ih he linear funcion of, as in Equaion (9), and hereby he ne fixed poin ieraion sep is:

5 P. CERCHIELLO, P. GIUDICI N T T d xd d d d d N T T d xd d d d d (14) As before e sop he ieraion a a fixed degree of approximaion and he coefficiens d are hose described in Secion 3. The ne s mainain he ords bursiness, as e shall sho in Secion 5 and hey are used o classify he documen by employing a Naive Bayes classifier. For our applicaions, in he nex secion e have used for boh models a value of of The ieraion sars ih equals o he percenage of each single cluser obained by he grouping analysis of vocabulary ords. 5. Model Performance In his secion e describe he evaluaion of he differen classifiers by using he parameers esimaed from he DCM disribuion. Thus, our raining daa se is compound of 6436 documens ih a vocabulary (already filered and semmed) of 15,655 ords so e have o esimae 15,655 parameers. The parameer is able o model he bursiness of a ord. In fac, he smaller he parameers are, he more bursy he emission of ords is. This phenomenon is characerisic of rare ords, herefore coefficiens are, on average, smaller for less couned ords. The average value of he overall parameers is , he sandard deviaion is and he maximum and minimum values are respecively and Once he coefficien vecor of s is obained e employ seven differen classifiers, hree of hich are described in [13] (normal (N), complemen (C) and mixed (M). The remaining ones are proposed as he appropriae combinaion of he previous ones, in order o improve heir characerisics. Those ne classifiers are se in funcion of he number of ords ha a es-documen has in common ih he se of documens ha compound a class; in his ay e creae a classifier in funcion of he number of ords in common. Thus e analyze he folloing addiional classifiers: COMPLE- MENT + MIXED + NORMAL (CMN), COMPLE- MENT + NORMAL (CN), COMPLEMENT + MIXED (CM), MIXED + NORMAL (MN). In order o evaluae he classificaion performance e employ hree performance indexes: Ind1: The proporion of rue posiive over he oal number of es-documens: D d 1 TPd 100; D Ind2: The proporion of classes ha e are able o classify: C c1 Ic 100; C here I c is an indicaor ha e se 1 if a leas one documen of he class is classified correcly, oherise e se 0. Ind3: The proporion of rue posiive ihin each class over he number of es documens presen in he class: 1 C C c1 TPc 100; Mc here M c is he number of es-documens in each class, TP c is he number of rue posiive in he class and C he number of opics (46). For he four combined classifiers such indexes have been calculaed by varying he number of ords in common beeen he es documen and he class. In paricular for our es e have used hree differen hresholds for he number of ords (n): 15, 10 and 5. For example, e indicae ih he iniials CM n he classificaion rule ha employs classifier C hen he number of common ords are less or equal o n and classifier M hen he number of ords in common is more han n. Insead, he iniial CMN n.m idenifies he using of classifier C unil n, he classifier N over m and he classifier M beeen n and m. For he daa a hand he number of ords in common beeen he o ses (raining and evaluaion se) varies beeen 1 and 268. The above menioned combinaion is based on he folloing idea: if he number of ords in common beeen he bag of ords and he correc class is lo, hen he mos informaion conen is in he complemen se. Oherise he needed informaion is conained eiher in he normal se or in he complemen one. Taing ino accoun such consideraion e have se up differen combinaion and e concluded ha he useful rade-off among classifiers is equal o 10 (Table 1). As e can see he bes classifiers are he mixed and he CM 10 ones. They are able o classify respecively 1237 and 1238 over 1609 documens ha are disribued no uniformly among classes (46). These classifiers are able o classify a leas a documen per class even if here are classes conaining only 2 documens. Beeen hem he CM 10 classifier has index hree slighly beer han mixed one. The orse classifier, in his case, is he complemen version alone.

6 2094 P. CERCHIELLO, P. GIUDICI Table 1. Classificaion resuls by varying cluser numbers and using marix C. Classifier Measure sbdcm 5 sbdcm 11 sbdcm 17 sbdcm 23 sbdcm 46 DCM LL i 282, , , , , ,385 LL o 205, , , , , ,286 AIC i 56, , , , , ,264 AIC o 410, , , , , ,066 Norm. Ind \\ Ind \\ Ind Comp Ind \\ Ind \\ Ind Mixed Ind \\ Ind \\ Ind e no verify he goodness of he sbdcm models described in Secion 3, o undersand if e can inser laen opics ino DCM by mainaining he bursiness and he same classificaion performance. To inds of marixes have been used in he cluser procedure. One marix conains he correlaions among ords C in he vocabulary and anoher G is consruced by calculaing he Krusal-allis index on he coun marix among ords. The laer index g is defined as follos: g K 12 N 1 niri N N i1 p 3 ci i1 3 c i 2 (15) N N here n i is he number of sample daa, N he oal observaion number of he samples, he number of samples o be compared and r i he mean ran of i-h group. The denominaor of he index g is a correcion facor needed hen ied daa are presen in he daa se, here p is he number of recurring rans and c i is he imes he i-h ran is repeaed. The index g depends on he differences among he averages of he groups r i and he general average. If he samples come from he same populaion or from populaions ih he same cenral endency, he arihmeic averages of he rans of each group ri rij n i should be similar o each oher and o he j general average N 1 2 as ell. The raining daase conains 2051 documens ih a vocabulary of 4096 ords for boh approaches. The evaluaion daase (again he same for boh models) conains 1686 documens hich are disribued over 46 classes. In Tables 1 and 2 e repor he resuls from he o ess obained respecively by marixes C and G and by varying he number of groups in he cluser. In he ables e indicae ih LL in he log-lielihood before he parameers updaing and ih LL ou afer he ieraion procedure hich is sopped hen he error = is reached. The same ih AIC cin and AIC cou ha is he correced Aaie Informaion Crierion (AIC c ) before and afer he uploading. The indexes Ind1, Ind2 and Ind3 have been described in Secion 5.1. As e can see in he o Tables 1 and 2, he percenages of correc classificaion (Ind1) are very close o he original ones ih a parameer for each ord (4096 parameers). Of course hey depend on he ype of classifier employed during he classificaion sep. Considering boh sbdcm and DCM, he differences produced by varying he number of groups are small. Moreover he AIC c is alays beer in he ne approach hen considering each ord as a parameer (DCM model). In paricular for ha concerns he approach based on he correlaion marix C (in Table 2) ih 17 groups and on he Mixed classifier, i can predic correcly he 68.43% of documens. The log-lielihood and he AIC c indexes along groups are quie similar, hoever he bes value is obained ih 5 groups (respecively 205,412 and 410,834). Considering again he approach based on he correlaion marix C, e can conclude ha, in erms of complexiy expressed by he AIC index, he sbdcm approach haever applied classifier is alays beer han he DCM. hen e use marix G (Table 2) he bes classificaion performance is for he complemen classifier based on 23 groups, ih a percenage of 68.72%, a log-lielihood of 204,604, he AIC c of 409,254. The bes log-lielihood and AIC c are for cluser ih 46 groups (respecively 204,362 and 408,816). Even if he sbdcm disribuion based on marix G is no able o improve he classificaion performance of DCM, e can say ha he sbdcm Index1 is alays very close o he bes one. In addiion he ne model is

7 P. CERCHIELLO, P. GIUDICI 2095 Table 2. Classificaion resuls by varying cluser numbers and using marix G. Classifier Measure sbdcm 5 sbdcm 11 sbdcm 17 sbdcm 23 sbdcm 46 DCM LL i 291, , , , , ,385 LL o 205, , , , , ,286 AIC i 582, , , , , ,264 AIC o 411, , , , , ,066 Norm. Ind \\ Ind \\ Ind Comp Ind \\ Ind \\ Ind Mixed Ind \\ Ind \\ Ind alays beer in erms of eiher AIC and log-lielihood indexes. Moreover, if e perform an asympoic chi- 2 squared es es considering he o cases (marixes G and C) o decide heher he difference among loglielihoods (LL), ih respec o DCM, are significan (i.e. he difference is saisically meaningful if he LL1 LL2 is greaer han 6), e can see from Tables 1 and 2 he es ih marix G has he bes performance. Performance of he Semanic-Based Dirichle Compound Mulinomial ih T Knon in Advance A differen approach needs o be assessed hen he number of available opic T is non in advance. In fac a ex corpora could be enriched by several descripions of reaed opics according o he noledge of field expers. In more deails, he analysis could be provided ih a priori noledge based on onological schemas ha describe he relaions among conceps ih regards o he general opics of he corpora. An onology (from hich an onological schema is derived) is a formal represenaion of a se of conceps ihin a domain and he relaionships beeen hose conceps [16]. I provides a shared vocabulary, hich can be used o model a domain, ha is, he ype of objecs and/or conceps ha exis, and heir properies and relaions. In Figure 3 e repor an example of graphical represenaion of an onological schema. For example, if a ex se deals ih repuaional ris managemen for corporae insiuions, an onology can be creaed on he basis of he four caegories of problems (inernal processes, people, sysems and exernal evens) defined by Basel II Accords. Hence e can suppose ha some specific sub-opics and ey ords, such as he possible causes of repueional losses, ill be almos surely reaed along he exs. Figure 3. Example of onological schema. Thereby, he onology srucure provides he se of relaions among he conceps o hich can be associaed, by a field exper(s), a cerain number of ey ords. Thus, e an o use he classes of a given onology and he associaed ey ords o define in advance he number T of opics. The onological schema hich e refer o, for he daa a hand, deals ih he so called repuaional ris [17]. I is no simple o define and consequenly o measure and o monior he repuaion concep since i involves inangible asses such as: honor, public opinion, percepion, reliabiliy, meri. By he ay, i is a maer of fac, ha a bad repuaion can seriously affec and condiion he performance of a company. Moreover companies end o ac once he adverse even has occurred. According o such approach e can say ha here is no a ris managemen aciviy, bu only a crisis managemen. ih regards o repuaional ris, media coverage plays a ey role in deermining a company s repuaion. This ofen occurs hen a company repuaion has been significanly damaged by unfair aacs from special ineres groups or

8 2096 P. CERCHIELLO, P. GIUDICI inaccurae reporing by he media. A deailed and srucured analysis of ha he media are saying is especially imporan because he media shape he percepions and expecaions of all he involved acors. Naural language processing echnologies enable hese services o scan a ide range of oules, including nespapers, magazines, TV, radio, and blogs. In order o enable he applicaion of he classificaion exual model sbdcm e have collaboraed ih he Ialian mare leader company in financial and economic communicaion, Sole24ORE. Sole- 24ORE eam provided us ih a se of 870 aricles abou Alialia, an Ialian fligh company, covering a period of one year (Sep 07-Sep 08). The 80% of he aricles are used o rain he model and he remaining 20% o validae he process. The objecive is o classify he aricles on he basis of he repuaion onology in order o undersand he argumen reaed in he aricles. The onology classes used for he classificaion are: Ideniy: he percepion ha saeholders have of he organizaion, person, produc. I describes ho he organizaion is perceived by he saeholders. Corporae Image: he persona of he organiaion. Usually for companies visibly manifesed by ay of branding and he use of rademars and involves he mission and he vision. I involves brand value. Inegriy: personal inner sense of holeness deriving from honesy and consisen uprighness of characer. Qualiy: he achievemen or excellence of an eniy. Qualiy is someimes cerificaed by a hird par. Reliabiliy: abiliy of a sysem o perform/mainain is funcions in rouine and also in differen hosile or/ and unexpeced circumsances. I involves cusomer saisfacion and cusomer fideliaion. Social Responsibiliy: social responsibiliy is a docrine ha claims ha an organizaion or individual has a responsibiliy o sociey. I involves foundaion campaign and susainabiliy Technical Innovaion: he Inroducion of ne echnical producs or services. Measure of he RD orienaion of an organizaion (only for companies). Value For Money: he exen o hich he uiliy of a produc or service jusifies is price. Those classes define he concep of repuaion of a company. To lin he onology classes o he exual analysis e use a se of ey ords for each class of he repuaion schema. Since he aricles are in Ialian, he ey ords are in Ialian as ell. For example he concep of Reliabiliy involves cusomer saisfacion and cusomer fideliaion and is characerized by he folloing se of ey ords: affidabilià, fiducia, consumaori, risorsa, organizzazione, commerciale, dinamicià, valore, mercao. On he basis of hese ey ords, e perform a grouping analysis considering he 9 classes. From he clusering, e derive he marix D. Empirical resuls sho good performance in erms of correc classificaion rae. e obain ha, given 171 aricles, e correcly classify he 68% of hem. 6. Conclusions This conribuion has shon ho o enrich he DCM model ih a semanic exension. e also have proposed a mehod o inser laen opics ihin he Dirichle Compound Mulinomial (DCM) ihou losing he ords bursiness : e call such a disribuion semanicbased Dirichle Compound Mulinomial (sbdcm). The approaches assessed depend on he noledge abou he opics T. In fac here can be o alernaive conexs: on one hand he opics are unnon in advanced, hus o be esimaed on he basis of daa a hand. On he oher hand a ex corpora could be enriched by several descripions of reaed opics according o he experience of he field exper(s). Specifically, he analysis can be empoered ih a priori noledge based on onological schemas ha describe he relaions among conceps ih regards o he general class argumen of he corpora. In order o inser opics e creae a ne coefficien vecor for each opic and laer on e obain he parameers as a linear combinaion of hem. The mehodology is based on a marix D conaining he degree of membership of each ord o a cluser (i.e. a opic) by using he cluser disance marix. Then e spli he ords coun vecors among laen opics and, by employing a fixed-poin ieraion, e generae he β coefficiens represening he opics eighs. In order o compare he o models DCM and sbdcm e have employed a Naive Bayes Classifier based on he esimaed disribuions as shon in [13]. Several classifiers have been proposed and esed, and among hem he bes performance is obained by means of he mixed formula and CM 10. Moreover, e run several ess o verify if he classificaion performance reached ih an for each ord (DCM) is mainained or improved by he sbdcm. Such an objecive has been accomplished employing o differen approaches. e propose o differen mehods o generae parameers, one based on he correlaion among ords C and he second based on he Krusal-allis index calculaed on he ords coun marix G. The resuls repor ha he es performances in erms of misclassificaion rae are quie close o each oher and o he performance reached by he DCM. Hoever he sbdcm disribuion is able o obain beer resuls in erms of AIC and log-lielihood especially in he case of marix G. Concluding, by using marix D o describe ho ords coun vecors can be spli among opics, he as eighs for he opic and as a linear combinaion e are able o obain an opimal classifi-

9 P. CERCHIELLO, P. GIUDICI 2097 caion performance and o follo he bursiness. 7. Acnoledgemens This or has been suppored by MUSING 2006 conrac number , and FIRB, ). The paper is he resul of he close collaboraion beeen he auhors, hoever he paper has been rien by Paola Cerchiello under he supervision of Paolo Giudici. REFERENCES [1] S. Deereser, S. Dumais, G.. Furnas, T. K. Landauer and R. Harshman, Indexing by Laen Semanic Analysis, Journal of he American Sociey for Informaion Science, Vol. 41, No. 6, 1990, pp doi: /(sici) (199009)41:6<391::aid-as I1>3.0.CO;2-9 [2] T. Hofmann, Probabilisic Laen Semanic Indexing, Proceedings of Special Ineres Group on Informaion Rerieval, Ne Yor, 1999, pp [3] D. M. Blei, A. Y. Ng and M. I. Jordan, Laen Dirichle Allocaion, Journal of Machine Learning Research, Vol. 3, 2003, pp [4] M. Girolami and A. Kaban, On an Equivalence beeen PLSI and LDA, Proceedings of Special Ineres Group on Informaion Rerieval, Ne Yor, 2003, pp [5] D. M. Blei and J. D. Laffery, Correlaed Topic Models, Advances in Neural Informaion Processing Sysems, Vol. 18, 2006, pp [6] D. Puhividhya, H. T. Aias and S. S. Nagarajan, Independen Facor Topic Models, Proceeding of Inernaional Conference on Machine Learning, Ne Yor, 2009, pp [7] J. E. Mosimann, On he Compound Mulinomial Disribuion, he Mulivariae B-Disribuion, and Correlaions among Proporions, Biomeria, Vol. 49, No. 1-2, 1962, pp [8] K. Sjolander, K. Karplus, M. Bron, R. Hughey, A. Krogh, I. S. Mian and D. Haussler, Dirichle Mixures: A Mehod for Improving Deecion of ea bu Significan Proein Sequence Homology, Compuer Applicaions in he Biosciences, Vol. 12, No. 4, 1996, pp [9] D. J. C. Macay and L. Peo, A Hierarchical Dirichle Language Model, Naural Language Engineering, Vol. 1, No. 3, 1994, pp [10] T. Mina, Esimaing a Dirichle disribuion, Unpublished Paper, hp://research.microsof.com/en-us/um/people/mina/pap ers/dirichle/ [11] R. E. Madsen, D. Kaucha and C. Elan, Modeling ord Bursiness Using he Dirichle Disribuion, Proceeding of he 22nd Inernaional Conference on Machine Learning, Ne Yor, 2005, pp [12] G. Doyle and C. Elan, Accouning for Bursiness in Topic Models, Proceeding of Inernaional Conference on Machine Learning, Ne Yor, 2009, pp [13] J. D. M. Rennie, L. Shih, J. Teevan and D. R. Karge, Tacling he Poor Assumpions of Naive Bayes Tex Classifier, Proceeding of he 20h Inernaional Conference on Machine Learning, ashingon DC, 2003, 6 p. [14] A. P. Dempser, M. N. Laird and D. B. Rubin, Maximum Lielihood from Incomplee Daa via he EM Algorihm, Journal of he Royal Saisical Sociey, Series B, Vol. 39, No. 1, 1977, pp [15] D. Böhning, The EM Algorihm ih Gradien Funcion Updae for Discree Mixure ih Kno (Fixed) Number of Componens, Saisics and Compuing, Vol. 13, No. 3, 2003, pp doi: /a: [16] S. Saab and R. Suder, Handboo on Onologies, Inernaional Handboos on Informaion Sysems, 2nd Ediion, Springer, Berlin, [17] P. Cerchiello, Saisical Models o Measure Corporae Repuaion, Journal of Applied Quaniaive Mehods, Vol. 6, No. 4, 2011, pp