METHODS OF EVALUATION AND IMPROVEMENT OF CONSISTENCY OF EXPERT PAIRWISE COMPARISON JUDGMENTS. Nataliya Pankratova, Nadezhda Nedashkovskaya

Transcription

1 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, METHOS OF EVALUATION AN IMPROVEMENT OF CONSISTENCY OF EXPERT PAIRWISE COMPARISON JUGMENTS Natalya Pakratova, Nadezhda Nedashkovskaya Absact: The oto of cosstecy s used to evaluate the coadcto level of expert parwse comparso udgmets ad ther sutablty for the purpose of calculato of weghts of decso alteratves. Several cosstecy coeffcets ad crtera are used to measure cosstecy of a parwse comparso max (PCM). Tradtoal approach to crease cosstecy of expert formato s to orgaze a feedback wth a expert. However, t s ot always possble due to facal ad tme lmtatos. The paper deals wth methods of mprovemet (crease) PCM cosstecy wthout partcpato of a expert. Computer smulato s used to provde a comparatve study of these methods. It s show that takg a admssbly cosstet PCM, for example, wth the cosstecy rato equal to CR=0. or CR=0.3, methods of mprovemet PCM cosstecy help to decrease cosstecy up to admssble level CR 0. for 5. Results reveal that these methods, ufortuately, are ot always effectve, that s, drawg ear to admssble cosstecy does ot esure closeess to the vector of weghts of decso alteratves. Also a aalyss of coeffcets ad crtera of cosstecy of expert parwse comparso udgmets depedg o PCM propertes ad level of ts cosstecy s carred out. Keywords: parwse comparso max, coeffcets of cosstecy, admssble level of cosstecy, cosstecy crtera, weak cosstecy, methods of cosstecy mprovemet wthout partcpato of a expert, methods of detfcato of the most cosstet udgmet, a effectve mprovemet of cosstecy ACM Classfcato Keywords: H.4.. Iformato System Applcato: type of system sategy Ioducto Parwse comparso methods are part of several popular decso support techologes such as the aalytc herarchy process [Saaty, 003, 008; Pakratova & Nedashkovskaya, 00], the goal evaluato of decso alteratves [Totseko, 000; Tsygaok, 00], PROMETHEE [Machars et al, 004]. These methods result coeffcets of relatve mportace or weghts w R, w of

2 04 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 decso alteratves based o expert parwse comparso maces (PCMs) PCMs. For stace, used the aalytc herarchy process are postve d 0, verse symmecal d / ad take values from the Saaty fudametal scale [Saaty, 008]. Calculato of weghts s d ofte based o the dea of mmzato of devato orm of expert PCM ( from ukow cosstet (theoretcal) PCM C w / w ). The cosstet PCM s supposed to be the best approxmato of expert PCM. Several parwse comparso methods are proposed to calculate weghts w R. Oe of the most popular oe s the egevector method (EM) [Saaty, 003]. epedg o the choce of max orm other parwse comparso methods are used, such as the least squares (LSM), the weghted least squares (WLS), the logarthmc least squares (LLSM), kow also as the row geomec mea method (RGMM), the arthmetc ormalzato (AN) method ad other (see overvew [Pakratova & Nedashkovskaya, 00]). These methods are cosdered approxmatos of the EM method, whch do ot requre calculato of egevalues ad egevectors of max. The oto of PCM cosstecy s used for evaluato of coadcto level of expert parwse comparso udgmets ad ther sutablty for the purpose of calculato of weghts of decso alteratves. Several cosstecy coeffcets ad crtera are proposed [Saaty, 008; Totseko, 000; Aguaro & Moreo-Jmeez, 003; Peláez & Lamata, 003; Ste & Mzz, 007]. urg last years the attempts to aalyze ad compare cosstecy coeffcets were made [Pakratova & Nedashkovskaya, 0, 03; Bruell et al, 03]. Ivestgatos of relablty of weghts based o cosstet, cosstet ad astve PCMs were doe [Lares, 009; Sra et al, 0; Nedashkovskaya, 04]. A PCM cosstecy s caused by usage of the Saaty fudametal scale, psychologcal lmtatos of expert, expert accuraces whe makg udgmets. Tradtoal approach to crease cosstecy of expert formato s to orgaze a feedback wth a expert that s expert s asked to overvew all or the most cosstet hs/her udgmets [Lpovetsky & Cokl, 00; Sra et al, 0; Nedashkovskaya, 03]. Expert s asked to repeat the overvew procedure utl admssble cosstecy of PCM s acheved. However, feedback wth a expert s ot always possble due to facal or tme lmtatos. Aother approach to mprove (crease) PCM cosstecy cossts of alterato of ts elemets usg adustmet algorthms wthout partcpato of a expert [Xu & a, 003; Beítez et al, 0; Nedashkovskaya, 03]. Purpose of the paper s to provde a comparatve study of several kow methods of mprovemet (creasg) of PCM cosstecy wthout partcpato of a expert ad also to provde ad aalyss of coeffcets ad crtera of cosstecy of expert parwse comparso udgmets depedg o PCM propertes ad level of cosstecy.

3 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, Problem statemet Let { d,,..., } be a parwse comparsos max (PCM) of decso alteratves a, a,..., a based o expert udgmets, such that d 0 ad d / d (property of verse symmey). The oto of cosstecy s used to estmate the qualty of expert udgmets ad ther sutablty for relable evaluato of decso alteratves. PCM s called cosstet (sogly cosstet) f astvtes dkdk,, k,..., [Saaty, 008]. d are hold for all PCM s called weak or ordal cosstet f ordal astvtes ( d ) ( d ) ( d ), ( d ) ( d ) ( d ), k k ( d ) ( d ) ( d ), ( d ) ( d ) ( d ) are hold [Sra et al, 0]. k k k k k k A weak cosstet PCM has at least oe cycle defed by plet of dexes (,, k ), such that ( d ) ( d ) ( d ) or ( d ) ( d ) ( d ), or k k ( d ) ( d ) ( d ), or ( d ) ( d ) ( d ). It does ot exst a rakg of k k k k k k decso alteratves that satsfy all elemets of weak cosstet PCM, amely a a f d, a a f d ad a a f d. Statemet : If PCM s cosstet, the t s weak cosstet. Some level of cosstecy of correspodg PCMs s admssble whe solvg -lfe decsomakg problems [Saaty, 008; Totseko, 000; Aguaro & Moreo-Jmeez, 003; Peláez & Lamata, 003; Ste & Mzz, 007]. It s defed by several cosstecy crtera. Several coeffcets are used to measure cosstecy of a PCM, amely cosstecy rato CR [Saaty, 008], geomec dex cosstecy GCI [Aguaro & Moreo-Jmeez, 003], harmoc cosstecy rato HCR [Ste & Mzz, 007], cosstecy dex of astvtes cosstecy k y [Totseko, 000]. CI [Peláez & Lamata, 003] ad specal coeffcet of

4 06 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 Cosstecy crtero [Saaty, 008; Aguaro & Moreo-Jmeez, 003; Peláez & Lamata, 003; Ste & Mzz, 007; Pakratova & Nedashkovskaya, 03]: PCM s sogly cosstet f CR( ) 0, GCI ( ) 0, HCR( ) 0, ad CI ( ) 0; PCM s admssbly cosstet f CR( ) CR or GCI ( ) GCI, or HCR( ) HCR, or CI ( ) CI (depedg o utlzed cosstecy coeffcet), where CR, GCI, HCR, CI are threshold values of correspodg coeffcets [Saaty, 008; Aguaro & Moreo-Jmeez, 003; Peláez & Lamata, 003; Ste & Mzz, 007]; PCM s admssbly cosstet, that s the PCM requres a adustmet, f the cosstecy coeffcets exceed ther threshold values; PCM does ot cota formato f ormalzed coeffcets CR( ) or HCR( ). Cosstecy crtero [Totseko, 000]: PCM PCM s sogly cosstet f k ( ) ; y s admssbly cosstet f k ( ) T ; y u PCM s admssbly cosstet, that s the PCM requres a adustmet, f ( k ( ) T ) ( k ( ) T ); y 0 y u PCM does ot cota formato f y( ) 0 k T, where T 0, Tu are threshold values that are defed, respectvely, o bases of specum that cota mmal amout of formato ad specum of acceptable accuracy. If PCM s admssbly cosstet terms of the cosstecy crtero or, the ts elemets are coflct to each other ad, therefore, requre adustmets purpose of mprovemet of ther cosstecy [Saaty, 008; Totseko, 000; Aguaro & Moreo-Jmeez, 003; Peláez & Lamata, 003; Ste & Mzz, 007]. The purpose of the paper s to vestgate the cosstecy coeffcets CR, GCI, HCR, CI ad k y, the cosstecy crtera ad ad methods of mprovemet of cosstecy of a PCM depedg o ts propertes.

5 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, Ivestgato of the cosstecy coeffcets ad crtera fferet methods result the same weghts of decso alteratves a, a,..., a o bass of a PCM { d,,..., } oly whe ths PCM s sogly cosstet. Otherwse the dfferet methods result dfferet weghts. A admssbly cosstet PCM ca be used for relable evaluato of decso alteratves ad, cosequetly oe ca ust to weghts ad rakg calculated o bass of ths PCM [Saaty, 008; Totseko, 000; Aguaro & Moreo-Jmeez, 003; Peláez & Lamata, 003; Ste & Mzz, 007]. But, as the followg example shows, dfferet cosstecy coeffcets result dfferet coclusos regardg admssble cosstecy of a PCM. Example : Let us cosder several PCM: 3 3 / 3 / 5 / / / / /, 5 / 3 / 3 5 / 3 3 / 5 / / / 3 / /, 5 / 5 / 3 3 / 3 / 5 / 3 / / /. 5 / 3 / / PCMs ad are weak cosstet ad 3 s weak cosstet by defto. Cosstecy coeffcets of the PCM are equal to CR =0.064, GCI=0.3, HCR=0.043 ad =0.785 ad do ot exceed correspodg threshold values CR 0., GCI 0.37, HCR 0. ad CI.39. Therefore the PCM s admssbly cosstet terms of the cosstecy crtero. However, the vector of weghts w ( ) based o usg the AN method results rakg of decso alteratves that dffers from the rakg usg the EM ad RGMM methods (Table ). Cosstecy coeffcets of the PCM lead to dfferet results regardg the admssble cosstecy. For example the PCM s admssbly cosstet terms of the HCR coeffcet ad s admssbly cosstet terms of the CR, GCI, CI =.64, k y =0.664). CI ad CI k y coeffcets (HCR=0.084, CR=0.5, GCI=0.440, The cosstecy crtero defes weak cosstet PCM 3 as admssbly cosstet terms of all coeffcets (CR=0.094, GCI=0.3, HCR=0.047, CI =.3), so the cosstecy crtero does ot detfy volato of ordal astvty (a cycle) the PCM 3. I terms of the cosstecy crtero the PCM 3 requres adustmet ( k y =0.78).

6 08 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 The vectors of weghts based o PCMs, ad 3 usg the AN method result rakgs that dffer from the rakgs usg the EM ad RGMM methods (Table ). Table. The vectors of weghts ad rakgs based o the PCMs, ad 3 usg the EM, RGMM ad AN methods PCM Method Vector of weghts of decso alteratves Rakg of decso alteratves ЕМ w=(0.3, 0.06, 0.04, 0.30, 0.60) a4>a>a5>a>a3 RGMM w=(0.34, 0.0, 0.0, 0.30, 0.6) a4>a>a5>a>a3 AN w=(0.36, 0.090, 0.5, 0.3, 0.47) a>a4>a5>a3>a ЕМ w=(0.34, 0.6, 0.095, 0.306, 0.4) a>a4>a5>a>a3 RGMM w=(0.34, 0.09, 0.09, 0.35, 0.44) a>a4>a5>a>a3 AN w=(0.350, 0.094, 0.06, 0.33, 0.8) a>a4>a5>a3>a 3 ЕМ w=(0.300, 0.5, 0.40, 0.39, 0.7) а4>a>a3>a5>a RGMM w=(0.36, 0.03, 0.34, 0.33, 0.4) a4>а>a3>a5>a AN w=(0.37, 0.090, 0.38, 0.33, 0.) a>a4>a3>a5>a The example shows that cosstecy coeffcets CR, GCI, HCR ad CI may lead to dfferet results regardg a admssble cosstecy of a PCM. Results terms of the cosstecy crtera ad also may be dfferet. A admssble cosstecy of a PCM terms of the cosstecy crtera ad does ot guaratee the same rakg of decso alteratves usg the EM, RGMM ad AN methods. The cosstecy crtero does ot detfy volato of ordal astvty (a cycle) a PCM. Therefore t s suggested to requre a addtoal property of weak cosstecy of a PCM to be carred out for the purpose of relable evaluato of qualty of ths max.

7 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, Computer modelg of PCMs wth wde rage of varato of ther cosstecy level shows that equalty of results terms of the cosstecy crtera ad deped o propertes of PCMs. For stace, f PCM s close to sog cosstet the results terms of these crtera are equal more tha 98% of expermets. If PCM s weak cosstet ad 7 the crtera ad lead to the same results more tha 90% of expermets. I the case of weak cosstet PCMs of small dmesos ( 3,4,5, 6 ) the equalty of results occurs lesser umber of expermets. I most expermets the cosstecy crtero results ecessty of adustmet of these PCMs whereas the PCMs are admssbly cosstet terms of the crtero. Results terms of the crtera ad are expected to be more detcal for these PCMs, f the threshold T u value the crtero wll be reduced. The small lear depedece betwee CR ad k was detfed [Pakratova & Nedashkovskaya, y 03], amely the coeffcet of determato R the regressos was equal to the values 0.6, 0.0, 0.8 ad 0.04 depedg o the level of cosstecy of PCMs. 3. Improvemet of cosstecy of a PCM wthout partcpato of a expert I the paper dfferet methods to mprove the cosstecy of a PCM wthout partcpato of a expert are vestgated, ad some recommedatos to applcato of these methods depedg o propertes of PCMs are gve. 3.. Multplcatve ad Addtve Methods to Improve the Cosstecy of a PCM Let be tal PCM ad CI, GCI, HCI ad CI be cosstecy dex, geomec cosstecy dex, harmoc cosstecy dex ad cosstecy dex of astvtes of the PCM respectvely. Let us deote adusted PCM wth mproved cosstecy by. Let CI, GCI, HCI ad CI be cosstecy dexes of. Multplcatve ad addtve methods to adust a PCM for the purpose of mprovg ts cosstecy are proposed [Xu & a, 003]. Geeralzatos of these methods are doe ad ext two statemets are proved [Nedashkovskaya, 03]. Statemet (the multplcatve method) [Nedashkovskaya, 03]: Let us deote elemets of the adusted PCM by w d ( d ) ( ), where (0,). The w CI CI, GCI GCI, HCI HCI ad CI CI ; ad equatos CI CI, GCI GCI, HCI HCI ad CI CI hold f s cosstet.

8 0 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 Statemet 3 (the addtve method) [Nedashkovskaya, 03]: Let us deote elemets of the adusted PCM w by d d ( )( ), f w ad d w ( d ( )( )), f, w where (0,). The CI CI, GCI GCI, HCI HCI ad CI CI ; ad equatos CI CI, GCI GCI, HCI HCI ad CI CI hold f s cosstet. Statemets ad 3 show that adustmet of a PCM usg the multplcatve ad addtve methods helps to crease cosstecy level of ths PCM terms of coeffcets HCR HCI MRHCI, GCI ad CI, where MRCI ad MRHCI are tabular values. CI CR, MRCI Adustmet algorthm Accordg to ths algorthm, elemets of a PCM are teratvely altered based o the multplcatve or addtve methods tll ts cosstecy level become admssble. Let us use the cosstecy rato CR to measure cosstecy level of a PCM. The algorthm cossts of the followg stages [Xu & a, 003]:. efe parameter (0,). Calculate CR value of the PCM.. Whle CR CR :.. Calculate weghts w w,, w T.. Calculate adusted PCM : based o the PCM. d w d d or w w d ( ), w d, w d ( ) w,,..., ;,,...,,3,..., ;,,...,.3. Calculate CR value of the PCM.4. :..

9 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 Statemet 4 (covergece of the algorthm): For the descrbed algorthm k lm CR 0, where k values of k are calculated usg the algorthm. (k ) CR,,,... It s essetal to take to accout, whe defg the parameter (0,), that greater values of lead to smaller devatos of adusted PCM from the tal oe ad, cosequetly, the greater umber of teratos of the algorthm s eeded to acheve the admssble cosstecy. It s suggested to use Methods of Idetfcato of the Most Icosstet Elemet of a PCM The CI method was suggested [Lpovetsky & Cokl, 00] ad geeralzed [Nedashkovskaya, 03]. It s based o statemet that the cosstecy coeffcets (CC) CI, GCI, HCI ad CI possess ther mmal values ad the specal coeffcet of cosstecy value whe PCM s cosstet. The CI method cossts of several stages [Nedashkovskaya, 03]: k y possesses ts maxmum. Calculate the PCM ( ) ( ) by excepto -th row ad -th colum of tal PCM. Calculate the cosstecy coeffcets CC of the. Fd two mmal values of cosstecy coeffcet: ( ) ( ) arg m CC( ( ) ( ) ),,.., arg m CC( ( ) ( ) ),..,,,,...,. The elemet d s the most cosstet terms of cosstecy coeffcets CI, GCI, HCI ad CI. Fd two maxmum values of the coeffcet k y : arg max k y ( ( ) ( ) ),,.., arg max k y ( ( ) ( ) ),..,,. The elemet d s the most cosstet oe terms of the specal coeffcet. k y

10 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 The Corr method s based o statemet that the correlato betwee vectors-rows of a PCM ad also the correlato betwee vectors-colums of a PCM becomes close to uty whe cosstecy level of ths PCM s creased. The Corr method cossts of several stages [Lpovetsky & Cokl, 00]: r. Calculate expectato values M R of coeffcets of correlato betwee -th ad all other vectors-rows of a PCM, ad also expectato values M R c of coeffcets of correlato betwee -th ad all other vectors-colums of a PCM,,,...,. r. Fd the mmum value of vectors M R ad M R c : r arg mm R, arg m M R The elemet d s the most cosstet oe. c. The X method s based o the Х-square crtero [Lpovetsky & Cokl, 00]:. Calculate values for each elemet d of a PCM: d t, where t t dk dl / dkl. k l k l. Calculate the expectato value ad varace of values {,,,..., }. Calculate the cofdece terval. 3. Fd values whch are outsde the cofdece terval. The elemets d defed by the correspodg pars of dexes (, ) are the most cosstet. The Outflow method [Sra et al, 0]:. Calculate the outflow for each decso alteratve a - the umber of decso alteratves a, such that a outperforms a, amely d.. Fd the maxmum value of dfferece : d : max( ), f, d,., The elemet d s the most cosstet oe. Suppose that several elemets d result maxmum value of dfferece. The t s ecessary to fd a elemet amog them whch lead to more cosstecy, amely a elemet, whch

11 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 3 result the maxmum value of expresso (l d, l( d, k dk, )) k, where k. The Tr method s based o aalyss of astvtes of a PCM [Nedashkovskaya, 03]:. Calculate the set of astvtes { u }, u,..., NT : u { d, d k, d k}, k,,,...,, k,! NT, 3 of a PCM ad values of determats of these astvtes: ( 3)!3! dd k et {det( u )} d, k det( ). u d d d d. Fd values d k d S k, for each,,..., ad ther maxmum value k dk dd k (, ) : max S,, k k The elemet d defed by the correspodg par of dexes (, ) s the most cosstet oe.. Adustmet algorthm Algorthm of cosstecy mprovemet of a PCM s based o the CI, Corr, X, Outflow ad Tr methods. It teratvely fds the most cosstet elemets of a PCM, whle admssble cosstecy wll be acheved. Let us use the cosstecy rato CR to measure cosstecy level of a PCM. The algorthm cotas several stages:. Calculate CR value of gve PCM.. Whle CR CR :.. Fd the most cosstet elemet d of the PCM usg oe of the CI, Corr, X, Outflow or Tr methods. d.. Calculate the adusted PCM, whch oly elemet d ad ts verse symmecal elemet d / d are adusted. I accordace wth the Saaty scale, elemet d s gve a value that esures the greatest cosstecy of all PCM, that s, the mmum value of CR..3. Calculate CR value of the PCM.4. :..

12 4 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 Let us provde a comparatve study ad aalyss of descrbed adustmet algorthms ad methods of detfcato of the most cosstet elemets of a PCM wthout partcpato of a expert A Comparatve Study of Methods of Improvemet of PCM Cosstecy wthout Partcpato of a Expert Computer smulato s used to provde a comparatve study of methods of mprovemet (crease) of PCM cosstecy wthout partcpato of a expert. Test sets of sogly cosstet PCMs ad admssbly cosstet PCMs (deoted as smulato. We suppose the PCMs dsturbed so that ther cosstecy s creased. Let PCM. PCMs bass of adusted PCM adustmet). ) Saaty scale are geerated the course of the represet ratos of weghts. Further the PCMs are dsturb be result of dsturbace of PCM dsturb are adusted usg the algorthms ad. We aalyse how much weghts o are closer to weghts the weghts o bass of tal PCM (before the Improvemet of cosstecy s cosdered to be effectve f vector of weghts w o bass of adusted PCM PCM dsturb : s closer to vector of weghts w the vector of weghts w o bass of tal where dst s agular dstace: dst( w, w ) dst( w, w ), dst( x, y) x x y y. The measure dst s vald whe evaluatg effectveess of parwse comparso methods [0]. Suffcetly large test sets of PCMs for each value of 9 3,4,5,..., were geerated. Several techologes of dsturbace of PCM dsturb (l) were used. I each l th expermet dsturbed PCM was adusted usg the algorthms ad ad the multplcatve ad addtve methods wth dfferet values of parameter, ad also the CI, Corr, X, Outflow ad Tr methods. The followg agular dstaces were calculated:

13 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 5 the dstace betwee vector of weghts based o PCM dst( l) dst( w( l), w ( l)) () dsturb ad vector w, dst ( l) dst( w ( l), w ( l)) () the dstace betwee vector of weghts w (l) based o adusted PCM ad vector w, where PCM s a result of usage of -th adustmet method ad l s a umber of expermet. sbutos of the data seres { dst ( l)} () ad { dst ( l)} () were, geerally, close to the ormal dsbuto. The ma statstcal characterstcs a sample mea ad sample varace values (Table 4) were calculated. I these tables the par ( m ; ) represets the statstcal characterstcs of the data seres { dst ( l)} (). Values of dstaces () ad () descrbe errors of evaluato of weghts w (l) ad w (l). Therefore, the -th adustmet method s more effectve o average f sample mea ad sample varace values of data seres { dst ( l)} () possess lesser values. Table. Expectato values ad stadard devatos for the seres () ad () whe PCMs weak cosstet ad coeffcet СR possesses values terval ( CR,.5 CR ) dsturb are Before adustmet ( m ; ) (0.347; 0.084) (0.34; 0.068) (0.338; 0.058) Adustmet algorthm mult mult ( m ; ) (0.346; 0.084) (0.34; 0.066) (0.340; 0.057) ad ad ( m ; ) (0.346; 0.084) (0.343; 0.066) (0.34; 0.057) CI CI ( m ; ) (0.36; 0.04) (0.367; 0.075) (0.36; 0.060) Adustmet algorthm Tr Tr ( m ; ) (0.36; 0.04) (0.343; 0.074) (0.34; 0.060) Oflow Oflow ( m ; ) (0.35; 0.099) (0.357; 0.074) (0.354; 0.063)

14 6 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 Table 3. Expectato values ad stadard devatos for the seres () ad () whe PCMs dsturb are weak cosstet ad coeffcet СR possesses values terval (3.5 CR, 5.5 CR ) Before adustmet ( m ; ) (0.9; 0.09) (0.38; 0.088) (0.388; 0.076) Adustmet algorthm mult mult ( m ; ) (0.35; 0.089) (0.393; 0.08) (0.40; 0.07) ad ad ( m ; ) (0.35; 0.09) (0.398; 0.084) (0.408; 0.07) CI CI ( m ; ) (0.36; 0.098) (0.467; 0.085) (0.469; 0.073) Adustmet algorthm Corr Corr ( m ; ) (0.33; 0.08) (0.444; 0.099) (0.436; 0.087) Tr Tr ( m ; ) (0.36; 0.098) (0.468; 0.083) (0.474; 0.07) Oflow Oflow ( m ; ) (0.400; 0.099) (0.497; 0.09) (0.480; 0.077) Table 4. Expectato values ad stadard devatos for the seres () ad () whe PCMs weak cosstet wth oe cycle dsturb are Before adustmet ( m ; ) (0.55; 0.55) (0.40; 0.56) (0.335; 0.45) Adustmet algorthm mult mult ( m ; ) (0.5; 0.4) (0.367; 0.35) (0.80; 0.9) ad ad ( m ; ) (0.59; 0.76) (0.38; 0.7) (0.97; 0.5) CI CI ( m ; ) (0.035; 0.045) (0.034; 0.045) (0.07; 0.04) Adustmet algorthm Corr Corr ( m ; ) (0.5; 0.84) (0.5; 0.8) (0.067; 0.9) Tr Tr ( m ; ) (0.035; 0.045) (0.034; 0.047) (0.05; 0.04) Oflow Oflow ( m ; ) (0.035; 0.045) (0.033; 0.046) (0.05; 0.039)

15 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 7 Results Tables 4 correspod to dfferet techologes of dsturbace of PCM. Thus Tables ad 3 results correspod to dsturbaces of all elemets of PCMs, ad PCMs were sogly cosstet. sturbaces have dfferet testy for the purpose of vestgato PCMs dsturb wth dfferet levels of cosstecy, that s, dfferet levels of chagg of CR values. Ifluece of weak cosstecy of PCM dsturb o result s also vestgated. Values Table 4 correspod to sog dsturbaces of small umber of elemets of PCMs lead to weak cosstet PCMs that dsturb wth cycles. Let us ote that tal PCMs were admssbly cosstet wth CR values CR( ) CR, ad are weak cosstet. Results of computer smulato lead to the followg coclusos:. Effectveess of cosdered adustmet algorthms (algorthms of cosstecy crease of a PCM) depeds o the percet of accurate elemets a PCM, that s, elemets that are sgfcat dsturbaces of values.. Let us cosder the case whe all elemets of a admssbly cosstet PCMs are sgfcatly dsturbed. The adustmets of the PCMs usg the algorthms ad wthout partcpato of a expert result admssble cosstet PCMs. But these adustmets do ot esure closeess to the vector of weghts (Tables ad 4). I partcular, let us cosder a admssbly cosstet PCM wth relatvely small level of cosstecy (Table ) ad addtoal property of weak cosstecy. Whe these tal PCM are adusted usg the adustmet algorthm the weghts o bass of the adusted PCM, o average, are practcally equally dstat from weghts o bass of the tal PCM. Weghts o bass of the adusted PCM, o average, are slghtly dstat from weghts o bass of the tal PCM whe these tal PCM are adusted usg the algorthm. If tal PCM has a greater level of cosstecy (Table 3), the weghts o bass of the adusted PCM, o average, are cosderably dstat from weghts o bass of the tal PCM. Effectveess of the adustmet algorthm does ot deped o parameter. The algorthm s more effectve tha the algorthm. Effectveess of the adustmet algorthms ad s decreased whe level of cosstecy of tal PCM s creased. Effectveess of the adustmet algorthms ad s slghtly decreased wth the growth of the dmeso of tal PCM. Effectveess of the adustmet algorthm s sgfcatly hgher, f tal PCM has a addtoal property of weak cosstecy ad 4 or 5. Ths fluece s reduced for bgger values of. It s worth ote that umber of weak cosstet PCMs s decreased wth the growth of.

16 8 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, The adustmet algorthms ad are effectve for the PCM, whch has small umber of sgfcatly dsturbed elemets (Table 4). For these PCMs the adustmet algorthm s cosderably more effectve tha the adustmet algorthm. Results Tables 4 ad 5 show that the CI, X, Tr ad Outflow methods are more effectve tha the Corr method. The Tr method has several advatages. It does ot requre computato of the prcpal egevalue of a PCM as opposed to the CI method. The Tr method detfes astvty whch defes a cycle a PCM ad provdes more formato about cosstecy of a PCM comparso wth other vestgated methods. The X method results a set of the most cosstet elemets of a PCM, that s, t does ot detfy whch oe has to be corrected the frst place. Table 5. The percet of correctly detfcato of the most cosstet elemet of a PCM usg the CI, Corr, X, Tr ad Outflow methods Method CI ,4 Method Corr Method X Method Outflow Method Tr I some cases the Corr ad Outflow methods result correct detfcato of a outler a PCM. For example, the Outflow method geerally results a correct detfcato of a cycle whe tal PCMs were close to sogly cosstet. However, these PCMs the Outflow method sometmes leads to correct outler that has to be adusted. I cosequece, all allowable adustmets of ths elemet do ot elmate a cycle a PCM. If PCM has several the most cosstet elemets, the CI, Corr, X, Tr ad Outflow methods may lead to dfferet results (example 3), that s ot ther drawback. Let us llusate these coclusos usg several examples.

17 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 9 Example llusates effectve applcato of the adustmet algorthm. Let us cosder the weak cosstet PCM. Oe of elemets of ths PCM s a sgfcat dsturbace of some value: / 4 / 3 / / 4 / 3 / 5 4, / 3 / / 4 / 3 / 5 / 4. The PCM s based o the PCM d,5 of the PCM. Suppose that the followg rakg based o the PCM decso alteratves: ad cotas a expert error whle defg the elemet s the rakg of fve a 5 a4 a3 a a (3) Suppose that provdg parwse comparsos of the decso alteratves, expert makes a error elemet d, 5 (ths elemet s a outler) ad gve the PCM. I accordace wth the cosstecy crtera ad the PCM requres adustmet sce t s admssbly cosstet (CR=0.9, GCI=0.676, HCR=0.70, CI =3.508 ad k y =0.546). The adustmet algorthm ad the multplcatve method wth values 0.5, 0.7, 0. 9 result the rakg a4 a3 a a5 a, whch dffers from the rakg (3). Thus, the multplcatve method ad the adustmet algorthm are effcet gve example. The smlar cocluso s made whe the adustmet algorthm ad addtve method are used to mprove the cosstecy of the PCM. Example 3: a) / 3 4 / 5 / / / 7 / 3 / 9 / 6 / 6 /8 / / / / 3 / 9 / /, b) 8 / 4.

18 0 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 The PCMs ad are weak cosstet. A ple of elemets ( d 3, 4, d 4, 5, d 3, 5 ) forms a cycle the PCM, sce ( 53 d 34 ) ( d45 ) ( d ). Elemet d 34 s the most cosstet the PCM. All descrbed methods besdes the Outflow method correctly defe ths elemet. The adustmet d : / results elmato of the cycle. Icosstecy level of adusted PCM s decreased ad the PCM becomes close to sogly cosstet sce ts cosstecy rato s equal to CR= Therefore the elemet d 34 was a outler. The Outflow method results elemet d 53 of the PCM. Ths result s faulty the case of PCM, sce all allowable by the Saaty scale adustmets of elemet d 53 help to mprove the cosstecy ot to the best advatage, amely these adustmets decrease the cosstecy oly up to the value CR= I weak cosstet PCM cycle s formed by a ple of elemets ( d, 5, d 5, 4, d 4, 34 ). The cosstecy rato of the PCM s equal to CR=0.. CI, Corr ad Tr methods result elemet d 4, 5 of, the Outflow method elemet d, 5 ad the X-squared method elemets d, 5, d 4, 3 ad d 5, 4. Adustmets of elemets d 4, 5 ad d, 5 ad ther ew values d 4,5 :, d,5 : lead to elmato of the cycle ad decreasg of cosstecy level of up to the same value CR=0.05. Thus, both elemets d, 5 ad 4, 5 d (ad also d 5, 4 ) are the most cosstet the gve example. Coclusos The paper deals wth the vestgato of several kow algorthms of cosstecy mprovemet (crease) of a PCM. Takg a admssbly cosstet PCM, for example, wth the cosstecy rato equal to CR 0. or CR 0. 3 these algorthms helps to decrease cosstecy up to admssble level CR 0. for 5. At the same tme, drawg ear to admssble cosstecy does ot esure closeess to the vector of weghts, that s, these algorthms are ot always effectve ths respect. Effectveess of cosdered adustmet algorthms (algorthms of creasg the cosstecy level of a PCM) depeds o the percet of accurate elemets a PCM, that s, elemets that are sgfcat dsturbaces of values. Ths percet may be estmated usg the astvtes method, proposed [Pakratova & Nedashkovskaya, 03]. Adustmet algorthms that were vestgated the paper are effectve for the PCM, whch has small umber of sgfcatly dsturbed elemets. For these PCMs the adustmet algorthm s cosderably more effectve tha the adustmet algorthm.

19 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 If all elemets of a admssbly cosstet PCM are sgfcatly dsturbed, the the adustmet algorthms ad wthout partcpato of expert result admssble cosstecy of ths PCM but do ot always esure closeess to the vector of weghts. For detfcato of the most cosstet elemets of a PCM t s recommeded to use the Tr method amog other kow methods. It has several advatages: does ot requre computato of the prcpal egevalue of a PCM as opposed to the CI method, also detfes astvty whch defes a cycle a PCM ad provdes more formato about cosstecy of a PCM comparso wth other vestgated methods. It was show that well-kow coeffcets of cosstecy CR, GCI, HCR ad CI do ot always correctly evaluate qualty of expert udgmets about parwse comparsos of decso alteratves. For the purpose of more relable evaluato of qualty of expert udgmets (PCMs) t s suggested to use a addtoal property of weak cosstecy of these PCMs. Bblography [Aguaro & Moreo-Jmeez, 003] J.Aguaro, J.M. Moreo-Jmeez. The geomec cosstecy dex: Approxmated thresholds. Europea Joural of Operatoal Research 47 (003) [Beítez et al, 0] J. Beítez, X. elgado-galvá, J. Izquerdo, R. Pérez-García. Achevg max cosstecy AHP through learzato. Appled Mathematcal Modellg. 35 (0) [Bruell et al, 03] M. Bruell, A. Crtch, M. Fedrzz. A ote o the proportoalty betwee some cosstecy dces the AHP. Appled Mathematcs ad Computato. 9 (03) [Lares, 009] P. Lares. Are cosstet decsos better? A expermet wth parwse comparsos. Europea Joural of Operatoal Research. 93 (009) [Lpovetsky & Cokl, 00] Lpovetsky S., Cokl W.M. Robust estmato of prortes the AHP. Europea Joural of Operatoal Research. 37, (00) 0. [Machars et.al, 004] Machars C., Sprgael J., Brucker K.., Verbeke A. PROMETHEE ad AHP: The desg of operatoal syerges multcrtera aalyss. Segtheg PROMETHEE wth deas of AHP. Europea Joural of Operatoal Research. 53, (004) [Nedashkovskaya, 03] N.I. Nedashkovskaya. Method of cosstet parwse comparsos whe estmatg decso alteratves terms of qualtatve crtero System research ad formato

20 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 techologes. 4 (03) ( ukraa). Access mode: oural.asa.kp.ua/artcle/vew/33943 [Nedashkovskaya, 04] N. I. Nedashkovskaya. ecso makg uder cosstet expert udgmets of parwse comparsos. System research ad formato techologes. 4 (04) ( ukraa) Access mode: oural.asa.kp.ua/artcle/vew/3747 [Pakratova & Nedashkovskaya, 00] N.. Pakratova, N. I. Nedashkovskaya. Models ad methods of aalyss of herarches. Theory ad applcatos. Kev (00). ( ukraa) [Pakratova & Nedashkovskaya, 0] N. Pakratova, N. Nedashkovskaya. Specal coeffcet of cosstecy of fuzzy expert formato ad estmato of ts sestvty to fuzzy scales whe solvg foresght problems. Iteratoal Joural «Iformato Techologes ad Kowledge», 6, 4. (0) Access mode: [Pakratova & Nedashkovskaya, 03] N. Pakratova, N. Nedashkovskaya. The Method of Estmatg the Cosstecy of Pared Comparsos. Iteratoal Joural «Iformato Techologes ad Kowledge» 7, 4. (03) Access mode: p05.pdf [Peláez & Lamata, 003] Peláez J.I., Lamata M.T. A New Measure of Cosstecy for Postve Recprocal Maces. Computers ad Mathematcs wth Applcatos. 46 (003) [Saaty, 003] Saaty T.L. ecso-makg wth the AHP: Why s the prcpal egevector ecessary. Europea Joural of Operatoal Research 45 (003) [Saaty, 008] Saaty T.L. ecso makg wth the aalytc herarchy process. It J Serv Sc () (008) [Sra et al, 0] S. Sra, L. Mkhalov, J. Keae. A heurstc method to rectfy astve udgmets parwse comparso maces. Europea Joural of Operatoal Research. 6 (0) [Ste& Mzz, 007] Ste W.E., Mzz P.J. The harmoc cosstecy dex for the aalytc herarchy process. Europea Joural of Operatoal Research Vol.77,. P [Totseko, 000] Totseko V.G. Specal Method for etermato of Cosstecy of Expert Estmate Sets. Egeerg Smulato. 7 (000), pp [Tsygaok, 00] Tsygaok V.V. Ivestgato of the aggregato effectveess of expert estmates obtaed by the parwse comparso method. Mathematcal ad Computer Modellg. 5, 3-4 (00)

21 Iteratoal Joural Iformato Theores ad Applcatos, Vol., Number 3, 05 3 [Xu & a, 003] Xu Z., a Q. A approach to Improvg Cosstecy of Fuzzy Preferece Max. Fuzzy Optmzato ad ecso Makg., (003) 3. [Zgurovsky & Pavlov, 009] M. Z. Zgurovsky, A. A. Pavlov. ecso makg etwork systems wth lmted resources. Kev, (00) Authors' Iformato Natalya Pakratova Ts, Professor, epute drector of Isttute for appled system aalyss, Natoal Techcal Uversty of Ukrae KPI, Av. Pobedy 37, Kev 03056, Ukrae; e-mal: ataldmp@gmal.com Maor Felds of Scetfc Research: System aalyss, Theory of rsk, ecso support systems, Appled mathematcs, Appled mechacs, Foresght, Scearos, Sategc plag, formato techology Nadezhda Nedashkovskaya Caddate of Sceces, Isttute for appled system aalyss, Natoal Techcal Uversty of Ukrae KPI, Av. Pobedy 37, Kev 03056, Ukrae; e-mal:.edashkvska@gmal.com Maor Felds of Scetfc Research: System aalyss, ecso support systems, Multple crtera aalyss, Itellectual systems, Foresght, Scearos, Sategc plag, Iformato techology