Classification of Risk in Software Development Projects using Support Vector Machine

Transcription

1 Classfcatn f Rsk n Sftware Develpment Prjects usng Supprt Vectr Machne M.Zavvar 1, A.Yavar 2, S.M. Mrhassanna 1, M.R.Neh 1, A.Yanp 1 and M.H.Zavvar 1 1 Department f Cmputer Engneerng, Sar Branch, Islamc Azad Unversty, Sar, Iran. 2 Mazandaran Unversty f Scence and echnlgy. Zavvar.develper@gmal.cm Abstract radtnally, the lack f cnfdence n the system lfe cycle s expressed usng the cncept f rsk. Nwadays, sftware develpment prjects face varus rsks. Hwever, the estmatn and classfcatn f rsk, ncreased estmatn f accuracy and reduced f uncertanty ultmately mprve prject utcmes. herefre, n ths paper, a Supprt Vectr Machne (SVM) s used t mdel rsk classfcatn n sftware develpment prjects. he prpsed algrthm s cmpared wth ther methds n the lterature such as Self Organzng Map (SOM) and K-Means based n measures f Classfcatn Accuracy Rate (CAR) and Area Under Curve (AUC). Accrdng t the results, the prpsed methd exhbts superr CAR and AUC. Index erms Classfcatn; Rsk; Sftware; Supprt Vectr Machne; Area under Curve. I. INRODUCION All sftware prjects are asscated wth rsk. he cncept f rsk has been defned n varus ways. Fr example, n [1] rsk s ntrduced as a tradtnal way f expressng the lack f relablty n the system lfe cycle. Als, events that ccur durng a sftware prject and threaten ts success are knwn as rsk [2]. It s nherented n all the prjects and cannt be cmpletely elmnated; nevertheless, t s pssble t reduce ts effects thrugh effectve management. Rsk refers t the expsure t ecnmc r fnancal lss, physcal njury r damage r delay, resultng frm uncertanty asscated wth a partcular current f a jb. Generally, sftware prject rsks are dvded nt tw grups: general and specfc. Whle the frmer may happen n all sftware prjects, the latter vares wth the type f prject. It s extremely dffcult t dentfy such rsks, partcularly n estmatng ther prbablty f ccurrence; and predctng ther mpact. Several factrs cntrbute t ths dffcultes, such as prject sze, cmplexty, structure, cntent, lng-term plannng and vlatle changes. herefre, rsk management results n reducng dsaster, duplcatn, cncentratn, and balancng the requred effrt; furthermre, t leads t smulatn f wn-wn cndtns [3]. Sftware develpment prjects are ften challenged by a varety f rsks. he mst mprtant factrs that culd lead t prject falure nclude lw effcency, tme pressure, pr qualty and hgh cst [4]. Effectve rsk management s a cmplex task that requres a gd evaluatn f the underlyng factrs. Due t the cmplexty f rsk factrs and the ntercnnected nature f uncertantes asscated wth future resurces, rsk cannt be expressed wth mathematcal precsn durng the early phases f the lfe cycle [5]. Rsk management f sftware prjects plays an mprtant rle n achevng the desred result. Actvtes carred ut n sftware prjects are nherently hgh rsk, and ths results n varyng degrees f functnalty. Managng sftware rsk has many benefts, ncludng ncreased relablty, mre accurate estmatns, and preventng unnecessary effrt [6]. As a result f the rsk assessment, accuracy s ncreased whle reducng the uncertanty asscated wth the prject. Mrever, snce team members are aware f the rsk cntrl measures, t s pssble t avd duplcatn and wasted effrt [4]. Management f sftware prjects cmprses fur phases: dentfcatn, evaluatn, plannng and cntrllng. Accrdng t Bhm, an estmatn f rsk s btaned by multplyng the prbablty f rsk ccurrence by ts effect. Qualtatve analyss f rsk and ts mpact are dependent n analyst experence as well as statstcal data [7]. Rsk n the applcatn depends n several small and large factrs; thus, they need t be classfed accrdng t sme crtera. One such classfcatns n the feld f sftware rsks s prpsed by Wallace. Varus reasns can be cted fr the applcatn f ths classfcatn ncludng ts use f up-t-date nfrmatn and the fact that t reflects a cnsensus amng the members f PMI. Als, n rder t prve the cmpetence and credblty f the framewrk, the SEM methd s used [8]. hs framewrk s cmpsed f sx dmensns and each dmensn crrespndng t rsks are dscussed. In able 1, dfferent aspects f ths classfcatn and the asscated rsks are mentned. Gven the mprtance f the classfcatn f rsk n sftware develpment prjects and the factrs presented n able 1, n ths paper, we prvde a methd based n a Supprt Vectr Machne (SVM) t classfy rsks nvlved n sftware develpment prjects. II. RELAED WORKS In [9], key rsks btaned frm a grup f nfrmatn technlgy prject managers n Hng Kng were ntrduced. In ther study, a number f new rsks were dentfed frm the perspectve f the seller. Accrdng t the authrs, prject managers beleved that peratns perfrmed n fregn cuntres are asscated wth greater rsk than thse carred ut dmestcally. ISSN: e-issn: Vl. 9 N. 1 1

2 Jurnal f elecmmuncatn, Electrnc and Cmputer Engneerng able 1 Wallace classfcatn f sftware rsk factrs [8] Rsk Dmensns User Requrements Cmplexty f the prject Plannng and Cntrl eam Organzatnal envrnment Rsk f Sftware Members resstance t changes Cnflct between users Users wth negatve atttudes twards the prject Users wh are nt cmmtted t the prject Lack f cperatn between Member Cnstant changes n system requrements System requrements are nt fully dagnsed N clearly stated system requrements Anachrnstc system requrements he use f new technlgy Hgh-level techncal cmplexty Immaturty f technlgy Usng technlges that have nt been used n prevus prjects Absence f prject management methdlgy that s effectve and effcent Lack f suffcent mntrng n prject prgress Incmplete estmate f the resurces requred Pr prject plannng Prject mlestnes are nt defned n a transparent manner he prject manager s nt experenced enugh Ineffectve cmmuncatn Insuffcent experence wth smlar prjects eam members are nt traned enugh Lack f specalzed sklls by team members Organzatnal management changes durng the prject Negatve mpact f trade plces n the prject Instablty n the rganzatnal envrnment Organzatnal restructurng durng the prject Furthermre, [10] fcused n the experences f I prject manager. her reprts unveled mre rsk and cntrls that allw us t cntrl the ccurrence f rsks n the future because f the dfferent factrs, whch are well explaned. Later, n [11], the same authrs used the mdel t mprve the qualty f nfrmatn technlgy prjects n rganzatns. In ther study, a nvel ch-square test was used t cntrl rsks n sftware prjects [12]. In [3], the authrs utlzed regressn testng and effect sze testng t mprve ther prevus wrk n reducng the rsk f sftware develpment prjects. Mrever, n [4], they prpsed new rsk management methd n ther sftware prjects by usng stepwse regressn. hs study was perfrmed by usng regressn analyss n rder t cntrl the cmparsn f each rsk factr s that the effect f each factr n the mplementatn phase culd be determned. he authrs n [13] cncluded that rsk management encmpasses prcess, methdlgy and specfc tls t assess and reduce rsk factrs asscated wth the develpment sftware lfe cycle. In [14], rsk management s argued t prvde a slutn t the standardzatn f rsk assessment and mtgatn n sftware develpment. In [15], by usng the Delph methd, sftware develpment rsks were extracted wth the help f experts. In ths study, a ttal f 53 rsks were classfed nt 14 grups. In [16], rsk management was shwn t cnsst f fve phases: rsk dentfcatn (plannng, dentfyng and prrtzng), analyzng and assessng rsk (rsk analyss, rsk assessment), rsk management, rsk cntrl, cmmuncatn, and dcumentatn. In [17], a mdel based n fuzzy lgc was prpsed fr evaluatng the rsk n sftware develpment prjects. he prpsed mdel was created usng the fve crtera, plannng, cmplexty, requrements; furthermre, three membershp functns were cnsdered fr nput t the prpsed fuzzy system. In ttal, 243 rules were desgned. he prpsed methd was shwn t have lwer estmatn errr cmpared t prevus methds n the lterature. Sme researchers n [18] emplyed neural netwrk (NN) and supprt vectr machne (SVM) appraches t establsh a mdel fr rsk evaluatn n prject develpment. In the mdel, the nput s a vectr f sftware rsk factrs that were btaned thrugh ntervew wth 30 experts, and the utput s the fnal utcme f the prject. he experment shws the mdel s vald. Interestngly. In ther study, the standard neural netwrk mdel had lwer predctn accuracy cmpared t SVM due t ts tendency n fndng lcal ptma. Yng et al. n [19] dentfed the key sftware rsk factrs respnsble n achevng successful utcme and used a neural netwrk apprach t establsh a mdel fr mnmzng the rsks attrbuted t faled prjects. In rder t enhance mdel perfrmance, prncpal cmpnent analyss and genetc algrthm were emplyed. he expermental result ndcates that the sftware rsk analyss can be mprved thrugh these methds and that the rsk analyss mdel s effectve. III. SUPPOR VECOR MACHINE A Supprt Vectr Machne s prncpally a lnear machne whse man dea s t create a hyperplane as a level f decsn-makng, s that the separatn between the pstve and the negatve samples s maxmzed. By usng a methd based n statstcal learnng thery, the technque acheves ths ptmzatn. Mre precsely, SVM s an mplementatn f apprxmatn f the "structural rsk mnmzatn". he structure f the SVM tranng algrthm s based n a cre f nner multplcatn between a supprt vectr such as x, and the vectr x derved frm the nput space. he smallest subset f tranng data extracted by the algrthm s knwn as the Supprt Vectr. Dependng n 2 ISSN: e-issn: Vl. 9 N. 1

3 Classfcatn f Rsk n Sftware Develpment Prjects usng Supprt Vectr Machne hw the multplcatn cre f the nner s frmed, dfferent tranng machnes wth nn-lnear decsn-makng planes may be btaned [20-22]. {( x, d )} N Cnsder tranng sample 1, where x s an nput algrthm fr the sample n and d s the crrespndng fnal utput, whch s dfferent frm the nput. We assume that classes dsplayed wth a subset f and d 1 d 1 are lnearly separated. A plane f decsnmakng equatn such as the abve plane s as fllws: w x b (1) where x dentes an nput vectr, w represents an adjustable weght vectr and b s a bas. he abve equatn can be wrtten as fllws: w x b fr d 1 w x b fr d 1 Fr a gven weght vectr w and a bas b, the dstance between the abve plane defned n Equatn (1) and the clsest data pnt s called the reslutn, represented by p. he bjectve f SVM s t fnd unque plane such that the reslutn p s maxmzed. In ths stuatn, the decsn level s cnsdered as the ptmal hyperplane. Fgure 1 shws the gemetry structure f the ptmal hyperplane fr a tw-dmensnal nput space. (2) Data pnts w x b 1 fr d 1 w x b 1 fr d 1 ( x, d ) (4), whch satsfy the cndtn f equalty fr bth frmulas (4) are called Supprt Vectr and hence they are called SVM. hese vectrs play a prmnent rle n the perfrmance f ths type f tranng machnes. Cnceptually, backup vectrs are data pnts lcated near the plane f makng decsns, and therefre they are dffcult t classfy. In rder t btan the ptmal hyperplane, nce the Lagrange multplers are appled and the necessary calculatns are perfrmed, the fllwng s perfrmed: {( x, d )} N Gven the tranng sample 1, fnd the Lagrange ceffcents { } N 1 functn s maxmzed: such that the fllwng bjectve 1 Q( ) a a a d d x x Subject t: N N N j j j j 1 (5) N ad (6) 1 0 C fr 1,2,..., N (7) where C s a pstve parameter dentfed by the user. IV. MEHODOLOGY Fgure 1: A schematc representatn f an ptmal hyperplane fr lnearly separable patterns [19]. Nw suppse w and b are the ptmal weght vectr and bas, respectvely. herefre, the ptmal hyperplane s defned as fllws: w x b (3) he man cncern n fndng the parameters w and b fr the ptmal hyperplane wth a set the tranng s {( x, d )}. hen, the par ( x, d ) must satsfy the fllwng cndtns: As mentned earler, rsk management durng the sftware develpment lfe cycle s asscated wth a large number f benefts, because t enables ncreased cnfdence, mre accurate estmate, and preventn f unnecessary effrts fr prject develpment. herefre, cnsderng the mprtance f addressng the ssue f rsk n sftware develpment prjects and relyng n the classfcatn by Wallace, we set ut t prpse a methd based n SVM fr classfcatn f the rsks nvlved n sftware develpment prjects. An SVM s a statstcal classfcatn mdel, whch ntally maps nn-lnear data n a hgh-dmensnal space thrugh several kernels and then tres t fnd the hyperplane that separates data wth the maxmum margn frm ths hyperplane. In ts general frm, SVM s used fr tw classes; thus, t s suffcent fr ths purpse. A crtcal feature f SVM s data classfcatn, whch s perfrmed by mnmzng data errr f test sets. In cntrast, n ther classfers such as neural netwrks perfrmance s based n mnmzng the errr f data f tranng sets. hus, n SVM, there s n cncern fr beng stuck n lcal mnma. In an attempt t enhance classfcatn accuracy n the prpsed methd and allw SVM nput data t be fcused n a specfc area, the nput data needs t be nrmalzed befre the prcedure cntnues. Data nrmalzatn s appled t data pnts that are nt n the same dman s that the values fall n the same range. ISSN: e-issn: Vl. 9 N. 1 3

4 Jurnal f elecmmuncatn, Electrnc and Cmputer Engneerng Als n the dataset, the desred utput s dvded nt ether hgh rsk and lw rsk n tw categres. Subsequent t the classfcatn, fr the sake f smplcty n the prpsed methd, hgh-rsk s replaced wth the value 1 and lw-rsk s replaced wth the value -1. Accrdngly, fr a gven sample, f the SVM generates an utput f 1, the rsk s assumed t be hgh, whereas an utput value f -1 represents a lw rsk. Als, t wrk wth SVM, the data shuld be dvded nt tw parts t be used fr tranng and test purpses. In ths paper, 70 percent f the data were used fr tranng the SVM whle the remanng were used fr testng. It shuld be nted that, all nput data fr bth tranng and testng were nrmalzed accrdng t the explaned prcedure prr t beng used. V. RESULS AND FINDINGS In ths sectn, the results f the applcatn f SVMbased methds are dscussed, and the results are cmpared wth thse f the SOM and K-Means algrthms. As stated earler, fr classfed peratns, the dataset was btaned frm sftware develpment prjects. hs data set ncludes 530 samples 70 percent f whch the data was used fr tranng, and 30 percent fr testng. In the prpsed methd fr SVM, the parameter C was cnsdered equal t 100 and accrdng t the data, a lnear functn was used as the kernel f the netwrk. he lnear functn s shwn n Equatn (8). K ( x, x ) x. x (8) j j he knwledge prduced n the learnng stage f the mdel must be analyzed n the evaluatn stage n rder t determne ts value, as well as the effcency f the learnng algrthm. hese measures can be calculated fr bth the tranng data set n the learnng phase and the test data set n the test phase. Als a cndtn fr success n the data mnng s the ablty t nterpret the btaned knwledge. One f the mprtant crtera used t determne the effectveness f a classfer s the CAR. In fact, ths crtern s the mst ppular crtern f standard algrthms and publc classfcatn that shws the percentage f ttal recrds crrectly classfer by the classfer. Usng equatn (9), the CAR s btaned [23]. Values f P and N are the mst mprtant values that shuld be maxmzed fr tw categres. In the prpsed methd, the value f CAR s equal t , and n the SOM methd, t s equal t and n methd f K- Means t s equal t N P CAR N FN P FP In addtn t the standard CAR, anther mprtant crtera used t determne the perfrmance f classfcatn crtera s the AUC. It represents the area under the Recever Operatng Characterstc (ROC) curve; larger values f the ROC are ndcatve f greater classfer margnal effcency. UAC s calculated thrugh Equatn (10) [24]: (9) 1 PR FPR AUC (10) 2 where PR P and P FPR FP. P and N FN FP N are the number f data pnts that have been crrectly classfed as pstve and negatve, respectvely. On the ther hand, FP and FN are the number f data pnts that has been falsely classfed as pstve and negatve, respectvely. A ROC curve allws a vsual cmparsn f a set f classfers; als numerus pnts n the ROC space are sgnfcant. he lwer left pnt (0, 0) ndcates the strategy that wll be generated n a pstve classfcatn. he ppste strategy, prduced wthut cndtn f pstve classfcatn, s determned wth tp rght pnt (1, 1). Pnt (0, 1) shws perfect grupng. Mre generally, cnsderng the tw pnts n the ROC space, ne s deemed better than the ther f mre space s lcated clser t the nrthwest crner. Als, t shuld be nted that the ROC curves shw a classfer behavr regardless f the dstrbutn f categres r cst f an errr; therefre, classfcatn perfrmance s separated frm these factrs [25]. Only when a classfer n the perfrmance space clearly dmnates the ther categres, t can be clamed t be superr. Fr ths reasn, the area under ROC curve shwng the AUC crtern can play a decsve rle n ntrducng categres f supremacy clause. In Fgure 2 the ROC curve, AUC value fr the prpsed apprach, and methds f SOM and K- Means are shwn. Unlke ther crtera fr determnng the effcency classfer, AUC crtern s ndependent f decsn-makng threshld f classfer. herefre, ths measure s ndcatve f the relablty f the utput f a classfcatn specfed fr dfferent data sets that ths cncept s nt cmparable by any ther perfrmance measures that derved the categry. As shwn n Fgure 2, the AUC s equal t n the prpsed methd, whereas t s equal t and fr SOM and K-Means, respectvely. Overall, the results regardng AUC and CAR shw that the prpsed methd utperfrms ther methds f rsk classfcatn n sftware develpment prjects. VI. CONCLUSION In ths paper, after hghlghtng the mprtance f the classfcatn f rsk n sftware develpment prjects and the factrs affectng t, an SVM-based methd was prpsed fr rsk classfcatn n sftware develpment prjects. hen, the classfcatn accuracy f the prpsed methd was cmpared wth that f SOM and K-Means based n the CAR and AUC. After examnatn, the CAR and AUC f the prpsed methd were equal t and , respectvely. he same values were and fr SOM and and fr K-Means. As thngs stand, the CAR and AUC n prpsed prcedure are hgher than the crrespndng values n SOM and K-Means. he results shw that the prpsed methd fr rsk classfcatn n sftware develpment prjects, exhbts better precsn and perfrmance. 4 ISSN: e-issn: Vl. 9 N. 1

5 Classfcatn f Rsk n Sftware Develpment Prjects usng Supprt Vectr Machne Fgure 2: ROC curve AUC value f the prpsed methd as well as that f ther methds REFERENCES [1] S. Cle, X. Gné, J. bacman, R. wnsend, P. palva and J. Vckery, "Barrers t husehld rsk management: evdence frm Inda", Amercan ecnmc jurnal. Appled ecnmcs, Vl.5, n.1, pp [2] P.L. "Bannerman, Rsk and rsk management n sftware prjects: A reassessment", Jurnal f Systems and Sftware, Vl.81, n.12, 2008, p [3] V.. Cvell, L. B. Lave, A. A. Mghss and V. R. R Uppulur, "Uncertanty n rsk assessment, rsk management, and decsn makng". Sprnger Scence & Busness Meda, Vl [4] M.K. Sadgrve, "he cmplete gude t busness rsk management". Ashgate Publshng, Ltd, [5] P. Bltn, H. Chen and N. Wang, "Market tmng, nvestment, and rsk management". Jurnal f Fnancal Ecnmcs, Vl.109, n.1, 2013, pp [6] J. Besss and B. O'Kelly, "Rsk management n bankng", Jhn Wley & Sns, [7] J. Lam, "Enterprse rsk management: frm ncentves t cntrls", Jhn Wley & Sns, [8] S.-J. Huang and W.-M. Han, "Explrng the relatnshp between sftware prject duratn and rsk expsure: A cluster analyss". Infrmatn & Management, Vl.45, n.3, 2008, pp [9] I. Rus, H. Neu and J. Münch, "A systematc methdlgy fr develpng dscrete event smulatn mdels f sftware develpment prcesses". In Prceedngs f the 4th Internatnal Wrkshp n Sftware Prcess Smulatn and Mdelng, [10] D. Ince and D. Andrews, "he Sftware lfe cycle", Butterwrth- Henemann, [11] S. Islam, H. Muratds and E.R. Weppl, "An emprcal study n the mplementatn and evaluatn f a gal-drven sftware develpment rsk management mdel". Infrmatn and Sftware echnlgy, Vl.56, n.2, 2014, pp [12] P. Hpkn, "Fundamentals f rsk management: understandng, evaluatng and mplementng effectve rsk management", Kgan Page Publshers, [13] A. Dumnt, P. Furner, M. Abrahamwcz, M. raré, S. Haddad, W.D. Fraser and QUARIE research grup, "Qualty f care, rsk management, and technlgy n bstetrcs t reduce hsptal-based maternal mrtalty n Senegal and Mal (QUARIE): a clusterrandmsed tral". he Lancet, Vl.382, n.9887, pp [14] D.R. Van Deventer, K. Ima and M. Mesler, "Advanced fnancal rsk management: tls and technques fr ntegrated credt rsk and nterest rate rsk management", Jhn Wley & Sns, [15] P. Chawan, J. Patl and R. Nak, "Sftware rsk management". Internatnal Jurnal f Cmputer Scence and Mble Cmputng, Vl.2, n.5, 2013, pp [16] R. Cnfrt, M. La Rsa, A.H. er Hfstede, G. Frtn, M. de Len, W.M. van der Aalst and M.J. Adams, "A sftware framewrk fr rsk-aware busness prcess management". n Prceedngs f the CASE'13 Frum at the 25th Internatnal Cnference n Advanced Infrmatn Systems Engneerng (CASE): CEUR Wrkshp Prceedngs, Vl [17] A.S. Khatavakhtan and S.H. Ow, "Develpment f a Sftware Rsk Management Mdel usng Unque Features f a Prpsed Audt Cmpnent". Malaysan Jurnal f Cmputer Scence, Vl.28, n.2, [18] Y. Hu, J. Huang, J. Chen, M. Lu and K. Xe, " Sftware Prject Rsk Management Mdelng wth Neural Netwrk and Supprt Vectr Machne Appraches". n hrd Internatnal Cnference n Natural Cmputatn, [19] H. Yng, C. Juhua, R. Zhenbang, M. Lu, and X. Kang, " A Neural Netwrks Apprach fr Sftware Rsk Analyss". n Sxth IEEE Internatnal Cnference n Data Mnng Wrkshps, [20] V.N. Vapnck, "he Nature f Statstcal Learnng hery", Secnd Edtn, Sprnger-Verlag New Yrk Inc, [21] S. Haykn, "Neural Netwrks: A Cmprehensve Fundatn. Secnd Edtn", Prentce-Hall Inc, [22] C.J. Burges, "A tutral n supprt vectr machnes fr pattern recgntn". Data mnng and knwledge dscvery, Vl.2, n.2, 1998, pp [23] S.-W. Ln and S.-C. Chen, "PSOLDA: A partcle swarm ptmzatn apprach fr enhancng classfcatn accuracy rate f lnear dscrmnant analyss", Appled Sft Cmputng, Vl.9, n.3, 2009, pp [24] J. Davs and M. Gadrch. "he relatnshp between Precsn- Recall and ROC curves". n Prceedngs f the 23rd nternatnal cnference n Machne learnng [25]. Fawcett, "An ntrductn t ROC analyss", Pattern recgntn letters, Vl.27, n.8, 2006, pp ISSN: e-issn: Vl. 9 N. 1 5