DETERMINING SIGNIFINT FTORS ND THEIR EFFETS ON SOFTWRE ENGINEERING PROESS QULITY R. Rdhrmnn Jeng-Nn Jung Mil to: rdhrmn_r@merer.edu jung_jn@merer.edu Shool of Engineering, Merer Universit, Mon, G 37 US strt This pper nlzes the qulit of n ongoing softwre mintenne projet using defet densit dt from prior nd urrent relese hnges. The ojetive is to test the signifine of ftors suh s developer experiene, the omplexit of the hnge, the size of the hnge (mesured in lines of ode), nd the vrious intertions ginst the defet densit of prtiulr hnge. The two phses of the hnge tht del diretl with ode nd impt qulit the most re ode design nd ode development, so the hve oth een nlzed to see if there re n signifint ftors. For the signifint ftors, regreion equtions hve een developed followed residul nlses. In the ode Design (D) phse of the softwre development, developer experiene nd ode hnge omplexit were found to e ftors tht n impt the ost nd shedule of the overll projet. For the ode Development Wlkthrough (W) phse, no ftors were found to e signifint. It is due to high vriilit in softwre development nd this m hnge with the ddition of more input dt. Sine defet densit is the item of importne in this stud, it would e helpful to rete upper nd lower ontrol limits for different tretment omintions. This would llow the projet mnger to monitor the proe performne nd t ordingl to n normlities. Kewords: Softwre engineering, ode design, ode development, proe qulit, performne mesures, signifint ftors. Introdution The use of sttistis hs long een regrded s w to mesure the performne nd qulit for ll tpes of engineering prolems [, ]. lthough most ommonl used in the mnufturing world, where there re unlimited supplies of dt, the use of sttistis hs slowl moved into the softwre engineering environment. Mn still question the use of sttistis in softwre engineering simpl euse of the nture of the work. In mn ses, proees re not repetle or one hnge is just muh smller or lrger thn the previous. The question then eomes: How do ou ompre lrge softwre hnge to smll softwre hnge nd feel omfortle with the results? The introdution of the Softwre Engineering Institute nd the MMI proe for developing softwre hs helped pioneer the use of sttistis in softwre engineering [3]. The MMI model fouses on using sttistis to mnge the performne of the projet with respet to meeting time nd udget onstrints s well s mintining the qulit of the softwre. Mnging time nd udget onstrints is not new prolem n streth of the imgintion nd is firl ommon ro ll tpes of engineering projets [4]. The ide of mnging qulit in softwre is somewht hrder to understnd for most people. For strters, how do ou define qulit? Is it the
numer of softwre defets relesed to the user, or does it inlude ll of the defets found during testing? If it is the first, how n ou ount defets if ou don t know the exist? Mn defets go undisovered for ers fter the softwre is relesed while others surfe lmost immeditel. One ommon w to mesure the qulit of the softwre is to tke qulit mesurements t predetermined milestones nd ompre the dt to onfidene intervls to p judgment on the qulit of the softwre [5]. In this pper, the qulit of n ongoing softwre mintenne projet is nlzed using defet densit dt from prior nd urrent relese hnges [6]. The ojetive is to test the signifine of ftors suh s developer experiene, the omplexit of the hnge, the size of the hnge (mesured in lines of ode), nd the vrious intertions ginst the defet densit of prtiulr hnge. The two phses of the hnge tht del diretl with ode nd impt qulit the most re ode design nd ode development, so the will oth e nlzed to see if there re n signifint ftors [7, 8]. For the ftors tht were found to e signifint, regreion equtions were developed followed residul nlses. Methodolog The lrgest prolem tht projet mnger fes when deling with softwre is how to mnge qulit [9]. Poor qulit often leds to shedule nd ost overruns tht n jeoprdize future worklod. Of the mn ftors tht go into softwre development, the most prominent ftors re experiene of the emploee (), omplexit of the hnge (), nd the lines of ode in the hnge (). sed on these three ftors, the projet mnger should e le to predit whether prtiulr hnge will enounter qulit iues in the future. prediting when there re going to e qulit iues, the projet mnger n e protive putting more experiened developer on the hnge or even split up the hnge to redue the size nd omplexit []. Sine there re three ftors, the est w to ddre the signifine or lk thereof is to ondut three ftor fixed effet experiment []. lthough more replites re etter, the dt set does meet the minimum requirement of t lest two replites without whih the error sum of squres, whih is n importnt prt of the nlsis, ould not e omputed. Using the populr dot nottion, the totl sum of squres nd the sum of squres for ftors,, nd re omputed from the following equtions: T i j k n l ijkl, () i j k i..., () n. j.., (3) n.. k.. (4) n
To ompute the two-ftor intertion sum of squres, the totls for the x, x, nd x re needed; the sum of squres for the two-ftor intertions re: i j ij.. n, (5) i k i. k. n, (6) j k. jk. n, (7) The three-ftor intertion is omputed from the three-w ell totls using: i j k ijk. n, (8) The error sum of squres is simpl the totl sum of squres minus the sum of squres for eh effet nd intertion. E SS. (9) T sutotls( ) fter omputing the sum of squres, the entire nlsis of vrine (NOV) tle n e ompleted. uming n lph vlue of, the signifine of ftor is verified using the f-test []. Further nlsis is needed on the dt set to ensure tht there re no violtions of si umptions tht ould invlidte the results. So the residul vlues of the experiment need to e lulted. In order to lulte the residuls, the min effets s well s the two-ftor nd the three-ftor intertion effets need to e estimted. The min effets re estimted using: [ + + + ()], () [ + + + ()], () [ + + + ()]. () The two-ftor intertion effets re estimted using: [ + () + + ], (3) [ + () + + ], (4)
[ + () + + ]. (5) The three-ftor intertion effet is estimted tking the verge differene etween the intertion t the two levels of or: [ + + + ()]. (6) The effet estimtes n then e used to develop regreion model to lulte the residuls of the experiment. The fitted regreion model is: where: 3 β + β x + β x + β x +... (7)... β ; β ; β ; β 3. (8) The remining oeffiients n e found in similr fshion. The tul regreion model will onl onsist of the oeffiients tht orrespond to the ftors deemed signifint. The residuls n then e lulted using the regreion model nd evluting the oserved vlues t eh tretment omintion. The results n then e plotted on norml proilit plot to illustrte n normlities []. Results nd Disuions The smples tken over period of three ers from softwre development projet t the 58 th Softwre Mintenne Squdron t Roins ir Fore se, Georgi, during ode Design (D) nd ode Development Wlkthrough (W) phses re shown in Tles nd respetivel. The tles outline different ode hnges ordered the experiene level of the developer who mde the hnge, the omplexit level of the hnge (tehnil diffiult), nd the size of the hnge (mesured in lines of ode). For the dt nlsis, the Minit sttistil nlsis softwre pkge ws used to perform NOV, regreion nlsis, nd omputtion of norml plots nd residuls []. Tle 3 shows the NOV for the D phse. The P-vlues indite Developer Experiene nd ode omplexit re signifint ftors (ounting P-vlue of le thn s signifint). The D phse R vlue is 7.43%. The norml proilit plot is shown in Figure. Figure shows the residuls plot. Tle. D phse defet densit 3
Tle. W phse defet densit Tle 3. D phse NOV Norml Proilit Plot (response is D) Versus Fits (response is D) 99 95 9 8 Perent 7 6 5 4 3-5 -.5 - -.5 -.5.5 Fitted Vlue.3.35.4 Figure. Norml proilit plot Figure. s plot Tle 4 shows the NOV for the W phse nd it does not indite n signifint ftors. In ddition, the W phse R vlue is poor (39.9%). The norml proilit plot nd the residuls plot re shown in Figures 3 nd 4 respetivel. Tle 4. W phse NOV
Norml Proilit Plot (response is W) Versus Fits (response is W) 99.3 95 9. Perent 8 7 6 5 4 3. -. 5 -. -.3 -. -....3 -.3.5 Fitted Vlue.5 Figure 3. Norml proilit plot Figure 4. s plot Tle 4 shows the NOV for the regreion nlsis during D phse onsidering developer experiene, R omplexit, nd R size s ftors. Tle 4. D phse NOV - Regreion nlsis The fitted regreion eqution is: D.63 + 3 Developer Exp. + R omplexit - 49 R Size (9) The Minit regreion nlsis of the D phse shows firl poor R vlue of 54.6%. This is due to Minit s ehvior of ounting ll ftors, even non-signifint ones, in the regreion nlsis. When the non-signifint ftor, R Size, is exluded from the nlsis, muh etter R vlue of 9% is found. Figure 5 shows the norml proilit plot. The residuls plot is shown in Figure 6. Norml Proilit Plot (response is D) Versus Fits (response is D) 99.5 95 9 Perent 8 7 6 5 4 3-5 - -. -....5 Fitted Vlue.5.3.35 Figure 5. Norml proilit plot Figure 6. s plot
When onl signifint ftors were inluded to fit stright line to the D phse dt, the R vlue of the signifint ftors (developer experiene nd R omplexit) is found to e 9.5% (Figure 7). This mens tht these two ftors must e n importnt prt of the predition model for ode Design. When plotted s n exponentil model, the R vlue is even higher, 95.44%, with one outlier (Figure 8). The residuls plot lso shows different pttern when onl signifint ftor vlues re used (Figure 9). Norml Sore Norml Proilit Plot 3.449x +.5 R.95..8.6.4. -.-.5 -.5 -.5.5 -.4 s Figure 7. Norml proilit plot Norml Sore Norml Proilit Plot.3759e.95x R.9544.8.6.4..8.6.4. -.5 -.5 -.5.5 s Figure 8. Norml proilit plot s vs Fitted s..5. -..5..5.3.35 -. -.5 -. -.5 Fitted Figure 9. s plot Tle 5 shows the NOV for the regreion nlsis during W phse onsidering developer experiene, R omplexit, nd R size s ftors. Tle 5. W phse NOV - Regreion nlsis The fitted regreion eqution is: W 785-37 Developer Exp. - 64 R omplexit - 496 R Size () The Minit regreion nlsis of the W phse shows poor R vlue of 8.6% nd P-vlue of.4. From the regreion nlsis, it is ler tht n useful predition model from the
olleted dt nnot e derived. The norml proilit plot is shown in Figure. Figure shows the residuls plot. Norml Proilit Plot (response is W) Versus Fits (response is W) 99.3 95 Perent 9 8 7 6 5 4 3 5.. -. -.3 -. -....3 -. Fitted Vlue.5 Figure. Norml proilit plot Figure. s plot onlusions This stud hs shown tht in the ode Design phse of the softwre development projet, developer experiene nd ode hnge omplexit should e onsidered s ftors tht n impt the ost nd shedule of the overll projet. For the ode Wlkthrough phse, no ftors s reorded n e onsidered signifint. lthough this m hnge with the ddition of more input dt, this result is not unexpeted due to the high vriilit of softwre development. The results of this stud n e used in the rel world in the res of Quntittive Projet Mngement nd Orgniztionl Proe Performne, whih involve the use of sttistis to mesure performne nd mke deisions [9]. From these results, the softwre tem will e le to etter predit res of onern in future development les nd determine w to est hndle them. This will help the tem mintin the gol of produing qulit produt while sting within time nd udget onstrints. This stud lso leves plent of room for future stud in determining onfidene nd predition intervls. Sine defet densit is the item of importne in this stud, it would e helpful to rete n upper nd lower ontrol limits for the different tretment omintions. This would llow the projet mnger to monitor the proe performne nd t ordingl to n normlities. Referenes [] Hines, W. W., Montgomer, D.., Goldsmn, D. M., nd orror. M., Proilit nd Sttistis in Engineering, 4 th Edition, John Wile & Sons, 3. [] Lewis, E.E.; Introdution to Reliilit Engineering, [3] rnegie Mellon, MMI Model V., Softwre Engineering Institute, 6. nd Edition, Wile & Sons, 996. [4] Jeffre, R.., Proilit nd the rt of Judgment, mridge Universit Pre, 99. [5] 58 SMXS Tehnil Stff, 58 th SMXS Squdron Defined Softwre Proe, 58 SMXS, ugust 4. [6] 58 SMXS Tehnil Stff, Softwre Qulit urne Pln V., 58 SMXS, Ferur 7. [7] 58 SMXS Flight D Tehnil Stff, M-3H Softwre Proedure Mnul V., 58 SMXS, Jnur 7.
[8] 58 SMXS Flight D Tehnil Stff, Softwre Development Pln for the M-3H omt Tlon II Opertionl Flight Progrm V3., 58 SMXS, Jnur 7. [9] 58 SMXS Flight D Tehnil Stff, Quntittive Projet Mngement Metri Inditors for the omt Tlon II, 58 SMXS, Deemer 6. [] 58 SMXS Flight D Tehnil Stff, M-3H omt Tlon II hnge Request omplexit Rnking Proedure V., 58 SMXS, pril 6.