Curren Trends in Technology and Science ISSN : 79-055 8hSASTech 04 Symposium on Advances in Science & Technology-Commission-IV Mashhad, Iran A New for Sofware Reliabiliy Evaluaion Based on NHPP wih Imperfec Debugging Shiva Akharian MSc Suden, Deparmen of Compuer Engineering and Informaion Technology, Payame Noor Universiy, Iran sh.akharian@gmail.com TahereYaghoobi Assisan Professor, Deparmen of Compuer Engineering and Informaion Technology Payame Noor Universiy, Iran.yaghoobi@pnu.ac.ir Absrac In new sofware producs, he necessiy of high reliabiliy sofware developmen is increasingly imporan o all sofware developers and cusomers. Sofware reliabiliy modeling based on Non Homogenous Poisson Process (NHPP) is one of he successful mehods in predicing sofware reliabiliy. One of he mos imporan assumpions in modeling sofware reliabiliy is imperfec phenomenon which describes he probabiliy of inroducing new fauls o he sofware during debugging process. In his paper a new imperfec model is proposed by considering complexiy and dependency beween fauls. Esimaing he model parameers has been done by using failure daa ses of four real sofware projecs hrough MATLAB sofware. Comparison of he proposed model is done wih wo exising models using various crieria. The resuls show ha he proposed model beer fis he real daa and hen providing more accurae informaion abou he sofware qualiy. Keywords Sofware Reliabiliy, Non Homogenous Poisson Process, simple and complex fauls, dependency, ime delay, imperfec debugging. INTRODUCTION An imporan approach in evaluaing sofware qualiy and performance is o deermine he sofware reliabiliy. Sofware reliabiliy defined as he probabiliy of error free sofware operaion in a specified environmen for a specified ime (Yamada, 04). In he las hree decades various sofware reliabiliy growh models (SRGMs) have been developed. Mos of he SRGMs based on NHPP presen he reliabiliy diagram as concave or S- shaped curve (Goel, 985; Pham, 006; Kapur, e al., 0; Lai, e al., 0). If he reliabiliy is uniformly growh, concave models are used oherwise i would be S-shaped. In 985 Goel and Okumoo proposed a concave model which assumed ha oal fauls in he sofware are independen and probabiliy of deecing fauls in a uni of ime is consan (Goel, 985). They also assumed deeced fauls are immediaely removed and no new fauls are inroduced during debugging (perfec debugging). In an S-shaped model deeced fauls are immediaely removed and perfec debugging is used bu faul deecion rae is ime dependen (Yamada, e al., 98; Yamada, e al., 984; Yamada, e al., 985). In los of recen researches imperfec debugging is considered in modeling, which shows he probabiliy of inroducing new fauls o he sofware during faul correcion and removal (Yamada, e al., 99; Pham, 006; Pham, 007; Gaggarwal, e al., 0; Gupa, e al., 0). In some of developed models deeced fauls are caegorized according o heir dependency or complexiy. Ohba proposed he idea of faul dependency in 984 by assuming ha independen fauls can be removed immediaely afer hey are deeced bu dependen fauls canno be removed unil heir leading fauls are eliminaed (Ohba, 984). Kapour and Younes in 995 affeced faul dependency ino G-O model (Kapur, e al., 995). They noiced ha ime delay effec facor beween deecing and removing dependen fauls is negligible. In sudies by Hung and Lin dependen and independen fauls are modeled based on NHPP wih some various imes dependen delay funcions (Huang, e al., 004; Huang, e al., 006). Karee and his colleagues proposed an S-shaped model wih faul complexiy by caegorizing fauls as simple and hard. In heir model simple fauls can be removed immediaely bu hey affeced wo seps of observing and removing o eliminae hard fauls (Kareer, e al., 990). Kapur In 999 developed a model by considering sofware esing as a hree sep process, including observing, isolaing and removing fauls. Based on his assumpion, he proposed a model wih hree ypes of fauls caegorized based on heir complexiy. In which simple fauls can be removed immediaely afer hey are deeced, hard fauls are hose ha need a ime delay facor beween observing and isolaing he faul and complex fauls need a ime delay effec facor beween observing, isolaing and removing he faul (Kapur, e al., 999). In his paper an imperfec model based on boh complexiy and dependency of fauls is developed. Sofware fauls are caegorized in erms of heir complexiy ino wo ypes of simple and complex, and complex fauls are classified ino independen and dependen. Then, hree differen ypes of fauls are Sofware Reliabiliy Growh Copyrigh 04 CTTS.IN, All righ reserved 6
proposed and for each ype, an appropriae modeling is presened based on some assumpions. In sofware reliabiliy modeling here are unknown parameers ha can be deermined by Maximum Likelihood Esimaion (MLE) or leas squared esimaion mehods (Zacks, 99; Pham, 006). The unknown parameers of he proposed model are deermined by using four real sofware faul daa ses by MLE. Then some of saisical crieria are used o compare he goodness of fi of he new model and wo basic models. The esimaed ime o sop esing for sofware is evaluaed by he proposed model. The remainder of paper is as follows: In Secion, a generic NHPP sofware reliabiliy model is explained. Secion is dedicaed o one applicaion of sofware reliabiliy esimaion, esimaing ime o sop sofware esing. In Secion 4, he parameers, assumpions and our modeling is described in deails. The nex wo secions, 5 and 6are specified o model esimaion, and model validaion respecively. The paper is concluded in Secion 7.. SOFTWARE RELIABILITY MODELING BASED ON NHPP The basic assumpion in NHPP modeling is ha he failure process is described by he NHPP. Mos of sofware reliabiliy growh models follow he general assumpions regarding he faul deecion process which can be considered as follows. Deecing / removing fauls are modeled by NHPP. All fauls are independen from each oher and hey are equally recognizable. The number of fauls deeced a any ime is proper wih he number of remaining fauls in he sofware. Each ime a failure occurs an error ha led o i is immediaely and compleely removed and no new errors inroduced in he meanime (perfec debugging). Assume ha N() is he number of deeced errors (failures observed) a ime and m() is he mean number of deeced fauls unil ime, hen is Min Value Funcion (MVF) is: m m Pr N n e n 0,,, Where 0 n! n () m s ds and is failure inensiy funcion (number of failures per uni of ime). λ () Therefore, he overall reliabiliy model based on NHPP is obained by solving he following differenial equaion wih he iniial condiion m (0) =0: λ b. a m () In which, "a" represens he oal number of iniial fauls in he sofware before esing. Expression [a m()] shows Copyrigh 04 CTTS.IN, All righ reserved 6 Curren Trends in Technology and Science ISSN : 79-055 8hSASTech 04 Symposium on Advances in Science & Technology-Commission-IV Mashhad, Iran he mean number of remaining fauls in he sofware a ime, and b() is faul deecion rae. General soluion for he differenial equaion is as follows: β β m e [C ab e ] (4) In which: β b And C is he consan of inegraion. Finally he sofware reliabiliy is achieved hrough he following formula: λ s ds 0 R e (5) Table [] shows deails of wo basic models for sofware reliabiliy, Goel - Okumoo exponenial model and delayed S-shaped model. Table. Deails of exponenial and delayed s-shape models name Mean value funcion Faul deecion rae Delayed S- shape b a e b a b e b b b b. ESTIMATING TIME TO STOP SOFTWARE TESTING Sofware esing is a cosly process, so i's necessary o erminae i in a suiable ime. One of he applicaions of esimaing sofware reliabiliy is o deermine he appropriae ime o erminae sofware esing.if he sofware reached o an accepable level of reliabiliy a he specified ime, hen decision o erminaing he es can be aken. To achieve his goal, he condiional reliabiliy funcion, R x is used. This funcion assumes ha he las failure has occurred a ime and no new failure happens a ime inerval, x Therefore (Pham, 006): m x m R x Pr N x N 0 e (6) 4.THE PROPOSED MODEL In his secion, firs he model parameers and assumpions are presened and hen, according o he provided assumpions, he modeling is carried ou. 4.. PARAMETERS OF THE PROPOSED MODEL a : oal number of iniial fauls in he sofware. a, a, a : oal number of iniial simple, independen complex, dependen complex fauls. b, b, b : faul deecion rae for Simple, independen complex, dependen complex fauls α, α, α : faul inroducion rae for simple, independen complex, dependen complex fauls. proporion of simple and complex errors in sofware
Curren Trends in Technology and Science ISSN : 79-055 8hSASTech 04 Symposium on Advances in Science & Technology-Commission-IV Mashhad, Iran : proporion of independen complex and dependen complex. φ φ : ime lag for complex independen fauls. : ime lag for complex dependen fauls. m : mean value funcion for oal sofware fauls. m : mean value funcion for simple fauls. m φ mean value funcion for independen complex fauls. m φ : mean value funcion for he dependen complex fauls. 4.. ASSUMPTIONS OFTHE PROPOSED MODEL Our proposed model assumpions are as follows: i. Deecing / removing fauls is modelled by NHPP. ii. Sofware fauls are divided ino simple and complex fauls. This division was done by considering he amoun of effor needed o remove hem. iii. Simple fauls can be correced and deleed immediaely afer deecion. iv. Complex fauls may need more effor o be removed. This kind of faul is considered in dependen and independen. v. Independen complex fauls need more imes han simple fauls o be removed. Therefore he ime delay facor φ is affeced o show he amoun of effors is needed for his ype of fauls. vi. Dependen complex fauls arise due o he presence of an independen complex faul. This kind of faul canno be removed unless is leading faul is eliminaed. According o he effor required in removing his ype of faul, ime delay facor is no negligible, herefore he laency φ will be affeced vii. During he debugging process, inroducing new fauls is possible o show he imperfec phenomenon. 4.. MODELING The oal number of deeced fauls during he ime (0, ) is equal o m = m + m - φ + m - φ (7) Le a be he oal number of iniial fauls before esing and a, a, a describe oal number of iniial simple, independen complex and dependen complex fauls respecively. So we have: a = pa a = - p qa a = - p - q a Deecion and removal process of simple fauls follows he general reliabiliy modeling (Equaion ) wih consan faul deecion rae. Of course ime lag o remove fauls is negligible and imperfec debugging is affeced by new fauls inroduced o sofware during debugging as shown in equaion (8): = b a + α m - m (8) By solving he differenial equaion (8), he mean value funcion of he deeced fauls is creaed as follows a -b -α pa -b -α m = - e = - e -α - α (9) Sofware debugging eam requires more ime o discover he causes of complex fauls o remove hem. For his ype of fauls general model of reliabiliy can be used, however i is necessary o affec ime delay facor and imperfec debugging in removal process. Assuming ha he mean value funcion of he independen complex fauls is proporional o he remaining number of independen complex fauls and new fauls inroduced o he sysem, so: = b a + α m - m (0) By solving he differenial equaion (0), in iniial condiion m 0 0 will have: a -b -α m = - e -α - p qa = - e -α -b -α () Time delay facor in debugging of independen complex fauls is considered as ascending funcion φ. Thus he finalequaion () is obained: φ = Ln + b b - p qa m - φ = - + b e -α -α -b -α () Based on assumpion v, he general modeling of reliabiliy is no direcly applicable on removing dependen complex fauls. The mean value of dependen complex fauls is proporional o he number of remaining dependen complex fauls, he mean value of independen complex fauls and new inroduced fauls o he sysem. Also, ime delay facor φ and imperfec debugging are affeced o remove his kind of fauls. So he differenial equaion for modeling hese fauls is as follows: m - φ = b a + α m - m () a + a By solving equaion (), we obain a m = (4) -α ab exp ( a a b s 0 s bs b s e ) ds Considering he ime required for debugging and removing complex dependen fauls, ime delay facor Copyrigh 04 CTTS.IN, All righ reserved 64
Curren Trends in Technology and Science ISSN : 79-055 8hSASTech 04 Symposium on Advances in Science & Technology-Commission-IV Mashhad, Iran φ should be affeced in Equaion (4) for he final ime 7 o 5 o esimae he sofware esing ime erminaion. equaion of complex dependen fauls. So we have: Ln + b φ = b m - φ = (5) a a b s bs exp ( b s e ds a a 0 b s According o equaion (7), he mean value funcion of oal sofware s fauls is obained by calculaing all hree mean value funcions. 5. MODEL ESTIMATION The proposed model is esed on four real sofware failure daa ses, real-ime command and conrol sysem (Ohba, 984a), IBM (Musa, e al., 987), OCS (Pham, e al., 00) and Misra (Misra, 98) and is parameers are esimaed by Malab0 sofware. Table () presens esimaed parameers for he proposed model, exponenial model and delayed S-shaped model. name New model G-O model Table.Parameer esimaion based on failure daases Misra a 78 b 0 / 69 b 0 / 6 b 0 / 09 s 0 / 06 q 0 / 0 / 06 0 / 54 0 / 99 a 9 b 0 / 007 OCS a 80 0 / 8 0 / 9 b 0 / 000 s 0 / 7 q 0 / 4 0 / 000 0 / 000 0 /00 a 4 0 / 06 b IBM a 7 0 / 0 0 / 09 0 / 99 s 0 / 4 q 0 / 56 0 / 000 0 /00 0 /00 a 400 0 / 005 b Real ime a 50 0 / 5 0 / b 0 / 000 s 0 / 4 q 0 / 94 0 / 0 0 / 000 0 / 85 a 4 0 / b Table Obained informaion afer each es of real-ime command and conrol sysem o deermine he erminaing ime of esing process using proposed model Tesing ime () Toal number of esimaed iniial fauls 4 / 8 95 / 860 49 / 6 9 / 759 / 99 89 / 805 5 / 90 6 / 68 49 / 949 Value of Esimaed Mean value funcion / 78 6 / 69 8 / 040 8 / 998 / 9 / 995 4 / 00 4 / 997 5 / 999 0 / 70 0 / 669 0 / 79 0 / 879 0 / 77 0 / 86 0 / 84 0 / 864 0 / 88 6. MODEL VALIDATION To show he validaion of he proposed model assumpions, we compare i wih exponenial model and delayed S-shaped model which are considered as basic models in predicing sofware reliabiliy, based on NHPP. Figures () o (4) shows he proposed model's goodness of fi compared wih he exponenial model and delay S- shaped model on all four daa ses menioned in he las secion. Figure.proposed model's goodness of fi compared wih, Exponenial model and delay S- shaped model on real ime command and conrol daase Delayed S- shaped model a 85 0 / 08 b a 5 0 / b a 7 0 / b a 6 0 / 8 b Now using obained parameers, mean value funcion and sofware reliabiliy can be esimaed in a ime inerval, 0.. Thus, he ime erminaion of sofware esing is specified. Table () presens he informaion of sofware esing process obained by using proposed model on real-ime command and conrol sysem from Copyrigh 04 CTTS.IN, All righ reserved 65 Figure.proposed model's goodness of fi compared wih, exponenial model and delay S- shaped model on IBM daase
Figure.proposed model's goodness of fi compared wih, Exponenial model and delay S- shaped model on OCS daase Curren Trends in Technology and Science ISSN : 79-055 8hSASTech 04 Symposium on Advances in Science & Technology-Commission-IV Mashhad, Iran Table.values of crieria for proposed model, exponenial model and delayed S-shaped model Misra daa se OCS daa IBM daa name se se New G-O Delayed S-shaped 05 / 0 / 05 707 / 95 8 / 6 00 / 75 / 06 / 657e 0 4 / 60 79 / 74 05 / 94 8 / 667e 0 96 8 / 84 0 / 46 79 / 04 9 / 44 7 / 07 0 / 0 9 / 88 76 / 8 / 6 / 9 74 / 74 6 / 8 8 / 66 0 / 49 9 / 76 / 4 77 / 7 / 04 70 / 8 8 / 78 / 04 / / 88 / 6 Real-ime Command and Conrol daa se 7 / 04 0 / 0 0 / 5 5 / 0 8 / 96 0 / 64 90 / 7 6 / 0 / 74 8 / 5 4 / 79e 0 MS 68 / 7 According o he resuls of Table 4,i can be seen hahe proposed model has a good finess on menioned daases and works beer han he exponenial model and delayed S-shaped model. Figure 4.proposed model's goodness of fi compared wih, Exponenial model and delay S- shaped model on Misra daase I can be seen from figures, he proposed model is properly consisen on daa ses. For accurae model comparison, oher crieria can be used. Three common ones are (Pham, e al., 00), (Pham, 006) and (Pham, 006), which can be expressed as follows: "y" is oal number of fauls observed a ime "" according o real daase and "m()" is esimaed cumulaive number of fauls a ime "" and "n" is he number of observaions. In model comparison, smaller values for each of hree crieria on he same daase, represens more appropriae model. Table (4) shows he obained values of crieria for proposed model, exponenial model and delayed S- shaped model on all four daases. Predicion Raio Risk Sum of Squared Errors Mean Squared Errors Copyrigh 04 CTTS.IN, All righ reserved 66 7. CONCLUSION Sofware reliabiliy growh models can be used o deermine sofware performance and conrol he esing process of sofware. Based on hese models, reliabiliy is esimaed in quaniaive mehods. Some measuremens such as number of iniial fauls, failure inensiy, sofware reliabiliy in a specified period of imeand mean ime beween failures can be deermined by using SRGMs. By now, various models have been developed by researchers based on sofware esing condiions. New sofware reliabiliy model proposed in his paper, caegorizes sofware fauls ino simple, independen complex and dependen complex. The model is also affeced by imperfec debugging and ime delay facor in complex fauls removal modelling. By considering new assumpions more realisic model proposed, which considers boh complexiy and dependency of fauls in is faul removal modelling. Imperfec debugging and ime delay facor in removing complex fauls are wo oher imporan assumpions in new model. So by making his model more closer o real sofware condiions i can esimae sofware reliabiliy more accurae han compared basic models. REFERENCE [] S. Yamada, Sofware reliabiliy modeling: Fundamenals and Applicaions.: Springer, 04. [] A. L. Goel, "Sofware Reliabiliy s:assumpions, Limiaions and applicabiliy," IEEE Trans.on sofware engineering, vol., pp. 4-4, 985. [] H. Pham, Sysem Sofware Reliabiliy. London: Springer-Verlag, 006. [4] P. K. Kapur, H. Pham, A. Gupa, and P. C. Jha, Sofware Reliabiliy Assessmen wih or Applicaions. London: Springer-Verlag, 0.
[5] R. Lai and M. Garg, "A Deailed Sudy of NHPP Sofware Reliabiliy s," Journal of Sofware, vol. 7, pp. 96-06, 0. [6] S. Yamada, M. Ohba, and S. Osaki, "S-shaped Reliabiliy Growh ing for Sofware Error Deecion," IEEE Trans on Reliabiliy, vol., no. 5, pp. 475-484, 98. [7] S. Yamada, M. Ohba, and S. Osaki, "S-shaped Sofware Reliabiliy Growh s and heir Applicaions," IEEE Trans.Reliabiliy, vol. R-, pp. 89-9, 984. [8] S. Yamada and S. Osaki, "Sofware reliabiliy growh modeling: models and applicaions," IEEE Trans Sofw Eng, pp. 4-47, 985. [9] S. Yamada, K. Tokunou, and S. Osaki, "S- Imperfec debugging models wih faul inroducion rae for sofware reliabiliy assessmen," Inernaional Journal of Science, vol., no., pp. 4-5, 99. [0] H. Pham, "An imperfec-debugging faul deecion dependen parameer sofware," Inernaional Journal of Auomaion and compuing, pp. 5-8, 007. [] A. Gaggarwal, P. K. Kapur, and A. S. Garmabaki, "Imperfec Debugging sofware reliabiliy growh model for muliple release," in 5h naional conference, India, 0. [] A. Gupa and S. Saxena, "Sofware Reliabiliy Esimaion Using Yamada delayed S Sape under Imperfec Debugging and Time Lag," in Inernaional Coference on Advanced Compuing & Communicaion echnologies, vol. 5, 0, pp. 7-9. [] M. Ohba, "Sofware Reliabiliy Analysis s," IBM Journal of Research and Developmen, vol. 8, pp. 48-44, 984. [4] P. K. Kapur and S. Younes, "Sofware Reliabiliy Growh wih error dependency," Elsevier Science, vol. 5, pp. 7-78, 995. [5] C. Y. Huang, C. T. Lin, J. H. Lo, and C. C. Sue, "Effec of Faul Dependency and Debugging Time Lag on Sofware Error s," IEEE Trans on Reliabiliy, pp. 4-46, 004. [6] C. Y. Huang and C. T. Lin, "Sofware Reliabiliy Analysis by Considering Faul Dependency and Debugging ime Lag," IEEE Trans on Reliabiliy, vol. 55, pp. 46-450, 006. [7] N. Kareer, P. K. Kapur, and P. S. Grover, "An S- shaped sofware reliabiliy growh model wih wo ypes of errors," Microelecron Reliab, pp. 085-090, 990. [8] P. K. Kapur, R. B. Garg, and S. Kumar, Conribuions o hardware and sofware reliabiliy. Singapore: Word Scienific, 999. [9] S. Zacks, Inroducion o Reliabiliy Analysis.: Springer-Verlag, 99. [0] M Ohba, Sofware reliabiliy analysis models.: IBM Developmen, 984a. [] J D Musa, A Lannino, and K Okumoo, Sofware Reliabiliy: Measuremen, predicion, and applicaion. New York: Mac Graw-Hill, 987. Copyrigh 04 CTTS.IN, All righ reserved 67 Curren Trends in Technology and Science ISSN : 79-055 8hSASTech 04 Symposium on Advances in Science & Technology-Commission-IV Mashhad, Iran [] H Pham and C Deng, "Predicive raio risk crierion for selecing sofware reliabiliy models," in Ninh Inernaional Conf on reliabiliy and qualiy in design, 00. [] P. N. Misra, "Sofware reliabiliy analysis," IBM Sysem Journal, pp. 6-70, 98. [4] H. Pham and C. Deng, "Predicive raio risk crierion for selecing sofware reliabiliy models," in Ninh Inernaional Conf on reliabiliy and qualiy in design, 00.