Effect of Test Coverage and Change Point on Software Reliability Growth Based on Time Variable Fault Detection Probability

Transcription

1 Effec of Tes Coverage ad Chage Poi o Sofware Reliabiliy Growh Based o Time Variable Faul Deecio Probabiliy Subhashis Chaerjee*, Akur Shukla Deparme of Applied Mahemaics, Idia School of Mies, Dhabad, Jharkhad, Idia. * Correspodig auhor. Tel.: ; chaerjee_subhashis@rediffmail.com Mauscrip submied July 31, 15; acceped November, 15. doi: /jsw Absrac: I pas four decades, may sofware reliabiliy growh models (SRGMs) have bee proposed o ehace he reliabiliy of he sofware sysem. Durig he esig process poeial faul sies are sesiized o deec he fauls. Faul deecio probabiliy icreases as learig ad mauriy of he esig persoel icreases. Therefore, i his paper a ime varia faul deecio probabiliy has bee iroduced ad iegraed io s-shaped coverage SRGM. Experimeal resuls shows ha he proposed model is beer ad will be helpful o improve he accuracy of he sofware reliabiliy esimaio. Key words: expoeial faul deecio probabiliy, o-homogeeous Poisso process (NHPP), sofware reliabiliy growh model, sofware reliabiliy, s-shaped esig coverage. 1. Iroducio Nowadays, he depedabiliy of he moder sociey o he sofware sysems is icreasig rapidly. Therefore, demad of qualiy sofware becomes he mos challegig ask for sofware professioals ad researchers. Sofware esig is oe of he major sofware developme aciviy, which helps i improvig he qualiy. The mai objecive of he sofware esig is o ucover ad removed he ihere ad iroduced fauls wih miimum expediure [1]. Sofware reliabiliy is he esseial par of a qualiy sofware, so i is ecessary o ehace he reliabiliy of he sofware. I pas four decades, differe echiques ad mehodologies have bee developed o improve he qualiy ad reliabiliy of sofware sysem, sill i is a major cocer o esimae he remaiig sofware fauls prese i a sofware sysem accuraely. Accordig o Musa [], Sofware reliabiliy is defied as he failure-free operaio of a sofware uder specified evirome ad specified ime. May effors have bee made by researchers o esimae ad predic he reliabiliy of sofware hrough sofware reliabiliy modelig. I pas, various SRGMs have bee developed wih differe approaches ad assumpios [3]-[16]. Tesig coverage is he oe of he impora facor which affecs he reliabiliy growh of sofware. I helps i developig more efficie es cases. The mai objecive of he esig coverage aalysis is o fid he faul sies which are ucovered ad addiioal es cases required o icrease he esig coverage. Durig he esig process, o deec he fauls prese a he faul sies, poeial faul sies are sesiized [1], [17]. Oe commo assumpio of he mos of he SRGM ha he faul prese a he poeial faul sies are deeced wih cosa probabiliy [1], [8]. Which is o realisic ad reasoable. I realiy, faul deecio probabiliy icreases as he learig ad mauriy of he eser icreases. Therefore, i he 11 Volume 11, Number 1, Jauary 16

2 prese sudy faul deecio probabiliy has bee cosidered as expoeial fucio of ime. Moreover, differe sudies have bee doe cosiderig he various behavior of he esig coverage such as expoeial, Weibull, s-shaped ec. [1]. Gokhale e al. [17] cocluded ha he selecio of he coverage fucio depeds o he crieria which fulfills he user s specific requiremes. The proposed sudy has bee carried ou wih he s-shaped coverage facor. Tesig coverage facor ad faul deecio probabiliy ca be affeced by differe facors such as esig evirome, esig sraegy, esig eam cosiuio ad efficiecy, es case effeciveess, resources, ec. Chages i hese eviromeal facors causes chage i esig coverage ad faul deecio probabiliy. These pois a which chages are possible is kow as chage poi. The cocep of chage poi firs iroduced by Zhao [18]. Various SRGMs based o he cocep of chage poi have bee proposed by researchers i pas few years [19]-[]. I his paper, a NHPP based SRGM has bee developed cosiderig esig coverage. Also, ime varia faul deecio probabiliy fucio has bee iroduced ad iegraed io he proposed SRGM. To sudy he effec of he differe evirome facor o esig coverage facor ad faul deecio probabiliy, he cocep of he chage poi has bee icorporaed. Res of he aricle is orgaized as follows: relaed work has bee preseed i Secio. The assumpio ad formulaio of he proposed model has bee preseed Secio 3. Secio 4 coais parameer esimaio of proposed model ad differe compariso crieria. Model validaio ad performace aalysis has bee carried ou i Secio 5. Fially, secio 6 cocludes he work.. Relaed Work Applicaios of esig coverage has bee cosidered by may researches i sofware reliabiliy growh modelig [1], [3]. Iiially, Gokhale e al. [17] proposed he NHPP based SRGM wih esig coverage which is based o he followig assumpios: 1) Sofware fauls are disribued uiformly over poeial faul sies. ) Whe a poeial faul sie is sesiized, faul prese a ha sie is deeced wih cosa probabiliy. 3) The deeced fauls are repaired isaaeously wihou iroducig ew fauls. They [17] have cosidered hree ypes of faul coverage fucios: expoeial, Weibull ad s-shaped, ad cocluded ha he selecio of esig coverage fucio will deped o crieria which fulfils he user s requireme. Laer, may SRGMs have bee proposed based o he assumpios of Gokhale SRGM [1], [8]. 3. Sofware Reliabiliy Modelig I his secio assumpios ad formulaio of he proposed model wih chage poi ad wihou chage poi has bee preseed Proposed SRGM The proposed model is based o he followig assumpios: 1) Fauls are uiformly disribued over all poeial faul sies. ) Sofware failures follows a NHPP. 3) Whe a poeial faul sie is sesiized a ime, ay faul prese a ha sie is deeced wih probabiliy d( ) which icreases as learig process icreases. 4) The esig coverage fucio is a s-shaped esig coverage fucio, i.e., c( ) 1 (1 b )exp( b ) where b is he parameers reflecig he qualiy of esig. 5) The deeced fauls are removed immediaely wihou iroducig ay fauls. From he above assumpios mea value fucio (MVF) of he proposed model ca be obaied by he 111 Volume 11, Number 1, Jauary 16

3 followig expressio: dm( ) dc( ) ad( ) d d (1) or m( ) a d( s )c ' ( s )ds () where a is he umber of fauls which are expeced o be deeced, m(), d() ad c(). Sice, d( ) icreases as learig process icreases herefore, i will be a icreasig fucio of ime. Le d( ) 1 exp( ) where is he posiive shape parameer. Solvig he equaio () MVF ca be obaied as: b b b m( ) a 1 e b (1 b ) e ( b ) (b ) (b ) b (3) The failure iesiy fucio ( ) ca be obaied by differeiaig m( ) w.r.., i.e., ( ) dm( ) d (4) The codiioal reliabiliy of he proposed SRGM ca be obaied usig he followig equaio (5) R( x ) e [m( x )-m( )] 3.. Proposed Model wih Chage Poi Due o chage i esig sraegy, esig evirome, esig effor, defec desiy, ec., chage pois ca occur i esig coverage facor ad faul deecio probabiliy. Therefore, coverage facor ad he faul deecio ca be defied wih muliple chage poi as follows: 1 exp( 1 ), 1 1 exp( ), 1 d( ) : 1 exp( ), (6) where i is he chage poi ad i is he shape parameer for i 1,... 1 (1 b1 )exp( b1 ), 1 1 (1 b )exp( b ), 1 c( ) : 1 (1 b )exp( b ), (7) where i is he chage poi ad bi is he parameers reflecig he qualiy of esig for i 1,.... Here, sigle chage poi has bee cosidered o reduce he compuaioal complexiy. MVF of he proposed model wih sigle chage poi ca be obaied from he followig expressio: m( ) m(, ] m(, ] (8) Usig equaio () ad (8) MVF ca be wrie as: 11 Volume 11, Number 1, Jauary 16

4 m( ) a d1 ( s )c1' ( s )ds d ( s )c' ( s )ds (9) Solvig he above expressio wih di () ad ci () for i 1,, MVF of he proposed model ca be obaied as follows: m( ) a e b (1 ) e ( b ) e b (1 ) e ( b ) m( ) (1) (b ) b (b ) b Failure iesiy ad codiioal reliabiliy of he proposed model ca be obaied from equaio (4) ad (5). 4. Parameer Esimaio ad Compariso Crieria I his secio parameer esimaio echique of he proposed model ad differe compariso crieria has bee discussed Parameer Esimaio The ukow parameers of he models has bee esimaed wih leas square echique usig SPSS [1]. The posiio of chage poi has bee obaied usig chagepoi package i R sofware [4]. 4.. Compariso Crieria The followig compariso crierio have bee used o evaluae he performace of he proposed model Mea square error (MSE) I is defied as [7]: MSE = 1 (y i yˆ i ) i 1 (11) where yi ad y i are he observed ad prediced fauls respecively, is he oal umber of observaios Bias I is defied as he sum of he deviaio of he esimaed curve from he acual daa, defied as [1]: Bias= 1 (m( k )- mk ) k 1 (1) smaller value of bias is beer goodess of fi Variace I is defied as follows [1]: Variace = 1 (mk m( k ) Bias) 1 i 1 (13) smaller value of variace is beer goodess of fi The 1p% upper ad lower limi for m() 113 Volume 11, Number 1, Jauary 16

5 I has bee give by [5] ad defied as follows: ˆ ( ) p m ˆ ( ) ad m ˆ ( ) p m ˆ ( ) m The bouds of m() approximalely as follows: ˆ ( ) p m ˆ ( ) m( ) m ˆ ( ) p m ˆ ( ) m where mˆ ( ) is he esimae of m ( ) ad p is he (1 p) 1 perce poi of he sadard 5. Model Validaio ad Performace Aalysis I his secio a umerical example has bee show o evaluae he performace of he proposed model. For his purpose a real sofware failure daa se has bee ake which is published i [6] as firs daa se. This daa se coais 481 cumulaive umber of fauls durig 111 days es period. The esimaed parameers of he proposed model has bee give i Table.. Chage poi for his daa se is foud a posiio 48. Table 1. SRGMs wih Coverage Facor Model Mea Value Fucio Proposed model (M1) Equaio (3) Proposed model wih chage poi (M) Equaio (1) Expoeial coverage model (M3) [17] m( ) a(1 exp( b )) Weibull coverage model (M4) [17] m( ) a(1 exp( b k )) S-shaped coverage model (M5) [17] m( ) a(1 (1 b )exp( b )) Table. Esimaed Parameers ad Differe Compariso Crieria. M1 Esimaed Parameers a b1 β b β k Compariso Crieria MSE Bias Variace R.9854 M M M M Model From he Table, i is clear ha he esimaed oal umber of fauls by proposed model wihou chage poi is which is very close o he acual umber of fauls, i.e., 481. I meas 7 fauls are sill prese i he sofware a he ed of he esig. Besides he esimaed umber of fauls by proposed model wih chage poi is 481. which is exacly equal o he acual umber of fauls prese i he sofware. I meas o fauls are prese i he sofware a ed of he esig. As show i Table, esig coverage ad faul deecio probabiliy icreases afer chage poi. This shows he realisic behaviour of esig coverage ad faul deecio probabiliy. 114 Volume 11, Number 1, Jauary 16

6 m() m() Esimaed fauls Acual fauls 95% cofidece upper boud 95% cofidece lower boud 1 Esimaed fauls Acual fauls Fig. 1. Esimaio of cumulaive fauls wihou chage poi usig proposed model Fig.. Esimaed cumulaive umber of fauls wih 95% cofidece boud. From Table, i is clear ha MSE, Bias, Variace ad R of he proposed models is lowes i compariso o he oher models meioed i Table 1, which shows ha he proposed models are beer i compare o he oher models. While he proposed model wih chage poi is producig lower value of MSE, Bias, Variace ad R ha he proposed model wihou chage poi. I meas chage poi plays a sigificace role o improve he performace of he proposed model. Graphical represeaio of fauls esimaed by he proposed model wihou ad wih chage poi has bee show i Fig. 1 ad Fig. 3 respecively. 95% cofidece boud of he esimaed fauls by proposed model wihou chage poi ad wih chage poi has bee show i Fig. ad Fig. 4 respecively. From hese figures i is clear ha he paer of he esimaed fauls are very close o he acual fauls. Hece, proposed model is beer fi for give daa se m() m() Esimaed fauls Acual fauls 95% cofidece upper boud 95% cofidece lower boud 1 Esimaed fauls Acual fauls Fig. 3. Esimaio of cumulaive fauls wih chage poi usig proposed model Fig. 4. Esimaed cumulaive umber of fauls wih 95% cofidece boud. 6. Coclusio I his paper, a SRGM has bee proposed wih faul coverage ad chage poi iroducig he ime varia faul deecio probabiliy. S-shaped esig coverage fucio ad expoeial faul deecio probabiliy has bee cosidered i he proposed model. Real sofware failure daa se has bee used o validae he proposed model. Experimeal resuls esablished he fac ha he proposed model is beer ha he oher models. Also, he proposed model is more flexible ad realisic. Hece, he proposed models ca be very helpful for idusry ad sofware professioals o improve he qualiy of sofware. Ackowledgemes 115 Volume 11, Number 1, Jauary 16

7 Auhors ackowledge Idia School of Mies, Dhabad, Idia, for providig ecessary faciliies for his work. Refereces [1] Kapur, P. K., Pham, H., Gupa, A., & Jha. P. C. (11). Sofware Reliabiliy Assessme wih OR Applicaio. New York, USA: Spriger. [] Musa, J. D., Iaio, A., & Okumoo, K. (1987). Sofware Reliabiliy, Measureme, Predicio ad Applicaio. New York: McGraw-Hill. [3] Musa, J. D. (1975). A heory of sofware reliabiliy ad is applicaio. IEEE Trasacios o Sofware Egieerig, SE-1 (3), [4] Xie, M. (1991). Sofware Reliabiliy Modelig. Sigapore: World Scieific. [5] Lyu, M. R. (1996). Hadbook of Sofware Reliabiliy Egieerig. New York, USA: McGraw-Hill. [6] Kapur, P. K., Garg, R. B., & Kumar, S. (1999). Coribuios o Hardware ad Sofware Reliabiliy. Sigapore: World Scieific. [7] Pham, H. (). Sofware Reliabiliy. Sigapore: Spriger-Verlag. [8] Pham, H. (6). Sysem Sofware Reliabiliy. New York: Spriger. [9] Goel, A. L., & Okumoo, K. (1979). A ime-depede error deecio rae model for sofware reliabiliy ad oher performace measure. IEEE Trasacios o Reliabiliy, R-8(3), [1] Yamada, S., Ohba, M., & Osaki, S. (1983). S-shaped reliabiliy growh modelig for sofware error deecio. IEEE Trasacios o Reliabiliy, R-3(5), [11] Yamada, S., Tokuou, K., & Osaki, S. (199). Imperfec debuggig models wih faul iroducio rae for sofware reliabiliy assessme. Ieraioal Joural of Sysems Sciece, 3(1), [1] Chaerjee, S., Mishra, R. B., & Alam, S. S. (1997). Joi effec of es effor ad learig facor o sofware reliabiliy ad opimal release policy. Ieraioal Joural of Sysem Sciece, 8 (4), [13] Yamada, S., Tamura, Y., & Kimura, M. (1999). A sofware reliabiliy growh model for a disribued developme evirome. Trasacios of IEICE Japa, J8-A(9), [14] Chaerjee, S., Nigam, S., Sigh, J. B., & Upadhyaya, L. N. (11).Trasfer fucio modelig i sofware reliabiliy. Compuig, 9(1), [15] Chaerjee, S., Nigam, S., Sigh, J. B., & Upadhyaya, L. N. (11). Applicaio of fuzzy ime series i predicio of ime bewee failures ad fauls i sofware reliabiliy Assessme, Fuzzy Iformaio ad Egieerig, 3(3), [16] Chaerjee, S., & Maji, B. (15). A ew fuzzy rule based algorihm for esimaig sofware fauls i early phase of developme. Sof Compuig. [17] Gokhale, S. S., Philip, T., Marios, P. N., & Trivedi K. S. (1996). Uificaio of fiie failure ohomogeeous Poisso process models hrough es coverage. Proceedigs of he 7h Ieraioal Symposium o Sofware Reliabiliy Egieerig (pp ). [18] Zhao, M. (1993). Chage-poi problems i sofware ad hardware reliabiliy. Commuicaios i Saisics.-Theory ad Mehods, (3), [19] Che, M. C., Wu, H. P., & Shyur, H. J. (1). Aalyzig sofware reliabiliy growh model wih imperfec-debuggig ad chage-poi by geeic algorihms. Proceedigs of he 9h Ieraioal Coferece o Compuers ad Idusrial Egieerig (pp. 5 56). [] Shyur, H. J. (3). A sochasic sofware reliabiliy model wih imperfec debuggig ad chage poi. Joural of Sysems ad Sofware, 66(), [1] Ioue S., & Yamada, S. (8). Opimal sofware release policy wih chage poi. Proceedigs of he 8 IEEE Ieraioal Coferece o Idusrial Egieerig ad Egieerig Maageme (pp ). 116 Volume 11, Number 1, Jauary 16

8 [] Chaerjee, S., Nigam, S., Bahadur, J., & Upadhyaya, L. N. (1). Effec of chage poi ad imperfec debuggig i sofware reliabiliy ad is opimal release policy. Mahemaical ad Compuer Modellig of Dyamical, 18(5), [3] Chaerjee, S., & Sigh, J. B. (14). A NHPP based sofware reliabiliy model ad opimal release policy wih logisic-expoeial es coverage uder imperfec debuggig. Ieraioal Joural of Sysems Assurace Egieerig ad Maageme, 5(3), [4] Killick, R., & Eckley, I. (14). Chagepoi: A Rpackage for chage poi aalysis. Joural of Saisical Sofware, 58(3), [5] Yi, L., & Trivedi, K. S. (1999). Cofidece ierval esimaio of NHPP-based sofware reliabiliy models. Proceedigs of he 1h Ieraioal Symposium o Sofware Reliabiliy Egieerig (pp. 6-11). [6] Tohma, Y., Jacoby, R., Muraa, Y., & Yamamoo, M. (1989). Hypergeomeric disribuio model o esimae he umber of residual sofware fauls. Proceedigs of he 13h Aual Ieraioal Compuer Sofware ad Applicaios Coferece (pp ). Subhashis Chaerjee was bor i 1964, i Bhilai, Chasisgarh, Idia. He obaied his B.Sc (mahemaics) from T.D.B. College, Raigaj, he Uiversiy of Burdwa, Idia. He did his M.Sc (mahemaics) ad Ph.D from IIT Kharagpur, Idia. He is a member of IEEE Reliabiliy Sociey. His area of research is sofware reliabiliy modelig. Presely Dr. Chaerjee is workig as a associae professor, i he Dep. of Applied Mahemaics, Idia School of Mies (ISM) Dhabad, Idia. He has served G.I.E.T, Orissa ad SMIT, Sikkim, Idia, as a faculy. He has oal sixee years of eachig ad research experiece. He has quie a good umber of ieraioal ad aioal publicaios. He has reviewed papers for various aioal ad ieraioal jourals. His areas of ieres are sofware reliabiliy, web sofware reliabiliy, O.R., sochasic process, ad fuzzy se. Akur Shukla was bor i 199, i Ballia disric of Uar Predesh, Idia. He obaied his B. Sc. (physics ad mahemaics) from Ewig Chrisia College, Allahabad, Uiversiy of Allahabad. He received his M.Sc. degree i mahemaics ad scieific compuig i 1 from MNNIT, Allahabad, Idia. He is currely pursuig Ph.D. i he Deparme of Applied Mahemaics from Idia School of Mies Dhabad, Idia. His research ieress iclude sofware reliabiliy modelig, ime series, fuzzy ime series, geeic algorihm ad arificial eural ework. 117 Volume 11, Number 1, Jauary 16