Optimal Allocation of Testing Effort: A Control Theoretic Approach

Transcription

1 Proeedings of he 4 h Naional Conferene; INDIACom- Compuing For Naion Developmen, February 5 6, Bharai Vidyapeeh s Insiue of Compuer Appliaions and Managemen, Ne Delhi Opimal Alloaion of esing Effor: A Conrol heorei Approah P.K. Kapur Udayan Chanda Vijay Kumar 3 Deparmen of Operaional Researh, Universiy of Delhi, Delhi-7 pkkaur@gmail.om; uhanda@or.du.a.in; 3 vijay_parashar@yahoo.om ABSRAC Alloaion of finanial budge o a sofare developmen proje during he esing phase is a saggering ask for sofare managers. he hallenges beome siffer hen he naure of he developmen proess is onsidered in he dynami environmen. Many sofare reliabiliy groh models (SRGM) have been proposed in las deade o minimize he oal esing-effor ependiures bu mosly under sai assumpion. he main purpose of his paper is o invesigae an opimal resoure alloaion plan o minimize he os of sofare during he esing and operaional phase under dynami ondiion. An elaborae opimizaion poliy based on he opimal onrol heory is proposed and numerial eamples are illusraed. he paper also sudies he opimal resoure alloaion problems for various ondiions by eamining he behavior of he model parameers. he eperimenal resuls grealy help us o idenify he onribuions of eah seleed parameer and is eigh. KEYWORD SRGM, esing Effor Alloaion, Opimal Conrol heory, Learning Curve Effe.. INRODUCION he developmen yle is a hallenging bu signifian omponen of any SDLC proje. he hallenges beome siffer hen he naure of he developmen proess is onsidered in he dynami environmen. In general ompanies insalled differen proje managemen ools o synhronize he proess ih all oher omponens of proje. Sofare is released o he users phase a he end of he esing phase of Sofare Developmen Life Cyle (SDLC). Wih larger developmen and esing effors, beer qualiy sofare an be ensured. Bu his ould be ime onsuming and is undesirable in he prevalen ompeiive marke ondiions. Alloaion of finanial budge o a sofare developmen proje during he esing phase in he dynami environmen is a riial deision ha a sofare manager s has o make. During esing, resoures suh as manpoer and ime (ompuer ime) are onsumed. he failure (faul idenifiaion) and removal are dependen upon he naure and amoun of resoures spen. Many sofare reliabiliy groh models have been proposed in las deade o minimize he oal esing-effor ependiures bu mosly under he assumpions ha he relaionship beeen he esing effor onsumpion and esing ime (he alendar ime) follos Eponenial and Rayleigh disribuion. he ime dependen behavior of he esing effor has been sudied earlier by Basili e al [], Huang [5], Puman [6] and Yamada e al. [9]. Eponenial urve is used if he esing resoures are uniformly onsumed ih respe o he esing ime and Rayleigh urve oherise. Logisi and Weibull ype funions have also been used o desribe esing effor. Anoher approah is due o Musa e al. [], here hey have assumed he resoure onsumpion as an eplii funion of number of fauls removed and alendar ime. Kapur e al. [] have disussed he opimizaion problem of alloaing esing resoures in sofare having modular sruure. And proposed ha alloaion of esing effor should depend upon he size and severiy of fauls and also suggesed ha for differen resoure onsrains one an develop a rade-off beeen he maimum number of fauls o be removed in eah module and he effor required. In he researh paper he auhors have proposed hree differen MEFs (Marginal esing Effor Funions) for he opimizaion problem. As disussed, over he las ouple of deades many sofare reliabiliy groh models (SRGM) have been proposed o minimize he oal esing-effor ependiures, bu mosly under sai assumpion. Here in his paper e have ried o invesigae an opimal resoure alloaion plan o minimize he os of sofare during he esing phase under dynami ondiion. he paper also sudies he opimal resoure alloaion problems for various ondiions by eamining he behavior of he model parameers.. CONRIBUION OF SUDY A Sofare Reliabiliy Groh Model (SRGM) eplains he ime dependen behavior of faul removal. Several SRGMs have been proposed in sofare reliabiliy lieraure under se of assumpions and esing environmen, ye more are being proposed. he proposed SRGM in his paper akes ino aoun he ime dependen variaion in esing effor. he esing effors (resoures) ha govern he pae of esing for almos all he sofare projes are [] : a) Manpoer hih inludes Failure idenifiaion personnel Failure orreion personnel. b) Compuer ime. he key funion of manpoer engaged in sofare esing is o run es ases and ompare he es resuls ih desired speifiaions. Any deparure from he speifiaions is ermed as a failure. On a failure he faul ausing i is idenified and hen removed by failure orreion personnel. he ompuer failiies represen he ompuer ime, hih is neessary for failure idenifiaion and orreion. he influene of esing effor has also been inluded in some SRGMs [5,6,7,8,9,,6

2 Opimal Alloaion of esing Effor: A Conrol heorei Approah and 8]. In 976, Myers proposed ha sofare sysem should be onsrued and esed in separae in sequenial sep. Yamada e al. [9], Hou e al. [4], Pham and Zhang [4] has reommended ha sysem-level sofare esing ourred only afer he sysem as ompleely developed. Reenly, Blakburn e al. [] hoever suggesed ha sofare onsruion and sysem debugging and esing should be vieed as onurren aiviies. Kapur and Bardhan [8] eplored he relaionship beeen he number of fauls removed ih respe o ime and/or esing effor. he auhors propose ha sine during he esing phase of a sofare developmen yle, fauls are removed in o phases: firs a failure ours, and hen he faul ausing ha failure is orreed hene he esing effor should be spen on o separae proesses; failure idenifiaion and faul removal. In here paper he auhors developed a SRGM inorporaing ime lag no only beeen he o phases bu also hrough he segregaion of resoures beeen hem and proposed o alernae mehods for onrolling he esing effor for ahieving desired reliabiliy or error deeion level. Chiang and Mookerjee [3] analyze a developmen proess in hih sysem inegraion ours hen he number of errors in he sysem reahes a erain hreshold. In his paper e propose an alernae raionale for opimal alloaion of esing resoures using learning urve phenomenon under dynami environmen. he arile is divided ino he folloing seions: model developmen, dynami opimizaion, heoreial resuls, speial ases and numerial analysis. Finally, he arile onludes ih a disussion on he appliaion, eension and limiaions of he model. 3. MODEL DEVELOPMEN Sofare indusry is he high ehnology indusry here innovaion and knoledge reaion forms he primary fuel for oninued firm groh. he innovaion-relaed faors like developmen proess, sofare esing & debugging proess and eam sruure have signifian impa on a firm s fuure groh poenial. In he las fe deades he orld of ne sofare developmen based on sophisiaed developmen mehod, ne ehnologies and adap ools has evolved rapidly due o he inensified marke ompeiion. Sofare has beome an inegral par of nearly every engineered produ and fuel for onrolling various sysems suh as manufauring proesses, publi ransporaion e. A he same ime he hrea due o sofare failure is also beoming more inensified. As he riialiy of hese appliaions gros, he imporane of delivering sysems ih high reliabiliy anno be overemphasized. o ensure reliabiliy, sofare esing is neessary. In his researh paper, e reommend ha sofare esing and debugging should be vieed as onurren aiviies. Our goal is o build a simple, sruured model of onurren esing and debugging ih a vie o gaining insighs. In he paper, e have assumed ha during he esing phase of a Sofare developmen Life Cyle (SDLC), he fied oal resoures (W ) a any poin of ime an be divided ino o porions and (here, is he urren effor onsumpion o fi faul a ime and is he urren effor onsumpion due o esing a ime ) as depied in he figure. Resoures alloaed for debugging he sofare during he Sofare Developmen Life Cyle a any poin of ime () (before release) oal Resoures Uilized during he Sofare Developmen Life Cyle a any poin of ime is W (before release) Resoures alloaed for esing of he sofare during he Sofare Developmen Life Cyle a any poin of ime () (before release) Figure : Alloaion of oal resoures o solve he dynami opimizaion problems for resoure alloaion, e have used opimal onrol heory approah and assumed ha sofare esing and debugging an run onomianly. he onrol variable here and manages he evoluion of a sysem in suh a ay ha an opimal ouome (here, minimum os) is ahieved by he end of he ime horizon. In he paper he relaion beeen failure raes of sofare and os o derease his rae is modeled by various ypes of learning urves effe. 4. DYNAMIC OPIMIZAION PROBLEM We begin our analysis by saing a general model ih a very fe assumpions. We are resriing our analysis o he ase of a firm ha onrols is resoures for esing and debugging under finie planning horizon. We are also onsisen ih he idea ha he laen fauls in he sofare sysem are deeed and eliminaed during he esing phase and he number of fauls remaining in he sofare sysem gradually dereases as he esing progresses. herefore, i is reasonable o assume he folloing differenial equaion: d m( ) b a m () here, is he number of faul removed a ime. m is he umulaive number of faul removed ill ime. a is he iniial faul onen. b is he deeion rae. Apar from he above noaions fe oher noaions are also used in he analysis, hey are as follos: : he planning period. m, : oal os per uni a ime for umulaive faul removed m and debugging effor. is he os of esing per uni esing effors.

3 Proeedings of he 4 h Naional Conferene; INDIACom- W is he oal resoures uilized during he Sofare Developmen Life Cyle a any poin of ime. No suppose he sofare firm ans o minimize he oal ependiure over he finie planning horizon. hen he objeive funion for he ompany an be given by min subje o dm here m and ; b, W 5. OPIMAL SOLUION o solve he problem, Maimum priniple an be applied. he urren value Hamilonian is as follos [7]: H (3) here, is he urren value adjoin variables (shado os ih saisfy he folloing differenial e of ) h quaion. d dh ( ) (4) dm m m ih he ransversaliy ondiion a, We an inerpre as he marginal value of fauls a ime, hih should be negaive beause inreasing he number of fauls ill inrease he debugging os. he physial inerpreaion of he Hamilonian H an be given as follos: sands for fuure os inurred as one more faul inrodued in he sysem (a ime ). hus he Hamilonian is he sum of urren os and he fu ure os. In shor, H represens he insananeous oal os of he firm a ime. he folloing is he neessary ondiion hold for an opimal soluion: H b a m ba m ( 5) Oher opimaliy ondiions is H (6) () here, and From he above opimaliy ondiions, e an ge he folloing resuls: b a m (7) and, W b a m (8) Inegraing equaion (4) ih he ransversiliy ondiion, e have he fuure os of removing one more faul an be given as ( ) m 6. HEOREICAL RESULS he general formulaion and haraerisis of he proposed model ill help in gaining some insigh ino imporan faors influening he opimal poliies. No aking ime derivaive on (7), e have m m m m m () From () depending on he sign of numeraor, e have he folloing hree ases: ase. hen m m m m m ill be monoonially inreasing. Case. (9) () m m m m m ill be monoonially dereasing. Case 3. () m m m m m ill aain sauraion poin. m (3)

4 Opimal Alloaion of esing Effor: A Conrol heorei Approah 6.. Speial Cases No, In general a he iniial level during he esing phase of he H ( ) b a m sofare developmen lifeyle he debugging oss onfine () he large hunk of developmen os due o unerain naure of he errors. Laer on as he poenial faul onen redues he From (), i i s lear ha he Hamilonian is linear in onrol os due esing apures he majoriy of he ependiure. By his e an assume ha he debugging os gradually dereases variable, e have he folloing bang bang and ih ime. In his seion e have onsidered o senarios o singular soluion form f or o maimize he depi he os of debugging on he opimal poliies of he Hamilonian. hus, onrol variable. an be given as Case. In his seion e are assuming ha he oal os per uni for umulaive faul removed a ime is onsan. (4) i.e. For onsan faul removal funion, he objeive funion an be rien as min b, m and ; subje o dm here W (5) And he orresponding Hamilonian an be given as H W b a m (6) W And, he adjoin variable d an be defined as ) ( ) b ( )( (7) ih he ransversaliy ondiion a, From (7) and he ransversaliy ondiion of, e have W if H undefined if H if H i.e. W if ( ) b( a m) * ( ) undefined if ( a m) if ( ) b( a m) Hene, if ( a m) ( ) undefined if ( a m) W if ( a m) W () () (3) ( ) b ( )( ( ) ( )) (8) Figure: Alloaion poliy for Debugging Effor he neessary ondiion for opimaliy is H (9)

5 Proeedings of he 4 h Naional Conferene; INDIACom- W m (9) Figure3: Alloaion poliy for esing Effor he physial inerpreaion of he above opimal poliies for effor alloaion an be given as: If he oal os of fiing a bug is less han he uni esing os, hen inves all he effors for debugging purpose only. On he oher hand if per uni fiing os beomes greaer han he esing os, hen inves hole effor on esing Case In his seion e have onsidered a os funion ha follos learning effe phenomenon (Pegels 969), hih is of he form m m (4) here, is he base os., For his ase, he Hamilonian an be defined as: (5) H hus for opimaliy he neessary ondiion is H (6) m Here, and (7) hus, W m (8) If he planning horizon is long enough and debugging os funion follos learning urve phenomenon, hen from (8) and (9) e have he folloing o general alloaion sraegies Poliy. For, he ase in hih m i.e. hen he os of per uni esing is muh less han he debugging os, hen he opimal alloaion pah of debugging effor ill monoonially inreases and he esing effor pah monoonially dereases over ime (see figure 4). Effor ime Figure 4: Alloaion poliies for he o Effors hen m his resul an also be visualized as a siuaion hen mos of he fauls are idenified during a erain ime poin of he esing phase of SDLC and no muh esing are required heneforh. Hene o fi he fauls a he earlies, he sofare firm may onsider inreasing he debugging effor. For he opimal esing effor, as he higher porion of he fauls are idenified hene ompany may keep i lo for someime. Poliy. For, he ase in hih m i.e. hen he os of debugging is less han he esing os, hen he opimal alloaion pah of debugging effor ill monoonially dereases and he esing effor pah monoonially inreases over ime (see figure 5). he physial inerpreaion of he above equaion is ha he opimal poliy for fiing effor is equal o he raio of oal os of fiing per uni bugs o per uni esing os muliplied by he number of errors removed a ime. Hene, he opimal poliy for an be epressed as:

6 Opimal Alloaion of esing Effor: A Conrol heorei Approah Effor Cumulaive Number of Faul Removed =.7 =.3 =.6 =.5 =.4 ime ime Figure 5: Alloaion poliies for he o Effors hen m 7. NUMERICAL ANALYSIS In his seion, he propery of he various opimal poliies has been desribed on he proposed model using numerial eample. he purpose of his sudy is o ge some insigh ino he resul and also o sudy he impa of hange in effors on he debugging os model and he orresponding opimal os model. Several simulaion runs ere ondued using various parameer values; resuls onverged quikly and ere sable. o do so, e have onsidered he Pegels (969) form of learning urve o define he debugging funion. Firs some base values ere onsidered and hen differen model parameers are varied individually. he base values are as follos: a. 4 b In his analysis, he objeive is o hek he signifiane of alloaion of debugging effor ( ), hene is value has been assumed o be onsan hroughou he produ life yle. In he analysis i has been seen ha as he value of is gradually inreased keeping he oher parameers onsan, he rae of faul removal inreases rapidly and he os due o fuure removal slos don remarkably. his siuaion may arise, hen he majoriy of he fauls ere idenified during he esing and hene larger effor on debugging an aelerae he faul removal rae. A he same ime i ill redues any os for fuure removal. Figure 6: Cumulaive Number of Fauls Removed vs. ime Shado Cos (Adjoin Variable) =.7 =.6 =.5 =.4 = ime Figure 7: Shado Cos vs. ime Analysis as also performed o eamine he rend in he debugging os. he resul indiaes ha inreasing he debugging effor ( ) ill redues he oal debugging os and a same ime i redues he rae of inrease of he oal ependiure. hese paerns ere revealed hroughou he numerial analysis. 8. CONCLUDING REMARKS In his paper e have sudied an opimal resoure alloaion plan o minimize he os of sofare during he esing and operaional phase under dynami ondiion. Using opimal onrol heorei approah e have obained number of poliies for fiing effor and esing effor for differen os funion. We observed ha hen os are onsans he opimal poliy for ependiure on fiing effor is o alloae he resoures on debugging only if he oal os of fiing a bug is less han he uni esing os. If per uni fiing os beomes greaer han he esing os in he absene of any debugging effor, hen inves hole effor on esing. Similar kind of inerpreaion an be dran from oher opimal soluions. Coninued on Page No. 438

7 Proeedings of he 4 h Naional Conferene; INDIACom- Coninued from Page No. 43 he heoreial resuls obained here onfirm ha he opimaliy ondiions as desribed in previous lieraure. Many addiional useful resuls have also been idenified. he resuls ere verified using simulaion ehnique. 9. FUURE SCOPE A fe limiaions in our approah ha suggess areas for fuure researh, are as follos. he model is based on he assumpion ha a any poin of ime he oal resoures alloaed for debugging and esing is fied, as a resul boh he variable (i.e. esing and debugging) beomes epliily inerdependen. Hene onrolling one variable ill auomaially onrol he oher. Bu in praie hey may no be epliily inerdependen. Also, i is been onsidered ha he os funion depends on only one sae funion (i.e. faul removal funion) hene he model ignores he oher sages of faul removal proess (i.e. deeion). he rae of deeion an heavily be influened by he orreion proess. hus, here is need o inorporae epliily he oher dimensions of faul deeion and removals learly. Inorporaing oher sae variables (i.e. dimensions viz deeion funion) an be areas of fuure researh. Finally, he model an be eended in several ays, e.g. by inorporaing oher kind of sofare reliabiliy groh models (e.g. S-shaped) in he opimizaion modelling frameork ha an be useful o gain insighs ino he orhiness of alloaion of resoures in sofare reliabiliy analysis.. REFERENCE [] Basili, V.R. and M.V. Zelkoiz (979), Analyzing medium sale sofare developmen, in Proeedings of he 3rd Inernaional Conferene on Sofare Engineering, 6-3. [] Blakburn, J. D., G. D. Sudder, L. N. Van Wassenhove. (). Conurren sofare developmen. Comm. ACM 43() 4. [3] Chiang, I. R., V. S. Mookerjee. (4). A faul hreshold poliy o manage sofare developmen projes. Inform. Sysems Res. 5() 3 9. [4] Hou, R. H., S. Y. Kuo, Y. P. Chang. (997). Opimal release imes for sofare sysems ih sheduled delivery ime based on he HGDM. IEEE rans. Compu. 46() 6. [5] Huang, C-Y, S-Y. Kuo and J.Y. Chen (997), Analysis of a sofare reliabiliy groh model ih logisi esing effor funion, Proeeding of 8h Inernaional Symposium on sofare reliabiliy engineering, [6] Ihimori,., S. Yamada and M. Nishiaki (993), Opimal alloaion poliies for esing-resoure based on a Sofare Reliabiliy Groh Model, Proeedings of he Ausralia Japan orkshop on sohasi models in engineering, ehnology and managemen, [7] Kapur P.K., R.B. Garg and S. Kumar (999), Conribuions o Hardare and Sofare Reliabiliy, World Sienifi, Singapore. [8] Kapur P K, Bardhan AK. (). esing Effor Conrol hrough Sofare Reliabiliy Groh Modelling. Inernaional Journal of Modelling and Simulaion, (): [9] Kapur PK, Gupa Anu, Shanai Omar and Yadavalli V.S.S. (6). esing effor Conrol using Fleible Sofare Reliabiliy Groh Model ih Change poin. Inernaional Journal of Performabiliy Engineering, (3): [] Kapur P.K., Bardhan A.K., Yadavalli VSS (7). On alloaion of resoures during esing phase of a modular sofare. Inernaional Journal of Sysems Siene, 38 (6): [] Musa, J.D., Iannino, A. and Okumoo, K. (987), Sofare reliabiliy: Measuremen, Prediion, Appliaions, M Gra Hill, Ne York. [] Myers, G. J. (976). Sofare Reliabiliy: Priniples and Praies. John Wiley & Sons, Ne York. [3] Pegels, C.C. (969). On sarup or learning urves: An epeed vie, AIIE rans, 7(4): 6-. [4] Pham, H., X. Zhang. (999). A sofare os model ih arrany and risk oss. IEEE rans. Compu. 48(): [5] Pillai, K. and Nair, V.S.S. (997), A Model for Sofare Developmen effor and Cos Esimaion. IEEE ransaions on Sofare Engineering; 3(8), [6] Punam, L (978), A general empirial soluion o he maro sofare sizing and esimaing problem, IEEE ransaions on Sofare Engineering SE-4, [7] Sehi SP, hompson GL. (5). Opimal Conrol heory Appliaions o Managemen Siene and Eonomis, nd Edn. Springer, Ne York. [8] Xie M. (99), Sofare reliabiliy modeling, World Sienifi, Singapore. [9] Yamada, S., J. Hishiani and S. Osaki (993), Sofare Reliabiliy Groh Model ih Weibull esing effor: A model and appliaion, IEEE rans. on Reliabiliy R-4, -5.