Weibull models for software reliability: case studies

Transcription

1 Weibull models for software reliability: case studies Pavel Praks a Tomáš Musil b Petr Zajac c Pierre-Etienne Labeau d mailto:pavel.praks@vsb.cz December 6, 2007 avšb - Technical University of Ostrava, Dept. of Applied Mathematics, Center of Quality and Reliability of Production (CQR.CZ), Université Libre de Bruxelles (ULB), Service de Métrologie Nucléaire (SMN) b Czech Technical University in Prague, Sun Microsystems Czech cvšb - Technical University of Ostrava d ULB, SMN, Av. F.D. Roosevelt 50, 1050 Bruxelles, Belgium 1

2 Motivation 1 Software Reliability is the probability of failure-free software operation for a specified period of time in a specified environment. Software Reliability is also an important factor affecting system reliability. It differs from hardware reliability in that it reflects the design perfection, rather than manufacturing perfection. The high complexity of software is the major contributing factor of Software Reliability problems. Jiantao Pan, Carnegie Mellon University,

3 Motivation 2: Notions from IEEE Standard Glossary of Software Engineering Terminology(IEEE Std ) Availability. The degree to which a system or component is operational and accessible when required for use. Often expressed as a probability. Reliability. The ability of a system or component to perform its required functions under stated conditions for a specified period of time. Software diversity. A software development technique in which two or more functionally identical variants of a program are developed from the same specification by different programmers or programming teams with the intent of providing error detection, increased reliability, additional documentation, or reduced probability that programming or compiler errors will influence the end results. 3

4 Motivation 3 Software-Reliability-Engineered Testing The concept of reliability here is defined as the probability of execution without failure for some specified interval, generally called mission time. John D. Musa, AT&T Bell Laboratories and James Widmaier, National Security Agency 4

5 Motivation 4 Software Reliability Techniques Reliability techniques can be divided into two categories: 1. Trending reliability tracks the failure data produced by the software system to develop a reliability operational profile of the system over a specified time. 2. Predictive reliability assigns probabilities to the operational profile of a software system; for example, the system has a 5 percent chance of failure over the next 60 operational hours. In practice, reliability trending is more appropriate for software, whereas predictive reliability is more suitable for hardware. Trending reliability can be further classified into four categories: Error Seeding, Failure Rate, Curve Fitting, and Reliability Growth. 5

6 Contents 1 Basic Mathematical Concepts 8 2 Bathtub curve model 9 3 Assumed Weibull models 13 4 Maximum likelihood estimation of Weibull models Case A (One parameter Exponential model) Case B (Three parameter Weibull model, with nonnegative location parameter) Case C (Three parameter Weibull model, with negative location parameter)

7 4.4 Case D (Two parameter Weibull model) Results of parameter estimations Estimation of 3-parameter Weibull model Estimation of parameters for a software reliability task Conclusions 40 7 Acknowledgment 42 7

8 1 Basic Mathematical Concepts Reliability R(t) is the probability that a system will be successful in the interval from time 0 to time t: R(t) = P(T > t), t 0 (1) where T is a random variable denoting the time-to-failure or failure time. Unreliability F(t) is defined as the probability that the system will fail by time t: F(t) = P(T t) = 1 R(t), t 0. (2) F(t) is the failure distribution function. 8

9 2 Bathtub curve model Maintenance modeling and consequent maintenance optimization of the system availability and or maintenance costs deeply depend especially on the ability of the model to predict the future behavior of the system. In our studies, we analyze a family of generalized Weibull models with 3 parameters which are based on the cumulative distribution function (CDF) F(t) = 1 e ( t ν η ) β, t ν 0 (3) 9

10 with the corresponding hazard function: ( ) β 1 t ν, t ν 0 (4) λ(t) = β η η The symbol η > 0 is called scale parameter, the symbol ν 0 is called location parameter. When ν = 0, the model is reduced to the 2 parameter Weibull. Knowledge about future behavior of the system is reflected by the shape parameter β. We speak about the bathtub curve reliability model, widely used in the reliability domain, see Figure 1. 10

11 Figure 1: The bathtub curve reliability model. 11

12 Weibull models appear to be very flexible not only for reliability models, but also for various technical applications. Weibull models were originally used for the scattering static strength of brittle materials [8], recently also for the modeling of wind speed[6], for analysis of unwanted radar echo signals [7], for image retrieval [3], and in the maintainability field for ageing models [5]. 12

13 3 Assumed Weibull models We assume the following probability density functions for our parameter estimation study: Weibull model with 2 parameters: f a (t) = β a η a ( t η a ) βa 1 e ( t ηa )β a (5) The cumulative distribution function (CDF) of the 2-parameter Weibull is: F a (t) = 1 e ( t ηa )β a (6) 13

14 The scale parameter symbol η a has the following meaning: Solving equation (6) for t = η a we have: F a (η a ) = 1 e ( η a η a )β a = 1 e 1 = = (7) So, the scale parameter η a is the 63rd percentile of the two parameter Weibull distribution. The scale parameter is sometimes called the Characteristic Life. The shape parameter β describes the speed of the ageing process in Weibull models and has no dimension. 14

15 Weibull model with 3 parameters including a positive shift ν > 0. The positive shift means that failures are not possible directly from the start of operation: f b (t) = β b η b ( t νb η b ) βb 1 e ( t νb η b ) βb (8) This probability density function is valid for t > ν b and it is expected that f b (t) = 0 otherwise. 15

16 Weibull with 3 parameters, including a negative shift ν < 0. The Weibull model with a negative shift can be used for model the pre-ageing components. The probability density function has the following conditional form: f c (t) = β c η c ( t νc η c ) βc 1 e ( t νc ηc )β c e ( ν c ηc )β c (9) This probability density function is valid for ν c 0 and it is expected that f c (t) = 0 for ν c > 0. 16

17 4 Maximum likelihood estimation of Weibull models In this section we describe parameter estimation studies using the Maximum Likelihood Estimation method. We assume the following probability density functions for the the distribution of failure times. Complete (i.e. no censoring) set of independent failure data t i > 0 for i = 1, 2,...,n are considered. 17

18 4.1 Case A (One parameter Exponential model) Let us start with a model with constant failure rate λ(t) = λ a > 0, which represents random failures. The corresponding likelihood function has then the following form: L a = n ( λa e λ at i ) = λ n a e λ n a t i i=1 (10) i=1 By the logarithming of L a we finally obtain: n ln(l a ) = n ln(λ a ) λ a t i (11) i=1 18

19 4.2 Case B (Three parameter Weibull model, with non-negative location parameter) The three parameter Weibull model, with non-negative location parameter ν b has the following failure rate: λ b (t) = β b η b ( t νb η b ) βb 1 H(t ν b ) (12) By assuming t i ν b for i = 1, 2,...,n, the likelihood function can be written as follows: n ( L b = λ b (t i )e ti ) [ n ( ) βb λ 1 ( b(u) du β b ti ν b 0 = e ti ) ] ν βb b η b η i=1 i=1 b η b (13) 19

20 By logarithming of L b we finally obtain: ( ) [ βb n ( ) ti ν b ln(l b ) = n ln + (β b 1) ln η b i=1 η b ( ) ] βb ti ν b η b (14) 20

21 4.3 Case C (Three parameter Weibull model, with negative location parameter) The three parameter Weibull model, with a location parameter ν c < 0 has the likelihood function with the following conditional form: ( ) βc 1 β n c ti ν c η c η c e ( ti νc ) β c ηc L c = e ( ν c ηc )β c (15) i=1 By logarithming of L c we finally obtain: ( ) βc νc ln(l c ) = ln(l b ) + n (16) η c 21

22 4.4 Case D (Two parameter Weibull model) The two parameter Weibull model is a special case of the three parameter Weibull model with the location parameter ν b = 0. Following (12), we can write the failure rate as follows: λ d (t) = β d η d ( t η d ) βd 1 (17) The expression of the likelihood function L d is also the same as the expression of L b with ν b = 0, see (13). The expression (14) with ν b = 0 is also suitable for the software implementation. 22

23 5 Results of parameter estimations In this section, we present results of parameter estimations of Weibull models by the maximum likelihood method. A Weibull law is postulated; a sample of failure data is drawn according to this reality. MLE computations are done on this basis mentioned in slide 24. In all cases, we use the following notation for diagrams of the estimated cumulative distribution functions: Case B (Three parameter Weibull model, with non-negative location parameter) is marked by the + symbols. Case C (Three parameter Weibull model, with negative location parameter) is marked by the - symbols. Case D (Two parameter Weibull model) is marked by the o symbols. 23

24 5.1 Estimation of 3-parameter Weibull model In order to verify the computer implementation of the maximum likelihood estimator, we first sampled randomly failure data from a three parameter Weibull distribution and than we analysed this random sample by the MLE using the simulated annealing approach accelerated by the local Nelder-Mead optimizer [4]. We assumed the following parameters of the reality (η, β, ν) = (2, 4, 0). We first estimated parameters of this Weibull model using N = 30, N = 100, N = 1000 and N = samples. The results of the MLE estimation seems to be correct, see following figures. 24

25 Notes: The symbol Zero represents the MLE results related to the Weibull model with 2 parameters. The symbol Neg represents the Weibull model with 3 parameters including a negative shift and finally the symbol Pos represents the Weibull with 3 parameters, including a positive shift. Of course, the estimated parameters are the same in this simple case and are very close to the parameter of the Weibull model, especially in case for N = 1000 and N =

26 Results of estimations (Figures of distribution functions) 26

27 27

28 28

29 29

30 30

31 Results of estimations (Figures of density functions) 31

32 32

33 33

34 34

35 35

36 5.2 Estimation of parameters for a software reliability task The developed software for parameter estimation of Weibull models was also used for reliability analyses of the real failure data related to a large software project (software reliability). The failure data were obtained from the company Sun Microsystems Czech. We analyzed three different sets of failure data. Results of parameter estimations are presented in the following figures: 36

37 37

38 38

39 39

40 6 Conclusions The ability of the model to predict the future behavior of the system is a crucial property for the correct maintenance modeling and consequent optimization of the system availability and/or maintenance costs. 1. We presented various experiments showing capability of Weibull models to fit various failure data from a software reliability task. These results are now analyzed. 40

41 2. We also used the developed software also for the analysis of failure times and/or stopping times (right-censored times) belonging to the secondary water circuit of nuclear plants [1]. We successfully verified the parameters of two parameter Weibull model presented in the PhD. thesis [1] and we also extended results by assuming the three parameter Weibull model. 3. All computations run smoothly, without technical difficulties. We also computed the Fisher information matrix by using the automatic BFGS approach [2]. The precision of the estimated Fisher information matrix is now checked. 41

42 7 Acknowledgment The support of the Université Libre de Bruxelles, Belgium related to the ARC Project - Advanced supervision and dependability of complex processes: application to power systems is graciously acknowledged by P. Praks. The research has been also supported by the Ministry of Education, Youth and Sports of the Czech Republic under the Research project 1M06047 The Center of Quality and Reliability (CQR.CZ) and by the K-Space NoE project. 42

43 References [1] Nicolas Bousquet. Analyse bayésienne de la durée de vie de composants industriels. PhD thesis, Département de Mathématiques d Orsay, Université Paris-Sud 11, [2] J. V. Burke, A.S. Lewis, and M.L. Overton. Hanso (hybrid algorithm for non-smooth optimization): a matlab package based on bfgs, bundle and gradient sampling methods. Courant Institute of Mathematical Sciences, New York University, [3] Jan-Mark Geusebroek and Arnold W. M. Smeulders. A six-stimulus theory for stochastic texture. Int. J. Comput. Vision, 62(1-2):7 16, [4] A. Hedar and M. Fukushima. Heuristic pattern search and its 43

44 hybridization with simulated annealing for nonlinear global optimization. Optimization Methods and Software, 19(3-4): , [5] P.-E. Labeau. Modélisation du vieillissement industriel et impact de la maintenance préventive. In Maintenance et Matrise des Risques, November [6] Edgar G. Pavia and James J. O Brien. Weibull statistics of wind speed over the ocean. Journal of Applied Meteorology, 25: , October [7] Matsuo Sekine and Yuhai Mao. Weibull Radar Clutter. Institution of Engineering and Technology, [8] W. Weibull. A statistical theory of the strength of materials. The Royal Swedish Institute of Engineering Research, Stockholm, 151:1 45,