2 Modern Stochastic Process Methods for Multi-state System Reliability Assessment

Transcription

1 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen The purpose of his chaper is o describe basic conceps of applying a random process heory o MSS reliabiliy assessmen. Here, we do no presen he basics of he measure-heoreic framework ha are necessary o pure mahemaicians. Readers who need his fundamenal framework and a more deailed presenaion on sochasic processes can find i in Kallenberg (2002), Karlin and Taylor (98) and Ross (995). For reliabiliy engineers and analyss, he books of Trivedi (2002), Epsein and Weissman (2008), Aven and Jensen (999), and Lisnianski and Leviin (2003) are especially recommended. A grea impac o sochasic processes applicaion o MSS reliabiliy evaluaion was done by Navig (985) and Navig e al. (985). In his chaper, he MSS sysem reliabiliy models will be consequenly sudied based on Markov processes; Markov rewards processes, and semi-markov processes. The Markov processes are widely used for reliabiliy analysis because he number of failures in arbirary ime inervals in many pracical cases can be described as a Poisson process and he ime up o he failure and repair ime are ofen exponenially disribued. This chaper presens a deailed descripion of a discree-ime Markov chain as well as a coninuous-ime Markov chain in order o provide for readers a basic undersanding of he heory and is engineering applicaions. I will be shown how by using he Markov process heory MSS reliabiliy measures can be deermined. I will also be shown how such MSS reliabiliy measures as he mean ime o failure, mean number of failures in a ime inerval, and mean sojourn ime in a se of unaccepable saes can be found using he Markov reward models. These models are also he basis for reliabiliy-associaed cos assessmen and life-cycle cos analysis. In pracice, basic assumpions abou exponenial disribuions of imes beween failures and repair imes someimes do no hold. In his case, more complicaed mahemaical echniques such as semi- Markov processes and embedded Markov chains may be applied. Corresponding issues are also considered in his chaper.

2 30 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen 2. General Conceps of Sochasic Process Theory A sochasic or random process is, essenially, a se of random variables where he variables are ordered in a given sequence. For example, he daily maximum emperaures a a weaher saion form a sequence of random variables, and his ordered sequence can be considered as a sochasic process. Anoher example is he sequence formed by he coninuously changing number of people waiing in line a he icke window of a railway saion. More formally, he sequence of random variables in a process can be denoed by X (), where is he index of he process. The values assumed by he random variable X () are called saes, and he se of all possible values forms he sae space of he process. So, a sochasic process is a sequence of random variables { X () T}, defined on a given probabiliy space, indexed by he parameer, where varies over an index se T. In his book, we mainly deal wih sochasic processes where represens ime. A random variable X can be considered as he rule for assigning o every oucome ς of an experimen he value X ( ς ). A sochasic process is a rule for assigning o every ς he funcion X (, ς ). Thus, a sochasic process is a family of ime funcions depending on he parameerς or, equivalenly, a funcion of and ς. The domain of ς is he se of all he possible experimenal oucomes and he domain of is a se of non-negaive real numbers. For example, he insananeous speed of a car movemen during is rip from poin A o poin B will be a sochasic process. The speed on each rip can be considered as an experimenal oucome ς, and each rip will have is own speed X (, ς ) ha characerizes for his case an insananeous speed of he rip as a funcion of ime. This funcion will be differen from such funcions of oher rips because of he influence of many random facors (such as wind, broad condiions ec.). In Figure 2. one can see hree differen speed funcions for hree rips ha can be reaed as hree differen realizaions of he sochasic process. I should be noiced ha he cu of his sochasic process a ime insan will represen he random variable wih mean V m. In real-world sysems many parameers such as emperaure, volage, frequency, ec. may be considered sochasic processes. The ime may be discree or coninuous. A discree ime may have a finie or infinie number of values; coninuous ime obviously has only an infinie number of values. The values aken by he random variables consiue he sae space. This sae space, in is urn, may be discree or coninuous. Therefore, sochasic processes may be classified ino four caegories according o wheher heir sae spaces and ime are coninuous or discree. If he sae space of a sochasic process is discree, hen i is called a discree-sae process, ofen referred o as a chain.

3 2. General Conceps of Sochasic Process Theory 3 Fig. 2. Three realizaions of sochasic process V() The sochasic process X (, ς ) has he following inerpreaions:. I is a family of funcions X (, ς ), where and ς are variables. 2. I is a single ime funcion or a realizaion (sample) of he given process if is a variable and ς is fixed. 3. I is a random variable equal o he sae of he given process a ime when is fixed and ς is variable. 4. I is a number if and ς are fixed. One can use he noaion X () o represen a sochasic process omiing, as in he case of random variables, is dependence on ς. For a fixed ime =, he erm X ( ) is a simple random variable ha describes he sae of he process a ime. For a fixed number x, he probabiliy of X x X denoed by he even ( ) gives he CDF of he random variable ( ) { } X( ) F( x ; ) = F ( x ) = Pr X( ) x. (2.) CDF F ( x ; ) is called he firs-order disribuion of he sochasic process{ X() 0. } Given wo ime insans and 2, X ( ) and X ( 2 ) are wo random variables in he same probabiliy space. Their join disribuion is known as he second-order disribuion of he process and is given by F( x, x2;, 2) = FX ( ) X ( )( x, x2) = Pr{ X ( ) x, X ( 2) x2}. (2.2) 2 In general, he nh-order join disribuion of he sochasic process X() 0 is defined by { },

4 32 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen F( x, x,..., x ;,,..., ) = F ( x, x,..., x ) = 2 n 2 n X( ) X( 2)... X( n ) 2 n { X x X x X x } = Pr ( ), ( ),..., ( ) 2 2 n n (2.3) for all < 2 < < n. The las formula represens a complee descripion of a sochasic process. In pracice, o ge such a complee descripion of a sochasic process is a very difficul ask. Forunaely, in pracice many sochasic processes permi a simpler descripion. The simples form of he join disribuion corresponds o a family of independen random variables. Then he join disribuion is given by he produc of individual disribuions. Definiion 2. A sochasic process { X ( ) 0} is said o be an independen process if is nh-order join disribuion saisfies he condiion F n n ( x, x2,..., xn;, 2,..., n ) = F( xi; i ) = Pr{ X ( i ) xi}. (2.4) i= i= The assumpion of an independen process considerably simplifies analysis, bu i is ofen unwarraned and we are forced o consider some kind of dependence. The simples and a very imporan ype of dependence is he firs-order dependence or Markov dependence. Definiion 2.2 A sochasic process { X ( ) 0} is called a Markov process if for any 0 < < 2 <... < n < n < he condiional disribuion of X() for given values of X ( 0), X ( ),..., X ( n ) depends only on X ( n ): Pr{ X ( ) x X ( ) = x, X ( ) = x,..., X ( ) = x, X ( ) = x} = = Pr{ X ( ) x X ( ) = x}. n n n n 0 0 n n (2.5) This is a general definiion, which applies o Markov processes wih a coninuous-sae space. When MSS reliabiliy is sudied, discree-sae Markov processes or Markov chains are mosly involved. In he nex secions we will sudy boh discree-ime and coninuous-ime Markov chains. In he Markov process, he probabiliies of he random variable a ime > n depend on he value of he random variable a n bu no on he realizaion of he process prior o n. In oher words, he sae probabiliies a a fuure insan, given he presen sae of he process, do no depend on he saes occupied in he pas. Therefore, his process is also called memoryless. In many cases he condiional disribuion (2.5) has he propery of invariance wih respec o he ime origin n :

5 2. General Conceps of Sochasic Process Theory 33 Pr{ X ( ) x X ( ) = x } = Pr{ X ( ) x X (0) = x }. (2.6) n n Such a Markov process is said o be homogeneous. In addiion we consider here wo imporan sochasic processes ha will be used in he fuure: poin and renewal processes. A poin process is a se of random poins i on he ime axis. For each poin process one can associae a sochasic process X () equal o he number of poins i in he inerval ( 0, ). In reliabiliy heory poin processes are widely used o describe he appearance of evens in ime (e.g., failures, erminaions of repair, ec.). An example of he poin processes is he so-called Poisson process. The Poisson process is usually inroduced using Poisson poins. These poins are associaed wih cerain evens, and he number N(, 2) of he poins in an inerval (, 2) of lengh = 2 is a Poisson random variable wih parameer λ, where λ is he mean occurrence rae of he evens: { N( 2) k} Pr, are no overlapping, hen he random vari- If he inervals (, 2) and ( 3, 4) ables N(, 2) and ( 3, 4) he sochasic process X () = N( ) n λ k e ( λ) = =. (2.7) k! N are independen. Using he poins i one can form 0,. The Poisson process plays a special role in reliabiliy analysis, comparable o he role of he normal disribuion in probabiliy heory. Many real physical siuaions can be successfully described wih he help of Poisson processes. A well-known ype of poin process is he so-called renewal process. This process can be described as a sequence of evens, he inervals beween which are independen and idenically disribued random variables. In reliabiliy heory, his kind of mahemaical model is used o describe he flow of failures in ime. To every poin process i one can associae a sequence of random variables yn such ha y =, y2 = 2,..., yn = n n, where is he firs random poin o he righ of he origin. This sequence is called a renewal process. An example is he life hisory of iems ha are replaced as soon as hey fail. In his case, yi is he oal ime he ih iem is in operaion and i is he ime of is failure. One can see a correspondence among he following hree processes: a poin process i ; a discree-sae sochasic process X () increasing (or decreasing) by a poins i ; and n

6 34 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen a renewal process consising of random variables yi such ha n = y yn. A generalizaion of his ype of process is he so-called alernaing renewal process. This process consiss of wo ypes of independen and idenically disribued random variables alernaing wih each oher in urn. This ype of process is convenien for he descripion of repairable sysems. For such sysems, periods of successful operaion alernae wih periods of idle ime. 2.2 Markov Models: Discree-ime Markov Chains 2.2. Basic Definiions and Properies As was described above, a Markov process is a sochasic process whose dynamic behavior is such ha he probabiliy disribuion for is fuure developmen depends only on he presen sae and no on how he process arrived a ha sae (Trivedi 2002). Generally Markov echnique is very effecive in many pracical imporan cases (Inernaional Sandard IEC ). When he sae space, S, is discree (finie or counably infinie), hen he Markov process is known as a Markov chain. Since he sae space is discree and counable, we can assume wihou loss of generaliy ha S = { 0,, 2,3,... }. If he parameer space, T (recall ha we usually will consider ime as he parameer), is discree oo, hen we have a discree-ime Markov chain. Since he parameer T = 0,,2,3,.... Thus, a Markov chain space is discree, we will le { } { X( n), n = 0,,2,... } is described by a sequence of random variables X ( 0 ) x, X ( ) x, X ( 2 ) x,... x, x, x, are ineger numbers. If = 0 = = 2, where 0 2 he sae of he sysem a ime sep n is j, we denoe i as X ( n) j. = Then X 0 is he iniial sae of he sysem a ime sep 0. By using hese designaions in analogy wih (2.5), he Markov propery can be defined as Pr{ X n = xn X 0 = x0, X = x,..., X n = xn } = = Pr{ X n xn X n = xn }. (2.8) As in he case of a general Markov process, Equaion 2.8 implies ha chain behavior in he fuure depends only on is presen sae and does no depend on is behavior in he pas.

7 2.2 Markov Models: Discree-ime Markov Chains 35 We designae he probabiliy ha a sep n he chain will be in sae j as p ( n). Thus, we can wrie p ( n) = Pr{ X j}. (2.9) j n = We also define he probabiliy p ij ( m, n) ha he chain makes a ransiion o sae j a sep n if a sep m i was in sae i. This probabiliy is a condiional probabiliy, and we can wrie he following p ij ( m, n) = Pr{ X ( n) = j X ( m) = i}, 0 m n. (2.0) Condiional probabiliy p ij ( m, n) is known as he ransiion probabiliy funcion of he Markov chain. Here we will only consider homogeneous Markov chains hose in which p ij ( m, n) depends only on difference n-m. For such chains, he simpler noaion p ij ( n) = Pr{ X ( m + n) = j X ( m) = i}, 0 m n (2.) j is usually used o denoe so-called n-sep ransiion probabiliies. In words, pij ( n ) is he probabiliy ha a homogeneous Markov chain will move from sae i o sae j in exacly n seps. If n =, for homogeneous Markov chains we can also wrie { } p () = Pr X( m+ ) = j X( m) = i = p = cons. (2.2) ij The probabiliies p ij are called one-sep ransiion probabiliies. For engineering applicaions here we will consider only finie and counable sae space S = { 0,,2,..., M}. The one-sep ransiion probabiliies can be condensed ino a ransiion (one-sep) probabiliy marix P, where ij p00 p0... p0m p p... p = p = pm0 pm... pmm 0 M P ij. (2.3) Since for all i, j S, 0 p ij, and each row in P adds up o, marix P is a sochasic marix.

8 36 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen The probabiliy mass funcion of he random value X ( 0) is called he iniial probabiliy row-vecor ( 0) ( 0 ), ( 0 ),..., ( 0) p (2.4) = p0 p p M and presens he iniial condiions of a Markov chain. An equivalen descripion of he Markov chain can be given by a direced graph called he sae-ransiion diagram (or sae diagram for shor) of he Markov chain. A node labeled i of he sae diagram represens sae i of he Markov chain and a branch labeled p ij from node i o j represens he corresponding one-sep ransiion probabiliy from sae i o sae j Compuaion of n-sep Transiion Probabiliies and Sae Probabiliies The problem being considered here is in obaining an expression for evaluaing he p n from he one-sep ransiion probabiliies n-sep ransiion probabiliy ij ( ) p p () ij = ij. Recall ha for a homogeneous Markov chain according o expression (2.) we have he following: p ij ( n) = Pr{ X ( m + n) = j X ( m) = i}, 0 m n. Le us consider he ransiion probabiliy p ij ( m + n) ha he process goes o sae j a he ( m+ n) sep, given ha a 0 sep i is in sae i. In order o reach sae j a he ( m+ n) sep he process firs reaches some inermediae sae k a sep m wih probabiliy pik ( m ) and hen moves from k and reaches j a sep ( m+ n) wih probabiliy pkj ( n ). The Markov propery implies ha here are wo independen evens. Then using he heorem of oal probabiliy we obain p ij ( m + n) = pik ( m) pkj ( n). (2.5) k Equaion 2.5 is one form of he widely known Chapman Kolmogorov equaion and provides efficien calculaion of he n-sep ransiions probabiliies. We designae as P(n) he marix of n-sep probabiliies or, in oher words, he p n. Then, if in (2.5) we le m = and replace n by n-, we can rewrie Equaion 2.5 in marix form: marix whose (i,j) enry is ( ) ij

9 2.2 Markov Models: Discree-ime Markov Chains 37 ( n) ( n ) n P = P P = P, (2.6) where P is he one-sep probabiliies of he Markov chain. In words, he n-sep ransiion probabiliy marix is he nh power of he onesep ransiion probabiliy marix. Based on he obained resuls he uncondiional sae probabiliies pj ( n ) can be examined. Their values depend on he iniial sae probabiliies a n = 0 and on he number of seps passed since n = 0. I can be wrien as follows: p ( n) = Pr{ X( n) = j) = j = Pr( X(0) = i) Pr( X( n) = j X(0) = i) = p (0) p ( n). i i i ij (2.7) In marix form expression (2.7) can be rewrien as ( n ) = ( 0) n p p P, (2.8) where p(0) and p(n) are he row-vecors of he sae probabiliies iniially (a sep n = 0 ) and afer he nh sep, respecively. This implies ha uncondiional sae probabiliies of a homogeneous Markov chain are compleely deermined by he one-sep ransiion probabiliy marix P and he iniial probabiliy vecor p(0). To illusrae he presened approach, we consider he following example. Example 2. (Bha and Miller 2002). Assume a wo-sae Markov chain wih he saes denoed by 0 and (Figure 2.2). Fig. 2.2 Two-sae discree-ime Markov chain The one-sep ransiion probabiliy marix will be as follow p p p p = = P p0 p p0 p, 0

10 38 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen since i mus hold ha p00 + p0 = and p0 + p =. Assume ha p0 = α and p 0 β. = Then α α P =. β β The iniial condiions a sep n = 0 are he following p 0 (0) = a and p (0) a, = so he iniial sae probabiliy row-vecor is given as ( 0 ) = [ p (0), p (0)] = [ a, a] p. 0 Find he n-sep ransiion probabiliy marix and uncondiional probabiliies p n of saes 0 and a sep n, respecively. p0 ( n ) and ( ) Soluion. According o he given one-sep ransiion probabiliy marix P we can wrie p () = p = α, p () = p = α, p () = p = β, p () = p = β. 0 0 For n >, using Equaion 2.6, we obain p ( n) = p () p ( n ) + p () p ( n ) = = ( α) p ( n ) + β p ( n ) Now since he row sums of marix 0 00 n P are uniy, we have p ( n ) = p ( n ). Subsiuing p ( ) 0 n ino he previous equaion we obain for n > p ( n) = ( α) p ( n ) + β[ p ( n )] = β + ( α β) p ( n ) By using he las recurren equaion we can wrie he following:

11 2.2 Markov Models: Discree-ime Markov Chains 39 p p p... p 00 () = α, (2) = β + ( α β)( α), (3) = β + β( α β) + ( α β) ( α), 2 00 ( n) β β( α β) β( α β)... = β α β + α β α n 2 n ( ) ( ) ( ) = β α β + α β α k = 0 n 2 k n ( ) ( ) ( ). Based on he formula for he sum of a finie geomeric series, we can wrie: n 2 k = 0 ( α β ) k n ( α β ) = ( α β ) ( α β ) = α + β n. Therefore, he expression for p ( ) 00 n can be rewrien in he following form: p 00 β α( α β ) ( n) = + α + β α + β n. Now p ( ) 0 n can be found: α α( α β ) p0( n) = p00 ( n) = α + β α + β n. Expressions for he wo remaining enries p ( n ) and p ( ) 0 n can be found in a similar way. (Readers can do i hemselves as an exercise.) Thus, he n-sep ransiion probabiliy marix can be wrien as n n β + α( α β) α α( α β) n α + β α + β P( n) = P =. n n β β( α β) α + β( α β) α + β α + β Based on his n-sep ransiion probabiliy marix and on he given iniial sae probabiliy row-vecor p ( 0, ) one can find sae probabiliies afer he nh sep by using Equaion 2.8

12 40 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen n n β + α( α β) α α( α β) n α + β α + β p( n) = p( 0 ) P = [ a, a] n n β β( α β) α + β( α β) α + β α + β n n β + ( α β) α ( α β) = [ a( α + β) β], [ a( α + β) β]. α + β α + β Therefore, he sae probabiliies afer he nh sep are as follows: n β + ( α β) p0 ( n) = [ a( α + β) β], α + β n α ( α β) p ( n) = [ a( α + β) β]. α + β 2.3 Markov Models: Coninuous-ime Markov Chains 2.3. Basic Definiions and Properies As in he previous secion we confine our aenion o discree-sae Markov sochasic processes or Markov chains. The coninuous-ime Markov chain is similar o ha of he discree-ime case, excep ha he ransiions from any given sae o anoher sae can ake place a any insan of ime. Therefore, for a discree-sae coninuous-ime Markov chain he se of values X() is discree, X ( ) {,2,...}, and parameer has a coninuous range of values, [0, ). In reliabiliy applicaions he se S of saes is usually finie, = { K} S,2,...,, and so X ( ) {,2,..., K}. A discree-sae coninuous-ime sochasic process { X ( ) 0} is called a Markov chain if for 0 < <... < n < n is condiional probabiliy mass funcion saisfies he relaion { Xn = xn Xn = xn X = x X0 = x0} = Pr { X ( ) = x X ( ) = x }. Pr ( ) ( ),..., ( ), ( ) n n n n (2.9)

13 2.3 Markov Models: Coninuous-ime Markov Chains 4 Inroducing he noaions = n and n = n +Δ he expression (2.9) simplifies o Pr{ X ( +Δ ) = i X () = j} = π (, +Δ ). (2.20) The following designaion is ofen used for he simplificaion: π (, + Δ) = π (, Δ). ji These condiional probabiliies are called ransiion probabiliies. If he prob- π ji, Δ do no depend on, bu only on he ime difference Δ, he abiliies ( ) Markov process is said o be (ime-) homogeneous. (, ) π jj Δ is he probabiliy ha no change in he sae will occur in a ime inerval of lengh Δ given ha he process is in sae j a he beginning of he inerval. Noe ha ji ji, if j = i, π ji (, ) = (2.2) 0, oherwise. Taking ino accoun (2.2) one can define for each j a non-negaive coninuous funcion a (): j a ( ) = lim j Δ 0 π jj (, ) π jj (, + Δ) = lim Δ Δ 0 π jj (, + Δ) Δ (2.22) and for each j and i j a non-negaive coninuous funcion a (): a ji ( ) = π ji(, ) π ji (, + Δ) lim = Δ Δ 0 Δ 0 ji π ji (, + Δ) lim. (2.23) Δ The funcion aji () is called he ransiion inensiy from sae i o sae j a ime. For homogeneous Markov processes, he ransiion inensiies do no depend on and herefore are consan. If he process is in sae j a a given momen, in he nex Δ ime inerval here is eiher a ransiion from j o some sae i or he process remains a j. Therefore π jj ( Δ ) + π ji ( Δ ) =. (2.24) i j Designaing ajj = aj and combining (2.24) wih (2.22) one obains

14 42 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen a = a = lim π ( Δ ) = a. jj j ji ji Δ 0 i j i j Le pi () be he sae probabiliies of X () a ime : { } Δ (2.25) p () = Pr X() = i, j =,..., K; 0. (2.26) i Expression (2.26) defines he probabiliy mass funcion (pmf) of X() a ime. Since a any given ime he process mus be in one of K saes, K pi () = (2.27) i= for any 0. By using he heorem of oal probabiliy, for given >, we can express he pmf of X() in erms of he ransiion probabiliies π ij (, ) and he pmf of X( ): p( ) = Pr( X ( ) = j) = Pr{ X ( ) = j X ( ) = i}pr{ X ( ) = i} j = i S π (, ) p ( ). ij i i S (2.28) If we le = 0 in (2.28), we obain he following equaion: p j ( ) = π ij (0, ) pi (0). (2.29) i S This means ha he probabilisic behavior of a coninuous-ime Markov chain in he fuure is compleely deermined by he ransiion probabiliies π (0, ) and p p p K. The ransiion probabiliies of a coninuous-ime Markov chain { X ( ) 0} saisfy for all i, j S, he Chapman Kolmogorov equaion, which can be wrien he iniial probabiliy vecor ( 0 ) = [ (0),..., (0)] for his case in he following form: π (, ) = π (, ) π (, ), 0 <. (2.30) ij ik 2 kj 2 2 k S ij The proof of his equaion is based on he heorem of oal probabiliy:

15 2.3 Markov Models: Coninuous-ime Markov Chains 43 { X = j X = i} = { X = j X2 = kx = i} { X2 = k X = i} Pr ( ) ( ) k S Pr ( ) ( ), ( ) Pr ( ) ( ). (2.3) The subsequen applicaion of he Markov propery (2.20) o expression (2.3) yields (2.30). The sae probabiliies a insan +Δ can be expressed based on sae probabiliies a insan by using he following equaions: pj( +Δ ) = pj( ) ajiδ + pi( ) aijδ, i, j =,..., K. i j i j (2.32) Equaion 2.32 can be obained by using he following consideraions. The process can achieve sae j a insan +Δ in wo ways.. The process may already be in sae j a insan and does no leave his sae up o he insan +Δ. These evens have probabiliies p () and a Δ, respecively. 2. A insan he process may be in one of he saes i j and during ime Δ ransis from sae i o sae j. These evens have probabiliies pi () and aijδ, respecively. These probabiliies should be muliplied and summarized for all i j because he process can achieve sae j from any sae i. or Now one can rewrie (2.32) by using (2.29) and obain he following: pj( +Δ ) = pj( )[ + ajjδ ] + pi( ) aijδ (2.33) i j j i j ji p ( +Δ) p ( ) j j K K K = p () a Δ + p () a Δ = p () a Δ p () a Δ. i ij j jj i ij j ji i= i= i= i j i j i j (2.34) Afer dividing boh sides of Equaion 2.34 by we ge Δ and passing o limi Δ 0, dp () = p () a p () a, j =,2,..., K. K K j i ij j ji d i= i= i j i j (2.35)

16 44 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen The sysem of differenial equaions (2.35) is used for finding he sae probabiliies pj (), j =,, K for he homogeneous Markov process when he iniial condiions are given as p ( ) = α, j =,..., K. (2.36) j j More mahemaical deails abou (2.35) may be found in Trivedi (2002) or in Ross (995). When a sae-ransiion diagram for coninuous-ime Markov chain is buil, Equaion 2.35 can be wrien by using he following rule: he ime derivaive of pj () for any arbirary sae j equals he sum of he probabiliies of he saes ha have ransiions o sae j muliplied by he corresponding ransiion inensiies minus he probabiliy of sae j muliplied by he sum of he inensiies of all ransiions from sae j. Inroducing he row-vecor p () = [ p(), p2(),..., pk () ] and he ransiion inensiy marix a a a a a a a a a a... a 2 K K = K K2 KK, (2.37) in which he diagonal elemens are defined as ajj = aj, we can rewrie sysem (2.35) in marix noaion: dp() d () = p a. (2.38) Noe ha he sum of he marix elemens in each row equals 0: each i ( i K). K aij = 0 for When he sysem sae ransiions are caused by failures and repairs of is elemens, he corresponding ransiion inensiies are expressed by he elemen s failure and repair raes. An elemen s failure rae λ () is he insananeous condiional densiy of he probabiliy of failure of an iniially operaional elemen a ime given ha he elemen has no failed up o ime. Briefly, one can say ha λ () is he ime-ofailure condiional probabiliy densiy funcion (pdf). I expresses a hazard of fail- j=

17 2.3 Markov Models: Coninuous-ime Markov Chains 45 ure in ime insan under a condiion where here was no failure up o ime. The failure rae of an elemen a ime is defined as where R() F() F ( +Δ) F ( ) f( ) λ( ) = lim 0 Δ R() =, (2.39) Δ R() = is he reliabiliy funcion of he elemen, F() is he CDF of he ime o failure of he elemen, and f () is pdf of he ime o failure of he elemen. For homogeneous Markov processes he failure rae does no depend on and can be expressed as λ = MTTF, (2.40) where MTTF is he mean ime o failure. Similarly, he repair rae μ () is he ime-o-repair condiional pdf. For homogeneous Markov processes a repair rae does no depend on and can be expressed as μ = MTTR, (2.4) where MTTR is he mean ime o repair. A sae i is said o be an absorbing sae; if once enered, he process is desined o remain in ha sae. A sae j is said o be reachable from sae i if for some >0, π ij ( ) > 0. A coninuous-ime Markov chain is said o be irreducible if every sae is reachable from every oher sae. In many applicaions, he long-run (final) or seady-sae probabiliies pi = lim pi ( ) are of ineres. For an irreducible coninuous-ime Markov chain hese limis always exis for every sae i S, pi = lim pi ( ) = lim πij ( ) = lim πi ( ) (2.42) and hey are independen of he iniial sae j S. If he seady-sae probabiliies exis, he process is called ergodic. For he seady-sae sae probabiliies, he compuaions become simpler. The se of differenial equaions (2.35) is reduced o a se of K algebraic linear equaions because for he consan probabiliies all dpi () ime-derivaives are equal o zero, so = 0, i =,..., K. d Le he seady-sae probabiliies pi = lim pi () exis. For his case in seady sae, all derivaives of sae probabiliies on he lef-hand side of (2.35) will be ze-

18 46 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen roes. So, in order o find he long-run probabiliies he following sysem of algebraic linear equaions should be solved: k 0 = p ( ) a p ( ) a, j =,2,..., K. i ij j ji i= i= i j i j K (2.43) The K equaions in (2.43) are no linearly independen (he deerminan of he sysem is zero). An addiional independen equaion can be provided by he simple fac ha he sum of he sae probabiliies is equal o a any ime: K pi =. (2.44) i= Thus, seady-sae probabiliies of ergodic coninuous-ime Markov chains can be found using expressions (2.43) and (2.44). Now we consider addiional imporan parameers of he process in seady sae: sae frequency and mean ime of saying in sae. The frequency fi of sae i is defined as he expeced number of arrivals ino his sae per uni ime. Usually he concep of frequency is associaed wih he long-erm (seady-sae) behavior of he process. In order o relae he frequency, probabiliy, and mean ime of saying in sae i, we consider he sysem evoluion in he sae space as consising of wo alernaing periods he says in i and he says ouside i. Thus, he process is represened by wo saes. Designae he mean duraion of he says in sae i as T i and ha of he says ouside i, T oi. The mean cycle ime, T ci, is hen Tci = Ti + Toi. (2.45) From he definiion of he sae frequency i follows ha, in he long run, f i equals he reciprocal of he mean cycle ime Muliplying by T i boh sides of Equaion 2.46 one ges fi =. (2.46) T ci T i Ti fi = = p. (2.47) i T ci Therefore

19 2.3 Markov Models: Coninuous-ime Markov Chains 47 f p i i =. (2.48) i T This is a fundamenal equaion, which provides he relaion beween he hree sae parameers in he seady sae. Uncondiional random value T i is minimal from all random values T ij ha characerize he condiional random ime of saying in sae i if he ransiion is performed from sae i o any sae j i: T = min{ T,..., T }. (2.49) i i ij All condiional imes T ij are disribued exponenially wih he following cumulaive disribuion funcions Fij ( Tij ) = e. All ransiions from sae i a ij are independen and, herefore, he cumulaive disribuion funcion of uncondiional ime Ti of saying in sae i can be compued as follows: F( T ) = Pr{ T > } = Pr{ T > } i i i ij j i a ij a ij j i = Fij ( Tij ) = e = e. j i j i (2.50) This means ha uncondiional ime Ti is disribued exponenially wih parameer a = a, and he mean ime of saying in sae i is asfollows: i j ij T i =. (2.5) a j i ij Subsiuing T i in expression (2.48) we finally ge f = p a. (2.52) i i ij j i Once sae probabiliies, p i or p i (), have been compued, reliabiliy measures are usually obained as corresponding funcionals of hese probabiliies.

20 48 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen Markov Models for Evaluaing he Reliabiliy of Muli-sae Elemens According o he generic MSS model (Chaper ), any sysem elemen j can have k j differen saes corresponding o he performance raes, represened by he se g {,..., }. j = g j g jk j The curren sae of he elemen j and, herefore, he curren value of he elemen performance rae Gj () a any insan are random variables. Gj () akes values from g j: Gj () g j. Therefore, for he ime inerval [0,T], where T is he MSS operaion period, he performance rae of elemen j is defined as a sochasic process. Noe ha we consider only he Markov process where he sae probabiliies a a fuure insan do no depend on he saes occupied in he pas. In his subsecion, when we deal wih a single muli-sae elemen, we can omi index j for he designaion of a se of he elemen s performance raes. Thus, his se is denoed as g = { g,..., g }. k We also assume ha his se is ordered so ha g i+ g i for any i. The elemens can be divided ino wo groups. Those elemens ha are observed only unil hey fail belong o he firs group. These elemens eiher canno be repaired, or he repair is uneconomical, or only he life hisory up o he firs failure is of ineres. Those elemens ha are repaired upon failure and whose life hisories consis of operaing and repair periods belong o he second group. In he following subsecions, boh groups are discussed Non-repairable Muli-sae Elemen As menioned above, he lifeime of a non-repairable elemen lass unil is firs enrance ino he subse of unaccepable saes. In general, he accepabiliy of an elemen s sae depends on he relaion beween he elemen s performance and he desired level of his performance (demand). The demand W() is also a random process ha akes discree values from he se w= { w,, w }. M The desired relaion beween he sysem performance and he demand can be expressed by he accepabiliy funcion F(G(),W()). Firs consider a muli-sae elemen wih only minor failures defined as failures ha cause elemen ransiion from sae i o he adjacen sae i. In oher words, a minor failure causes minimal degradaion of elemen performance. The saespace diagram for such an elemen is presened in Figure 2.3. The elemen evoluion in he sae space is he only performance degradaion ha is characerized by he sochasic process {G() 0 }. The ransiion inensiy for any ransiion from sae i o sae i is λ, 2,...,. ii, i = k

21 2.3 Markov Models: Coninuous-ime Markov Chains 49 Fig. 2.3 Sae-ransiion diagram for non-repairable elemen wih minor failures When he sojourn ime in any sae i (or in oher words, he ime up o a minor failure in sae i) is exponenially disribued wih parameer λii,, he process is a coninuous-ime Markov chain. Moreover, i is he widely known pure deah process (Trivedi 2002). Le us define he auxiliary discree-sae coninuous ime sochasic process { X () 0, } where X () {,..., k}. This process is sricly associaed wih he sochasic process { G() 0. } When X () = i, he corresponding performance rae of a muli-sae elemen is gi : G() = gi. The process X() is a discree-sae sochasic process decreasing by a he poins i, i =,, k, when he corresponding ransiions occur. The sae probabiliies of X() are Noe ha p ( ) = Pr{ X( ) = i}, i =,..., k for 0. (2.53) i k p i ) = i= ( (2.54) for any 0, since a any given ime he process mus be in some sae. According o he sysem (2.35), he following differenial equaions can be wrien in order o find sae probabiliies for he Markov process presened in Figure 2.3:

22 50 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen dpk () = λkk, pk(), d dpi () = λi+, ipi+ ( ) λi, i pi( ), i = 2,3,..., k, d dp () = λ2, p2(). d (2.55) One can see ha in sae k here is only one ransiion from his sae o he sae k wih he inensiy of λk, k and here are no ransiions o sae k. In each sae i, i = 2,3,, k, here is one ransiion o his sae from he previous sae i+ wih he inensiy λi +, i and here is one ransiion from his sae o sae i wih he inensiy λ. ii, Observe ha here are no ransiions from sae. This means ha if he process eners his sae, i is never lef. Sae for non-repairable mulisae elemens is he absorbing sae. We assume ha he process begins from he bes sae k wih a maximal elemen performance rae of g k. Hence, he iniial condiions are p (0) =, p (0) = p (0) =... = p (0) = 0. (2.56) k k k 2 Using widely available sofware ools, one can obain he numerical soluion of he sysem of differenial equaions (2.55) under iniial condiions (2.56) even for large k. The sysem (2.55) can also be solved analyically using he Laplace Sieljes ransform (Gnedenko and Ushakov 995). Using his ransform and aking ino accoun he iniial condiions (2.56) one can represen (2.55) in he form of linear algebraic equaions: sp k() s = λk, k p k(), s sp i( s) = λi, ip + i+ ( s) λi, i p i( s), i = 2,3,..., k, sp () s = λ2,p 2(), s (2.57) s k k k 0 where = { } = p () s L p () e p () is he Laplace Sieljes ransform of a func- dpk () ion p k () and L = sp k() s pk(0) is he Laplace Sieljes ransform of d he derivaive of a funcion pk (). The sysem (2.57) may be rewrien in he following form:

23 2.3 Markov Models: Coninuous-ime Markov Chains 5 p k () s =, s + λ kk, λ p s = p s i = k λ2, p () s = p 2(). s s i+, i i() i+ (), 2,3,...,, s + λik, (2.58) Saring o solve his sysem from he firs equaion and sequenially subsiuing he obained resuls ino he nex equaion, one obains p k () s =, s + λkk, λ λ λ p s = i = k λ2, λ3,2 λ4,3 λkk, p ( s) =.... s ( s+ λ2,) ( s+ λ3,2 ) ( s+ λk, k 2 ) ( s+ λk, k ) i+, i i+ 2, i+ k, k i ( )..., 2,3,...,, ( s+ λii, ) ( s+ λi+, i) ( s+ λk, k 2 ) ( s+ λkk, ) (2.59) Now in order o find he funcions pk (), he inverse Laplace Sieljes ransform L { p k() s } = pk() should be applied (Korn and Korn 2000). In he mos common case when F( gi, w) = gi w (he elemen performance should no be less han he demand) for he consan demand level g i w > + gi ( i =,..., k ) he accepable saes are he saes i+,,k, where he elemen performance is above level g i. The probabiliy of he sae wih he lowes performance p () deermines he unreliabiliy funcion of he muli-sae elemen for he consan demand level g. 2 w > g Therefore, he reliabiliy funcion defined as he probabiliy ha he elemen is no in is wors sae (oal failure) is R () = p (). (2.60) In general, if he consan demand is g i w > + g, i =,..., k i, he unreliabiliy funcion for he muli-sae elemen is a sum of he probabiliies of he unaccepable saes,2,,i. Thus, he reliabiliy funcion is i Ri ( ) = p j ( ). (2.6) j=

24 52 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen The mean ime up o muli-sae elemen failure for his consan demand level can be inerpreed as he mean ime up o he process enering sae i. I can be calculaed as he sum of he ime periods during which he process remains in each sae j > i. Since he process begins from he bes sae k wih he maximal elemen performance rae g k [he iniial condiions (2.56)], we have k MTTF = i, i,2,..., k λ =. (2.62) j=+ i j, j According o (.23) one can obain he elemen mean insananeous performance a ime as k E = g p (). (2.63) i i i= The elemen mean insananeous performance deficiency for he consan demand w according o (.29) is D = k pi i= ( )max( w g,0). (2.64) i Example 2.2 We consider an elecric generaor insalled in an airplane where is mainenance is impossible during fligh. This generaor assumed as a nonrepairable muli-sae elemen ha can have only minor failures. The generaor has 4 possible performance levels (in saes 4, 3, 2, and is capaciies are g 4 = 0 KW, g 3 = 8 KW, g 2 = 5 KW and g = 0, respecively) and he following failure raes: λ 4,3 = 2 year, λ 3,2 = year, and λ 2, = 0.7 year. The iniial sae is he bes sae 4. Each fligh duraion is T fligh = 0 h. The airplane was designed for N fligh = 50 flighs up o general mainenance on he ground. Thus, he service ime up o he general mainenance is defined as T service = 500 h. The failure is defined as decreasing of generaing capaciy down he demand level 6 KW. Our objecive is o find he expeced energy no supplied o he airplane's consumers during he airplane service ime, he probabiliy ha he failure occurs during he service ime, and he mean ime up o he failure. Soluion. In order o find sae probabiliies he following sysem of differenial equaions should be solved according o (2.55):

25 2.3 Markov Models: Coninuous-ime Markov Chains 53 dp4 () = λ4,3 p4(), d dp3 () = λ4,3 p4() λ3,2 p3(), d dp2 () = λ3,2 p3 () λ2, p2(), d dp () = λ2, p2(), d wih he iniial condiions p 4 (0) =, p3(0) = p2(0) = p(0) = 0. Using he Laplace Sieljes ransform, we obain p s p s 4,3 4() =, 3() =, s+ λ4,3 ( s+ λ3,2 )( s+ λ4,3) λ λ λ λ λ λ p () s =, p () s =. ( )( )( ) ( )( )( ) 3,2 4,3 2, 3,2 4,3 2 s+ λ2, s+ λ3,2 s+ λ4,3 s s+ λ2, s+ λ3,2 s+ λ4,3 Using he inverse Laplace-Sieljes ransform, we find he sae probabiliies as funcions of ime : λ4,3 4 () = e, p λ p e e 4,3 λ3,2 λ4,3 3() = ( ), λ4,3 λ3,2 λ2, λ3,2 λ4,3 3,2 4,3 4,3 3,2 e + 2, 4,3 e + 3,2 2, e λ λ [( λ λ ) ( λ λ ) ( λ λ ) ] p2 () =, ( λ λ )( λ λ )( λ λ ) 3,2 2, 4,3 3,2 2, 4,3 p () = p () p () p () These probabiliies are presened in Figure 2.4. Now we can obain he reliabiliy measures for his muli-sae elemen. The reliabiliy funcions for differen demand levels are according o (2.6): R( ) = p( ), for g < w g2, R2( ) = p( ) p2( ), for g2 < w g3, R ( ) = p ( ) p ( ) p ( ) = p ( ), for g < w g These reliabiliy funcions are also presened in Figure 2.4.

26 54 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen Probabiliy p() p2() p3() p4() R() R2() Time (years) Fig. 2.4 Sae probabiliies and reliabiliy measures for non-repairable elemen wih minor failures According o (2.63) we obain he elemen mean insananeous performance a ime : 4 E = g p () = 0 p () + 8 p () + 5 p () + 0 p (). i i i= The demand is consan during he fligh and w = 6 KW. Therefore, according o (2.64), he elemen mean insananeous performance deficiency is 4 D = p ()max( w g,0) = p () + 6 p (). i i i= 2 Funcions E and D are presened in he Figure 2.5. Noe ha he expeced energy no supplied (EENS) o he airplane consumers during he service ime T = 500 h will be as follows: service T service EENS = D d KWh. 0

27 2.3 Markov Models: Coninuous-ime Markov Chains E D 6 Kw Time (years) Fig. 2.5 Mean insananeous performance and mean insananeous performance deficiency for non-repairable elemen wih minor failures Now based on (2.62) we obain he mean imes o failure MTTF = + + = 2.93 year for g < w g, 2 λ4,3 λ3,2 λ2, MTTF = + =.5 year for g < w g, λ4,3 λ3,2 MTTF = = 0.5 year for g < w g λ4,3 For he consan demand w = 6 KW, he mean ime o failure is equal o MTTF 2 =.5 years. The probabiliy ha his failure (decreasing he generaing capaciy lower han a demand level of 6 KW) will no occur during he service ime according o he graph in Figure 2.4 will be as follows: ( ) ( ) R2 = Tservice = R2 500 h = Now consider a non-repairable muli-sae elemen ha can have boh minor and major failures (a major failure is a failure ha causes he elemen ransiion from sae i o sae j: j < i ). The sae-space diagram for such an elemen represening ransiions corresponding o boh minor and major failures is presened in Figure 2.6.

28 56 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen Fig. 2.6 Sae-ransiion diagram for non-repairable elemen wih minor and major failures For he coninuous-ime Markov chain ha is represened by his sae-space diagram, he following sysem of differenial equaions for sae probabiliies can be wrien according o Equaions 2.35: k dpk () = pk() λk, e, d e= k i dpi () = λei, pe() pi() λie,, i = 2,3,..., k, d e=+ i e= k dp () = λe, pe(), d e= 2 (2.65) wih he iniial condiions (2.56). Afer solving his sysem and obaining he sae probabiliies pi (), i =,, k, he mean insananeous performance and he mean insananeous performance deficiency can be deermined by using (2.63) and (2.64). As in he case of he non-repairable muli-sae elemen wih minor failures, he unavailabiliy of he elemen wih boh minor and major failures is equal o he sum of he probabiliies of unaccepable saes. Therefore, for he consan demand w ( gi < w g i + ) one can use expression (2.6) for deermining he elemen reliabiliy funcion. The sraighforward mehod for finding he mean ime up o failure is no applicable for muli-sae elemens wih minor and major failures. The general mehod for solving his problem is based on he Markov reward model and is presened in a laer secion.

29 2.3 Markov Models: Coninuous-ime Markov Chains Repairable Muli-sae Elemens The more general model of a muli-sae elemen is he model wih repair. The repairs can also be boh minor and major. A minor repair reurns an elemen from sae j o sae j + while a major repair reurns i from sae j o sae i, where i > j+. The special case of he repairable muli-sae elemen is an elemen wih only minor failures and minor repairs. The sochasic process corresponding o such an elemen is called he birh and deah process. The sae-space diagram of his process is presened in Figure 2.7 (a). k µ k-,k λ k,k- µ 2,k k- λ k,2 µ k-2,k- λ k-,k-2 µ,k λ k, µ 2,3 λ 3,2 µ,k- 2 λ k-, µ,2 λ 2, (a) Fig. 2.7 Sae-ransiion diagrams for repairable elemen wih minor failures and repairs (a) and for repairable elemen wih minor and major failures and repairs (b) The sae-space diagram for he general case of he repairable muli-sae elemen wih minor and major failures and repairs is presened in Figure 2.7 (b). The following sysem of differenial equaions can be wrien for he sae probabiliies of such elemens: (b)

30 58 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen k k dpk () = μek, pe() pk() λke,, d e= e= k i i k dpi () = λei, pe() + μei, pe() pi()( λie, + μie, ), d e=+ i e= e= e=+ i i = 2,3,..., k k k dp () = λe, pe() p () μ, e, d e= 2 e= 2 (2.66) wih he iniial condiions (2.56). Solving his sysem one obains he sae probabiliies p (), i =,, k. i F gi w = gi w for he consan demand level g, i < w g i + he accepable saes where he elemen performance is above level g i are i+,, k. Thus, he insananeous availabiliy is When (, ) p A = k i ( ) e( ). (2.67) e= i+ The elemen mean insananeous performance and mean insananeous performance deficiency can be deermined by using (2.63) and (2.64). In many applicaions he seady-sae probabiliies lim pi ( ) are of ineres for he repairable elemen. As was said above, if he seady-sae probabiliies exis, he process is called ergodic. For he seady-sae probabiliies he compuaions become simpler. The se of differenial equaions (2.66) is reduced o a se of k algebraic linear equaions because for he consan probabiliies all ime-derivaives dpi () are equal o zero, hus, = 0, i=,,k. d Le he seady-sae probabiliies pi = lim pi ( ) exis. In order o find he probabiliies he following sysem of algebraic linear equaions should be solved k k 0 = μek, pm pk λke,, e= e= k i i k = λei, pe + μei, pe pi λie, + μie, i = k e=+ i e= e= e=+ i 0 ( ), 2,3,...,, k k 0 = λe, pe p μ, e. e= 2 e= 2 (2.68)

31 2.3 Markov Models: Coninuous-ime Markov Chains 59 The k equaions in (2.68) are no linearly independen (he deerminan of he sysem is zero). An addiional independen equaion can be provided by he simple fac ha he sum of he sae probabiliies is equal o a any ime: k p i =. (2.69) i= The deerminaion of he reliabiliy funcion for repairable muli-sae elemens is based on finding he probabiliy of he even when he elemen eners he se of unaccepable saes he firs ime. I does no maer which one of he unaccepable saes is visied firs. I also does no maer how he elemen behaves afer enering he se of unaccepable saes he firs ime. λ k,0 k, j j= i = λ λ i+,0 i+, j j= i = λ λ k,0 k, j j= i = λ Fig. 2.8 Sae-ransiion diagram for deerminaion of reliabiliy funcion R i() for repairable elemen (for a consan demand rae w: g i<w<g i+) In order o find he elemen reliabiliy funcion R i (), for he consan demand w ( g ), i < w g i + an addiional Markov model should be buil. All saes,2,,i of he elemen corresponding o he performance raes ha are lower han he demand w should be unied in one absorbing sae. This absorbing sae can be considered now as sae 0 and all repairs ha reurn he elemen from his sae back o he se of accepable saes should be forbidden. This corresponds o zeroing all

32 60 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen he ransiion inensiies μ0, m for m = i+,..., k. The ransiion rae λ m, 0 from any accepable sae m ( m > i) o he unied absorbing sae 0 is equal o he sum of he ransiion raes from sae m o all he unaccepable saes (saes,2,,i): λ = λ, m = k, k,, i+. (2.70) m,0 m, j j = i The sae-ransiion diagram for compuaion of he reliabiliy funcion is presened in Figure 2.8. For his diagram, he sae probabiliy p 0 () characerizes he reliabiliy funcion of he elemen because afer he firs enrance ino he absorbing sae 0 he elemen never leaves i: Ri () = p0 (). The sysem of differenial equaions for deermining he reliabiliy funcion of he elemen akes he following form: k k dpk () = μek, pe() pk() λke, + λk,0, d e=+ i e=+ i dp () k j j k j = λe, jpe() + μe, jpe() pj() λj, e+ λj,0 + μj, e, d e= j+ e= e= i+ e= j+ for i < j < k k dp0 () = λe,0 pe(). d e=+ i (2.7) Solving his sysem under iniial condiions ( ) ( ) ( ) p (0) =, p 0 = = p 0 = p 0 = 0 k k i 0 one obains he reliabiliy funcion as R k i ( ) = p0 ( ) = p j ( ). (2.72) j= i+ Obviously, he final sae probabiliies for sysem (2.7) are as follows: p = p = = p = 0, p =, k k i+ 0 because he elemen always eners he absorbing sae 0 when.

33 2.3 Markov Models: Coninuous-ime Markov Chains 6 Based on he compued reliabiliy funcion R k i ( ) = p j ( ) one can find he j= i+ mean ime o firs failure, when he elemen performance drops for he firs ime under demand level w, where g : i < w g i + MTTFi = Ri ( ) d. (2.73) 0 Once sae probabiliies, p i or p i (), have been compued, reliabiliy measures are usually obained based on hese probabiliies. Example 2.3 (Lisnianski and Leviin 2003). Consider a daa processing uni ha has k = 4 possible performance levels wih corresponding ask processing speeds: g 4 = 00 s, g 3 = 80 s, g 2 = 50 s, The uni has he following failure raes and g = 0 s. ( ) ( ) λ4,3 = 2 year, λ3,2 = year, λ2, = 0.7 year for minor failures, λ3, = 0.4 year, λ = 0.3 year, λ 4,2 4, = 0. year for major failures and he following repair raes μ3,4 = 00 year, μ2,3 = 80 year, μ,2 = 50 year ( for minor repairs ), μ,4 = 32 year, μ,3 = 40 year, μ2,4 = 45 year ( for major repairs ). The demand is consan w = 60 s. Find such elemen reliabiliy measures as availabiliy, mean performance, mean performance deficiency, reliabiliy funcion, and mean ime o firs failure. Soluion. The sae-space diagram for he uni is presened in Figure 2.9 (a). We assume ha he iniial sae is he bes sae 4. In order o find he sae probabiliies, he following sysem of differenial equaions should be solved: dp4 () = ( λ4,3 + λ4,2 + λ4,) p4() + μ3,4 p3() + μ2,4 p2() + μ,4 p (), d dp3 () = λ4,3 p4() ( λ3,2 + λ3, + μ3,4) p3() + μ,3 p () + μ2,3 p2(), d dp2 () = λ4,2 p4() + λ3,2 p3() ( λ2, + μ2,3 + μ2,4) p2() + μ,2 p (), d dp () = λ4, p4( ) + λ3, p3 ( ) + λ2, p2 ( ) ( μ,2 + μ,3 + μ,4 ) p ( ), d wih he iniial condiions p (0) =, 4 p3(0) = p2(0) = p(0) = 0.

34 62 2 Modern Sochasic Process Mehods for Muli-sae Sysem Reliabiliy Assessmen (a) Fig. 2.9 Sae-ransiion diagrams for four-sae elemen wih minor and major failures and repairs The elemen insananeous availabiliy can be obained for differen consan demand levels: (b) A () = p (), for g < w g, A () = p () + p (), for g < w g, A () = p () + p () + p () = p (), for g < w g These elemen insananeous availabiliies are presened in Figure Availabiliy A() A2() A3() Time (years) Fig 2.0 Insananeous availabiliy of four-sae elemen

35 2.3 Markov Models: Coninuous-ime Markov Chains 63 The elemen mean insananeous performance a ime is 4 E = g p () = 00 p () + 80 p () + 50 p () + 0 p (). k k k = For demand w = 60 s he elemen availabiliy will be he following w () = () A A 2. The mean insananeous performance deficiency (for consan demand w = 60 s ) is 4 D = pk ( ) max( w gk,0) = 0p2 ( ) + 60 p( ). k = The indices D and E, as funcions of ime, are presened in Figure 2.. Insananeous mean performance Time (years) (a) Performance deficiency Time (years) Fig. 2. Insananeous mean performance (a) and performance deficiency (b) of he four-sae elemen (b) If one wans o find only he final sae probabiliies he can do i wihou solving he sysem of differenial equaions. As was shown above, he final sae probabiliies can be found by solving he sysem of linear algebraic equaions (2.68) in which one of he equaions is replaced wih Equaion In our example, he sysem of linear algebraic equaions ha should be solved akes he form