6.1 Dependability Modeling. General Rules. Analysis

Size: px

Start display at page:

Download "6.1 Dependability Modeling. General Rules. Analysis"

Dorcas Simon
5 years ago
Views:

1 Dependable Systems Winter term 2018/2019 Dependable Systems 6 th Chapter Quantitative Analysis - Structural Models Christine Jakobs Professur Betriebssysteme Dependability is an umbrella term for a set of non-functional demands Questions about the planned / existing system When will it fail? How often will it fail? Are there weak parts in my architecture? What happens in an error situation? Real systems are way to complex to answer these questions directly Creation of a model for reduction of information A model is always of something and for something WS 2018/19 C. Jakobs 2 / 57 osg.informatik.tu-chemnitz.de Analysis General Rules Inductive analysis Starting from a special case ( temperature-related bit flip in memory ) Goal is a general conclusion ( system reliability in 2 years is 0.87 ) Deductive analysis Starting from failure ( Airplane crashed, 200 people dead ) Backward reasoning about root causes ( frozen speed sensor? ) Qualitative analysis Investigate only structural properties, ignore model parameters Quantitative analysis Assign numerical values to model parameters, calculation of results Numerical representation of dependability Event probability for a given point in time Event probability for any given point in time Parameter(s) for the distribution function of the random event Components are either fully working or completely failed Events are typically pair-wisely stochastically independent one-fault-at-a-time assumption WS 2018/19 C. Jakobs 3 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 4 / 57 osg.informatik.tu-chemnitz.de

Structural and State-based Models Success Space and Failure Space [Vesely] Quantitative analysis with structural models Reflect functional dependencies of the real system Easiest approach: Every

2 Structural and State-based Models Success Space and Failure Space [Vesely] Quantitative analysis with structural models Reflect functional dependencies of the real system Easiest approach: Every component failure is a system failure Realistic models represent potential error propagation chains Alternative: Quantitative analysis with state models Reflect state dependencies in the real system Easiest approach: There is one outage state reached on failure Realistic models consider multiple error states and degradation modes WS 2018/19 C. Jakobs 5 / 57 osg.informatik.tu-chemnitz.de Success Space and Failure Space [Vesely] (cont.) Structural and state models might be in failure or success space Failure space is more common Easier for engineers to agree on potential failures Example: Long delay in server roundtrip time Less unacceptable system states than normal states smaller model Failure models don t need to cover all system parts explicitly WS 2018/19 C. Jakobs 7 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 6 / 57 osg.informatik.tu-chemnitz.de Serial Case Help from probability theory: The probability of an event expressed as the intersection of independent events is the product of the probabilities of the independent events. Example: Chain of web server (a=0.9), application server (a=0.95) and database server (a=0.99) Benefit of replacing the database with an expensive model (a=0.999)? Benefit of replacing the web server with a new model (a=0.95)? WS 2018/19 C. Jakobs 8 / 57 osg.informatik.tu-chemnitz.de

Combination Parallel Case I Parallel case I Search engine, cluster node a=0.

.. (1 an )) n Y AS = 1 (1 ai ) Asite = alb AW Sset ADBset = alb [1 (1 aw S )nw S ] [1 (1

tu-chemnitz.de WS 2018/19 C. Jakobs 10 / 57 osg.informatik.tu-chemnitz.de Combination (cont.

) I Online brokerage site to be designed choice of components needed I Site availability

99% I Setup: Load balancer, similar web server hardware, replicated database I Question:

3 Combination Parallel Case I Parallel case I Search engine, cluster node a=0.85 (around 2 months outage / year) I How many servers to reach 5 nines of site availability? AS = 1 Palldown φs = clb (cw S1 cw S2 ) (cdb1 cdb2 ) AS = 1 ((1 a1 ) (1 a2 )... (1 an )) n Y AS = 1 (1 ai ) Asite = alb AW Sset ADBset = alb [1 (1 aw S )nw S ] [1 (1 adb )ndb ] i=1 nmin = d ln(1 AS ) e ln(1 a) WS 2018/19 C. Jakobs 9 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 10 / 57 osg.informatik.tu-chemnitz.de Combination (cont.) Combination (cont.) I Online brokerage site to be designed choice of components needed I Site availability aimed at 99.99% I Setup: Load balancer, similar web server hardware, replicated database I Question: What is the least expensive configuration that reaches 99.99%? I Choice between low-end (a=0.85) and high-end (a=0.999) servers I Must also consider purchase and maintenance costs per setup WS 2018/19 C. Jakobs 11 / 57 Load Balancer Web Servers DB Servers osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 12 / 57 osg.informatik.tu-chemnitz.de

k-out-of-n Generalization of serial and parallel structure is the k-out-of-n case At least k components, out of N components, must work k=1: Parallel case k=n: Serial case R S (t, k, N, R c ) = N i=k

4 k-out-of-n Generalization of serial and parallel structure is the k-out-of-n case At least k components, out of N components, must work k=1: Parallel case k=n: Serial case R S (t, k, N, R c ) = N i=k ( ) N R c (t) i (1 R c (t)) N i i k-out-of-n (cont.) For different component reliabilities [Jakobs]: e i =event with index i, for i = 1... N n =the set of numbers from 1 to n, N N = 1, 2,..., n N j =the set of all combinations of N n with j elements, N j {I : I N n, I = j} R i (t) =probability of occurrence of the event e i, i N n R I (t) =event that all components with an index out of I failed, F I (t) i I F i (t) R(t, k, n) = n ( 1) i k i=k ( ) i 1 k 1 I N j R i (t) i I XOR WS 2018/19 C. Jakobs 13 / 57 osg.informatik.tu-chemnitz.de Exclusive OR states that only one input event may occur For two possible events: F (t) = F 1 (t) + F 2 (t) 2 (F 1 (t) F 2 (t)); for n = 2 For more than two possible events: WS 2018/19 C. Jakobs 14 / 57 osg.informatik.tu-chemnitz.de 6.2 Reliability Block Diagram (RBD) 6.2 Reliability Block Diagram (RBD) Model logical interaction for success-oriented analysis of system reliability Building blocks: series structure, parallel structure, k-out-of-n structure System is available only if there is a path between s and t Granularity based on data availability and lowest actionable item concept Structure formula can be obtained from RBD by identifying the subset of nodes that disconnects s from t if removed F (t) = n F n i(t) (1 F j (t)) i=1 j=1 j i WS 2018/19 C. Jakobs 15 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 16 / 57 osg.informatik.tu-chemnitz.de

6.2 Reliability Block Diagram (RBD) RBD: k-of-n for Nonidentical Components Example: 2-out-of-3 different hard drives must remain functional Different manufacturers with different device reliability

2 Reliability Block Diagram (RBD) Break down into serial and parallel sections not always obvious, for example: A or B or C must work If A works, D must work If B works, than D or E must work If C

reliability is the probability of the union (=sum) of all mutually exclusive events (probabilities) that lead to system success Path Tracing method: Calculate probability of all possible paths

Jakobs 18 / 57 osg.informatik.tu-chemnitz.de Invented 1961 by H.

5 6.2 Reliability Block Diagram (RBD) RBD: k-of-n for Nonidentical Components Example: 2-out-of-3 different hard drives must remain functional Different manufacturers with different device reliability A S = a 1 a 2 a 3 + (1 a 1 )a 2 a 3 + a 1 (1 a 2 )a 3 + a 1 a 2 (1 a 3 Complex RBDs 6.2 Reliability Block Diagram (RBD) Break down into serial and parallel sections not always obvious, for example: A or B or C must work If A works, D must work If B works, than D or E must work If C works, E must work Decomposition method: Identify key component B, calculate system reliability with R B = 1 and R B = 0 and combine both results (total probability) Event space method: System reliability is the probability of the union (=sum) of all mutually exclusive events (probabilities) that lead to system success Path Tracing method: Calculate probability of all possible paths through the RBD, combine for system survival probability WS 2018/19 C. Jakobs 17 / 57 osg.informatik.tu-chemnitz.de 6.2 Reliability Block Diagram (RBD) Complex RBDs (cont.) WS 2018/19 C. Jakobs 18 / 57 osg.informatik.tu-chemnitz.de Invented 1961 by H. Watson (Bell Telephone Laboratories): Analysis of the launch control system of the intercontinental Minuteman missile Used by Boeing since 1966, meanwhile adopted by different industries Root cause analysis, risk assessment, safety assessment Basic idea Technique for describing the possible ways in which an undesired system state can occur Complex system failures are broken down into basic events WS 2018/19 C. Jakobs 19 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 20 / 57 osg.informatik.tu-chemnitz.de

calculated by lower level probabilities Graphical representation of structure formula, helps to identify fault classes Includes only faults that contribute to the top event In itself not a

6 Fault Trees (cont.) Static Fault Trees Basic events (faults) can be associated with component hardware failures, human errors, software errors, or any other pertinent events Probability of a higher-level event can be calculated by lower level probabilities Graphical representation of structure formula, helps to identify fault classes Includes only faults that contribute to the top event In itself not a quantitative model, but can be evaluated as one Events and gates are not system components! A Fails A D Fails G1 B Fails B B OR C Fail G2 C Fails C WS 2018/19 C. Jakobs 21 / 57 osg.informatik.tu-chemnitz.de Static Fault Trees (cont.) WS 2018/19 C. Jakobs 22 / 57 osg.informatik.tu-chemnitz.de Fault Tree construction Define FTA Scope Identify FTA Objective Define FT Top Event Define FTA Resolution Construct FT Evaluate FT Interpret/ Present Results Define FTA Ground Rules Objective should be phrased in terms of a system failure to be analyzed Define scope (design version, components to be included), resolution (based on available probability data) and ground rules (naming scheme for events and gates) Focus on necessary and sufficient immediate events WS 2018/19 C. Jakobs 23 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 24 / 57 osg.informatik.tu-chemnitz.de

Environmental stress boundaries - What is included (earthquake, bombs,... )? Level of resolution - How detailed should be the analysis for potential reasons?

7 Fault Tree construction (cont.) Fault Tree construction (cont.) Step 1: Define the undesired event to be analyzed what, where, when Step 2: Define boundary conditions for the analysis Physical boundaries What constitutes the system? Environmental stress boundaries - What is included (earthquake, bombs,... )? Level of resolution - How detailed should be the analysis for potential reasons? Step 3: Identify and evaluate fault events Primary failures as basic event, secondary failures as intermediate event Step 4: Complete the gates All inputs should be completely defined before further analysis of them Complete fault tree level by level Common mistakes [Misra] Ambiguous TOP event: Too general TOP event makes FTA unmanageable, too specific TOP event cannot provide a sufficient system analysis with FTA Ignoring significant environment conditions: External stress might be relevant Inconsistent fault tree event names - Same name for same fault event or condition Inappropriate level of resolution: Detail level of the fault tree should match the detail level of the available information Proper and consistent naming is very important (what failed and how) Statistically independent initiators, immediate contributors to an event Logic can be tested in success domain by inverting all statements and gates Analyze no further down than is necessary to enter probability data with confidence WS 2018/19 C. Jakobs 25 / 57 osg.informatik.tu-chemnitz.de Example: AND Gate WS 2018/19 C. Jakobs 26 / 57 osg.informatik.tu-chemnitz.de Example: OR Gate WS 2018/19 C. Jakobs 27 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 28 / 57 osg.informatik.tu-chemnitz.de

Example: INHIBIT Gate / Conditioning Event Example [Jakobs] Server Failure Power Network CPU Failure Mainboard RAID 15 Supply Card Failure Failure

23 λ=1500.23 Controller Failure Failure Failure λ=9708.74 λ=9708.74 λ=312132 k=2 HDD 1 HDD 2 HDD 3 HDD 4 HDD 5 HDD 6 λ=2008.12 λ=2008.

Jakobs 30 / 57 osg.informatik.tu-chemnitz.

8 Example: INHIBIT Gate / Conditioning Event Example [Jakobs] Server Failure Power Network CPU Failure Mainboard RAID 15 Supply Card Failure Failure Failure Failure λ= λ= Power Power CPU1 CPU2 CPU3 RAID HDD Failure Supply Supply λ= λ= λ= Controller Failure Failure Failure λ= λ= λ= k=2 HDD 1 HDD 2 HDD 3 HDD 4 HDD 5 HDD 6 λ= λ= λ= λ= λ= λ= WS 2018/19 C. Jakobs 29 / 57 osg.informatik.tu-chemnitz.de Example [Jakobs] (cont.) WS 2018/19 C. Jakobs 30 / 57 osg.informatik.tu-chemnitz.de Two kinds of evaluation Qualitative evaluation Identify event sets which cause failure Quantitative evaluation Determine failure probability Quantitative evaluation depends on qualitative evaluation WS 2018/19 C. Jakobs 31 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 32 / 57 osg.informatik.tu-chemnitz.de

Cut Sets Cut set: Any group of basic events which, if all occur at the same time, cause the TOP event Minimal cut set (mincut): Minimal combination of basic events that induce TOP,Minimal All basic

basic events whose nonoccurence at the same time ensures that TOP does not occur Cut Sets (cont.

9 Cut Sets Cut set: Any group of basic events which, if all occur at the same time, cause the TOP event Minimal cut set (mincut): Minimal combination of basic events that induce TOP,Minimal All basic events are needed to let the TOP event occur A long mincut shows low vulnerability, a short mincut shows high vulnerability A singleton cut set shows a single point (of) failure Path set: Set of basic events whose nonoccurence at the same time ensures that TOP does not occur Cut Sets (cont.) Analysze cut set for Weak points in the design Bypass of intended safety features Common cause problems Methods for cut set finding: Boolean reduction, bottom-up reduction, top-down reduction, mapping to binary decision diagram, Shannon decomposition, genetic algorithms,... WS 2018/19 C. Jakobs 33 / 57 osg.informatik.tu-chemnitz.de Method for Obtaining Cut Sets (MOCUS) [Rausand] WS 2018/19 C. Jakobs 34 / 57 osg.informatik.tu-chemnitz.de Boolean Reduction Example Start at the TOP event OR gate: Each input to the gate is written in separate rows AND gate: Each input to the gate is written in separate columns Iteratively replace gates in rows and columns Each resulting row forms a cut set B G3 C G1 A G4 C G2 A G4 B (A B) (C D) = (A C) (A D) (B C) (B D) A A = A A A = A A (A B) = A T OP =(B C A) (C A B) =(B C) (B A B) (C C) (C A B) (A C) (A A B) =(B C) (A B) C (C A B) (A C) (A B) =(B C) (A B) C (C A B) (A C) =A B C 2 resulting minimal cut sets (== all cut sets?) WS 2018/19 C. Jakobs 35 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 36 / 57 osg.informatik.tu-chemnitz.de

Qualitative Analysis Set of minimal cut sets describes all ways to cause the TOP event minimal failure set Set of minimal cut sets can also be determined for any intermediate event Can help with

event Event contribution to top event probability Decrease in top event probability if event would be removed Increase in top event probability if event was assured to occur Also known as sensitivity

routine OR gates can be protected by disallowing all inputs or by providing error recovery Example Protect G3 by preventing failure of A4 Protect G2 by preventing failure of A3 preventing failure of

10 Qualitative Analysis Set of minimal cut sets describes all ways to cause the TOP event minimal failure set Set of minimal cut sets can also be determined for any intermediate event Can help with quantitative analysis Finding the dominant minimal cut set: Calculate the probability of each minimal cut set, sort by probability Identification of event importance: Calculate importance measure per event Event contribution to top event probability Decrease in top event probability if event would be removed Increase in top event probability if event was assured to occur Also known as sensitivity test Fixing Cut Sets AND gates can be protected by disallowing one of the inputs Exhaustive testing or formal proof to show that the component cannot fail Test for failure condition and recovery routine OR gates can be protected by disallowing all inputs or by providing error recovery Example Protect G3 by preventing failure of A4 Protect G2 by preventing failure of A3 preventing failure of both A1 and A2 providing fault tolerance for G4 WS 2018/19 C. Jakobs 37 / 57 osg.informatik.tu-chemnitz.de Dynamic Fault Trees (DFT) Failure criteria of a system might depend not only on logical combination of basic events in the same time frame sequence-dependent failure Dynamic fault tree gates support sequences and dynamic probability changes Dynamic sub parts of the fault tree are typically analyzed by Markov model Example Failure of switch only relevant if it happens before outage of primary server What is the probability of switch fails before primary? Make analysis in closed form impossible Omitted in this lecture WS 2018/19 C. Jakobs 39 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 38 / 57 osg.informatik.tu-chemnitz.de FTA Report WS 2018/19 C. Jakobs 40 / 57 osg.informatik.tu-chemnitz.de

applying targeted upgrades minimize and optimize resources identify what is unimportant assist the system design monitor the performance of the system by FTA re-evalutation, based on former defects

11 FTA-based Decision Making RBD vs. FTA Use FTA to... understand the logic leading to the top event, especially in complex systems prioritize the contributors leading to the top event (typically 10% - 20%) proactively prevent the TOP event by applying targeted upgrades minimize and optimize resources identify what is unimportant assist the system design monitor the performance of the system by FTA re-evalutation, based on former defects and failures gain input data for FME(C)A WS 2018/19 C. Jakobs 41 / 57 osg.informatik.tu-chemnitz.de RBD vs. FTA (cont.) Convert fault tree to reliability block diagram Start from TOP event, replace gates successively Logical AND gate <-> parallel structure of the inputs of the gate Logical OR gate <-> serial structure of the inputs of the gate Elements in the fault tree: Failure events, blocks in the RBD: Functioning blocks Some FTA and RBD extensions are not convertible Example: Sequence-dependent gates in fault trees WS 2018/19 C. Jakobs 42 / 57 osg.informatik.tu-chemnitz.de Inductive analytical diagram in failure space, based on Boolean logic Developed during the WASH-1400 nuclear power plant safety study (1974) Fault trees became to large for proper analysis Condensation of system analysis into a manageable picture Make sure that the accident cases are sufficiently controlled Shows event sequences and accident progression in inductive analysis Popular approach in nuclear reactor safety engineering Starts with specific initiator (critical component failure) Companion to fault tree analysis, same stochastic foundation WS 2018/19 C. Jakobs 43 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 44 / 57 osg.informatik.tu-chemnitz.de

Event Tree Analysis Event Tree Analysis (cont.

human error that starts an accident scenario May be identified by other risk analysis technique Often

safety methods, to stop the accident Split to positive or negative progress, sometimes more than two

C. Jakobs 45 / 57 osg.informatik.tu-chemnitz.de Event Tree Analysis (cont.) WS 2018/19 C.

12 Event Tree Analysis Event Tree Analysis (cont.) Accident scenario: Series of events that result in an accident Initiating event: Technical failure / human error that starts an accident scenario May be identified by other risk analysis technique Often already identified and anticipated in the design phase Pivotal events: Intermediate events from the safety methods, to stop the accident Split to positive or negative progress, sometimes more than two outcomes Frequency of pivotal events in system parts can be obtained from fault tree analysis WS 2018/19 C. Jakobs 45 / 57 osg.informatik.tu-chemnitz.de Event Tree Analysis (cont.) WS 2018/19 C. Jakobs 46 / 57 osg.informatik.tu-chemnitz.de Event Tree Analysis (cont.) WS 2018/19 C. Jakobs 47 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 48 / 57 osg.informatik.tu-chemnitz.de

Event Tree Analysis (cont.) Possible event chains and the safety functions will be affected by hazard contribution factors Explosion or no explosion, time of the day, wind direction,.

13 Event Tree Analysis (cont.) Possible event chains and the safety functions will be affected by hazard contribution factors Explosion or no explosion, time of the day, wind direction,... For a sequence of n events, there will be 2 n branches Possible to split the outcomes into categories, based on severity Outcome frequency, loss of lives, material damage, environmental damage Reliability assessment of a safety function comes from FTA or RBD analysis Missing Data Quantitative dependability analysis is very hard in real life Ever-increasing complexity of software and hardware Faster product cycles, significant time-to-market constraints Imprecise architectural details Final component choices happen very late No budget for in-depth analysis, just a,side activity Trustworthy reliability data is an uncertainty factor Common solution: Qualitative-only analysis (FMEA) WS 2018/19 C. Jakobs 49 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 50 / 57 osg.informatik.tu-chemnitz.de Configurability + Missing Data IT systems constantly become more complex More systems become an IT system IT systems tend to be very flexible What is the best-possible redundancy mode here? Use proven slow component, or the new faster version of it?... System configurability also becomes a major factor for modeling What to add, what to leave out? Cost / dependability tradeoff? Typical solution:,copy-and-paste modeling Ignorance - No option in safety-critical systems Reduction Increasing effort, while having decreasing budgets Demands new concepts for system understanding Explizit statement Make ambiguity and uncertainty explicit New concepts for describing systems WS 2018/19 C. Jakobs 51 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 52 / 57 osg.informatik.tu-chemnitz.de

14 Example: Structural ambiguity Example: Structural ambiguity (cont.) WS 2018/19 C. Jakobs 53 / 57 osg.informatik.tu-chemnitz.de Example: Parameter uncertainty WS 2018/19 C. Jakobs 54 / 57 osg.informatik.tu-chemnitz.de Example: Parameter uncertainty (cont.) WS 2018/19 C. Jakobs 55 / 57 osg.informatik.tu-chemnitz.de WS 2018/19 C. Jakobs 56 / 57 osg.informatik.tu-chemnitz.de

15 Literature [Eri99] [Eri+15] [JTW16] [Lap92] [RA04] [Ves+81] Clifton A Ericson. Fault tree analysis. In: System Safety Conference, Orlando, Florida. Vol , pp. 1 9 Clifton A Ericson et al. Hazard analysis techniques for system safety. John Wiley & Sons, 2015 Christine Jakobs, Peter Tröger, and Matthias Werner. Configurable Fault Trees. In: Software Engineering for Resilient Systems: 8th International Workshop, SERENE 2016, Gothenburg, Sweden, September 5-6, 2016, Proceedings. Ed. by Ivica Crnkovic and Elena Troubitsyna. Springer International Publishing, 2016, pp Jean-Claude Laprie. Dependability: Basic concepts and terminology. Springer, 1992 Marvin Rausand and HÃ Arnljot. System reliability theory: models, statistical methods, and applications. Vol John Wiley & Sons, 2004 William E Vesely et al. Fault tree handbook. Tech. rep. Nuclear Regulatory Commission Washington dc, 1981 WS 2018/19 C. Jakobs 57 / 57 osg.informatik.tu-chemnitz.de

Risk Analysis of Highly-integrated Systems

Risk Analysis of Highly-integrated Systems RA II: Methods (FTA, ETA) Fault Tree Analysis (FTA) Problem description It is not possible to analyse complicated, highly-reliable or novel systems as black box