Assertions and Measurements for Mixed-Signal Simulation

Assertions and Measurements for Mixed-Signal Simulation PhD Thesis Thomas Ferrère VERIMAG, University of Grenoble (directeur: Oded Maler) Mentor Graphics Corporation (co-encadrant: Ernst Christen) October 28, 2016

Cyber-Physical Systems Both discrete and continuous modes of operation Example: a cell phone A design: A bug: (courtesy of Samsung and AppleInsider) Verification is needed 1 / 40

Mixed-Signal Simulation Integrated Circuits Modeling Digital: event-driven p q = 0 q = 1 p (courtesy of ST Microelectronics) Implement both analog and digital electronics Design uses HDL and net lists at several stages Analog: algebraic differential equations ( f p x, dx ) = 0 dt Mixed-Signal: analog events (x > 2.0) and digital control f q 2 / 40

Simulation-Based Verification During the design stage run multiple simulations Each simulation produces a trace Records evolution of quantities over time Real-valued and Boolean signals Monitoring: each traced need to be analysed Evaluate requirements: correctness, robusteness, diagnostics In general measuring some performance Automation of the monitoring activity: Additional observer blocks Declarative property or measurement languages 3 / 40

Declarative Languages in Industry Assertions Digital domain Languages psl and sva built using two layers: regular expression temporal logic Discrete time interpretation Measurements Analog domain extract commands: signal processing, offline meas commands: event-driven, online 4 / 40

Research on Realtime Properties Problem: mixed-signal characterized by a synchronous interaction Solution: use continous-time representation Metric Temporal Logic (Koymans, 1990) Signal Temporal Logic for real-valued signals (Maler and Nickovic, 2004) Quantitative semantics for robustness estimate (Fainekos and Pappas, 2009) Timed Regular Expressions (Asarin, Caspi and Maler, 1998) 5 / 40

Limitations of Existing Tools and Techniques Digital assertions bound to precision of sampling clock Realtime properties monitoring not implemented Robustness computation is not efficient No easy diagnostic of temporal logic properties failure Measurements not controllable by sequential conditions No analog measures in a digital context 6 / 40

Outline 1. Preliminaries 2. Robustness Computation 3. Diagnostics 4. Regular Expressions Monitoring 5. Pattern-Based Measurements 6. Analog Measures in Digital Environment 7. Conclusion 7 / 40

Signal Temporal Logic Propositions p: Boolean variables q, conditions x c, and events p Temporal operators: Until: ϕ U I ψ Eventually: I ψ = U I ψ Always: I ψ = I ψ Formulas can be written with [a,b] and U only Example: stabilization property ϕ = ( q [0,5] [0,5] x 0.2) x 0.2 0 q t 0 t 0 + 5 t 0 + 10 t 8 / 40

Monitoring Offline approach (Maler and Nickovic, 2004): for each subformula ϕ compute set of times [ϕ] w where ϕ holds according to w Definition (Satisfaction Set) [p] w = {t : p w (t) = 1} [ [a,b] ϕ ] w = [ϕ] w [a, b] [ϕ ψ] w = [ϕ] w [ψ] w [ ϕ] w = [ϕ] w 9 / 40

Computation Theorem For any ϕ and w with finite variability, [ϕ] w is finite union of intervals Eventually operator: ϕ T [a,b] ϕ T [a, b] t Worst-case complexity O( ϕ ) 2 w 10 / 40

Example 0.2 0 x q x 0.2 q [0,5] x 0.2 [0,5] [0,5] x 0.2 q [0,5] [0,5] x 0.2 ( q [0,5] [0,5] x 0.2) 0 5 t 11 / 40

Quantitative Semantics Robustness value ϕ w indicates how strongly ϕ is satisfied / violated by w Positive if satisfied / negative if violated Magnitude = conservative estimate of distance to satisfaction / violation boundary Definition (Robustness Signal) x c w = c x w [a,b] ϕ w = t sup t [t+a,t+b] ϕ w = ϕ w ϕ w (t ) ϕ ψ w = max{ ϕ w, ψ w } 12 / 40

Outline 1. Preliminaries 2. Robustness Computation 3. Diagnostics 4. Regular Expressions Monitoring 5. Pattern-Based Measurements 6. Analog Measures in Digital Environment 7. Conclusion 12 / 40

Principle Theorem For any ϕ and w piecewise linear, ϕ w is piecewise linear Until rewrite rules preserve the robustness value Timed eventually computed using optimal streaming algorithm of (Lemire, 2006) adapted to variable-step sampling 13 / 40

Eventually Computation Problem: compute g(t) = sup t [t+a,t+b] f(t ) Solution: take maximum of f at t + a, t + b and sampling points inside (a, b) f i 2 t + a t + b 14 / 40

Eventually Computation Problem: compute g(t) = sup t [t+a,t+b] f(t ) Solution: take maximum of f at t + a, t + b and sampling points inside (a, b) f i 1 i 2 i 3 i 4 t + a t + b 14 / 40

Eventually Computation Problem: compute g(t) = sup t [t+a,t+b] f(t ) Solution: take maximum of f at t + a, t + b and sampling points inside (a, b) f i 1 i 2 i 3 i 5 i 4 t + a t + b 14 / 40

Eventually Computation Problem: compute g(t) = sup t [t+a,t+b] f(t ) Solution: take maximum of f at t + a, t + b and sampling points inside (a, b) f i 1 i 2 i i 5 3 t + a t + b 14 / 40

Eventually Computation Problem: compute g(t) = sup t [t+a,t+b] f(t ) Solution: take maximum of f at t + a, t + b and sampling points inside (a, b) f i 1 i 2 i 5 t + a t + b 14 / 40

Eventually Computation Problem: compute g(t) = sup t [t+a,t+b] f(t ) Solution: take maximum of f at t + a, t + b and sampling points inside (a, b) f i 2 i 5 t + a t + b 14 / 40

Example x 0.2 x 0.2 0 [0,5] x 0.2 0 [0,5] [0,5] x 0.2 0 0 5 t 15 / 40

Evaluation Worst-case complexity in 2 O( ϕ ) w Implementation benchmarked with random signals: w 10 2 10 3 10 4 10 5 [1,2] 0.0031 0.0030 0.0040 0.019 [1,11] 0.0029 0.0026 0.0039 0.017 [1,21] 0.0027 0.0026 0.0041 0.018 [1,31] 0.0030 0.0028 0.0041 0.021 Cost of computing [a,b] independent from b a Improves on related works by several orders of magnitude 16 / 40

Publications Donzé, Ferrère, and Maler. Efficient robust monitoring for STL. In Computer Aided Verification (CAV), 2013. 16 / 40

Outline 1. Preliminaries 2. Robustness Computation 3. Diagnostics 4. Regular Expressions Monitoring 5. Pattern-Based Measurements 6. Analog Measures in Digital Environment 7. Conclusion 16 / 40

Motivation Find small segment of w sufficient to cause violation of ϕ Example: violation of ( q [0,5] [0,5] x 0.2) x 0.2 0 q 0 Sub-traces = temporal implicants 5 t 17 / 40

Propositional Implicants Implicant of ϕ partial valuation whose extensions satisfy ϕ Definition Implicant of ϕ = term γ such that γ ϕ Prime implicant of ϕ = implicant of ϕ maximal relative to For diagnostic: implicant compatible with observed values v Problem (Diagnostic) For given ϕ and v, find γ ϕ such that v = γ 18 / 40

Temporal Implicants Temporal implicant of ϕ partial trace whose extensions satisfy ϕ Syntactical considerations: Terms with conjunctions t T θ(t) over intervals Limit values handled by non-standard reals t +, t Example: p(t) [1,2] p t [0.5,3.0] Theorem Every realtime property ϕ has a prime implicant Relies on boundedness of the time domain and non-standard extension 19 / 40

Computation for Signal Temporal Logic Diagnostic operators E, F such that: Explanation E(ϕ) ϕ Falsification F (ϕ) ϕ Definition (Diagnostic Signal) E(p) = p E( ϕ) = F (ϕ) E( [a,b] ϕ) = t E(ϕ)(ξ(t)) F ( [a,b] ϕ) = t F (ϕ)(t ) with selection function ξ such that ξ(t) [t + a, t + b] t [t+a,t+b] 20 / 40

Selection Function Compute ξ over some interval T where [a,b] ϕ holds: Current time t is at start of T Select last witness s of ϕ to account for [a,b] ϕ at t Remove from T the part R that has been accounted for ϕ [a,b] ϕ t already covered T 21 / 40

Overview Example: x 0.2 q [0,5] x 0.2 [0,5] [0,5] x 0.2 q [0,5] [0,5] x 0.2 ( q [0,5] [0,5] x 0.2) 0 5 t Worst-case complexity O( ϕ ) 2 w 22 / 40

Publications Ferrère, Maler, and Nickovic. Trace diagnostics using temporal implicants. In Automated Technology for Verification and Analysis (ATVA), 2015. 22 / 40

Outline 1. Preliminaries 2. Robustness Computation 3. Diagnostics 4. Regular Expressions Monitoring 5. Pattern-Based Measurements 6. Analog Measures in Digital Environment 7. Conclusion 22 / 40

Signal Regular Expressions Propositions p: Boolean variables q, threshold conditions x c Atomic expressions: holding p, events p Concatenation: ϕ ψ Kleene star: ϕ Duration restriction: ϕ I 23 / 40

Example Pulse pattern: ψ = r q p q [5,6] r 7.0 4.0 p = (x 4.0) x q = (4.0 < x 7.0) r = (x > 7.0) r q p q r t [5, 6] 24 / 40

Monitoring For any w expression ϕ defines a set of segments (t, t ) such that w[t, t ] matches ϕ Offline approach: for all subexpressions ϕ compute the complete set of matches [ϕ] w of ϕ relative to w Definition (Match Set) [ p ] w = {(t, t ) : t < t < t p w (t ) = 1} [ ϕ I ] w = {(t, t ) : t t I} [ϕ] w [ϕ ψ] w = [ϕ] w [ψ] w [ϕ ψ] w = [ϕ] w [ψ] w [ϕ ψ] w = [ϕ] w [ψ] w [ϕ ] w = i 0 [ ϕ i ] w 25 / 40

Match Set Representation A zone = convex set with horizontal, vertical and diagonal boundaries Represents a set of signal segments t Theorem For any ϕ and w with finite variability, [ϕ] w is a finite union of zones t 26 / 40

Match Set Representation A zone = convex set with horizontal, vertical and diagonal boundaries Represents a set of signal segments t t p t, t Theorem For any ϕ and w with finite variability, [ϕ] w is a finite union of zones 26 / 40

Match Set Representation A zone = convex set with horizontal, vertical and diagonal boundaries Represents a set of signal segments t p t s s t, t Theorem For any ϕ and w with finite variability, [ϕ] w is a finite union of zones 26 / 40

Example p [2,4] q [1,2] t Match set of p Match set of p [2,4] Match set of q [1,2] Match set of p [2,4] q [1,2] p t q t, t 27 / 40

Example p [2,4] q [1,2] t Match set of p Match set of p [2,4] Match set of q [1,2] Match set of p [2,4] q [1,2] p t q s s t, t 27 / 40

Example p [2,4] q [1,2] t Match set of p Match set of p [2,4] Match set of q [1,2] Match set of p [2,4] q [1,2] p t q t, t 27 / 40

Example p [2,4] q [1,2] t Match set of p Match set of p [2,4] Match set of q [1,2] Match set of p [2,4] q [1,2] p t q s s s t, t 27 / 40

Kleene Star On bounded traces w the sequence n i=0 ϕi converges to a fix-point in finitely many steps Assume w can be split in m constant segments v of length less that 1 Over each segment either [ϕ] v = [ ] v or [ϕ] v = [ ] v Lemma [ϕ n ] w [ ϕ n 1] w for any n > 2m + 1 Compute n i=0 ϕi by squaring: ɛ, ϕ, ϕ 2, ϕ 4,..., ϕ 2k up to k > log(2m + 1) 28 / 40

Evaluation Worst-case complexity: w O( ϕ ) without star Implementation using DBM for efficient zones computation Benchmarked for with randomized traces: ϕ = ( p p [0,10] ) ( q q [0,10] ) [80, ] w [ϕ] w time 3654 0 0.27 6715 10 1.35 13306 23 2.73 26652 47 5.83 Observed performance linear in w 29 / 40

Publications Ulus, Ferrère, Asarin, and Maler. Timed pattern matching. In Formal Modeling and Analysis of Timed Systems (FORMATS), 2014. Ulus, Ferrère, Asarin, and Maler. Online timed pattern matching using derivatives In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), 2016. 29 / 40

Outline 1. Preliminaries 2. Robustness Computation 3. Diagnostics 4. Regular Expressions Monitoring 5. Pattern-Based Measurements 6. Analog Measures in Digital Environment 7. Conclusion 29 / 40

Measurement Language Motivation: automate the extraction of mixed-signal measures Signal Regular Expressions control when the measure takes place Measure: aggregating operator duration, min, max, and average Example: average( (x > 1.0) (x > 1.0) (x > 1.0)) measures average value of x on high portions 30 / 40

Conditionals and Events Construct expressions delimited by events conditional operators:?ϕ begins a match of ϕ!ϕ ends a match of ϕ event-bounded expressions ψ: event p, p conditional event ψ?, ψ! sequence ψ ϕ ψ Theorem For any w and ψ event-bounded, [ϕ] ψ is finite 31 / 40

Case Study: Distributed System Interface DSI3 is a protocol for electronics in automotive industry Based on pulse communication Requirements about magnitude of signals and timing of events Implementation: behavioral model Controler i Sensor R e(t) v a(t) C 32 / 40

Timing Requirement x 7.0 4.0 r r q p q ψ r ψ? ψ t [5, 6] time between consecutive pulses 33 / 40

Results Pulse description: ψ = r q p q [5,6] r Measure expression: Computation time cost: ϕ = duration(ψ r ψ?) w quantize match extract total 1 10 6 0.047 0.617 0.000 0.664 5 10 6 0.197 0.612 0.000 0.809 1 10 7 0.386 0.606 0.000 0.992 2 10 7 0.759 0.609 0.000 1.368 34 / 40

Publications Ferrère, Maler, Nickovic, and Ulus. Measuring with timed patterns. In Computer Aided Verification (CAV), 2015. 34 / 40

Outline 1. Preliminaries 2. Robustness Computation 3. Diagnostics 4. Regular Expressions Monitoring 5. Pattern-Based Measurements 6. Analog Measures in Digital Environment 7. Conclusion 34 / 40

Analog Measurements and Digital Testbench Simulator-implemented measures provide guarantees: accuracy reproducible Unfortunately only accessible in analog environment Digital testbench enables structured verification assertion tracking coverage indicators... Mixed-signal verification often done with user-defined monitors 35 / 40

Measurement Tasks We propose new measurements functions as system tasks Input: (x, p), output: (y, q) task µ (x, p, y, q, e, r) Control: enable event e and reset event r Accessed in a variety of context: module, class, etc. Prototype implementation using VPI with functions: initialize µ, update µ, status µ, and evaluate µ 36 / 40

Phase Locked Loop Digital testbench using the Universal Verification Methodology: Measure relative jitter online, locking time and enforce safe operating area of current through VDD Computation time < 1s for measurements, 300s for simulation 37 / 40

Outline 1. Preliminaries 2. Robustness Computation 3. Diagnostics 4. Regular Expressions Monitoring 5. Pattern-Based Measurements 6. Analog Measures in Digital Environment 7. Conclusion 37 / 40

Contributions Diagnostic procedure for realtime assertions Efficient algorithms for robustness computation Monitoring of regular expressions Pattern-based measurements Bring practice of analog and digital verification closer 38 / 40

Publications 1. Donzé, Ferrère, and Maler. Efficient robust monitoring for STL. In Computer Aided Verification (CAV), 2013. 2. Ulus, Ferrère, Asarin, and Maler. Timed pattern matching. In Formal Modeling and Analysis of Timed Systems (FORMATS), 2014. 3. Ferrère, Maler, Nickovic, and Ulus. Measuring with timed patterns. In Computer Aided Verification (CAV), 2015. 4. Ferrère, Maler, and Nickovic. Trace diagnostics using temporal implicants. In Automated Technology for Verification and Analysis (ATVA), 2015. 5. Ulus, Ferrère, Asarin, and Maler. Online timed pattern matching using derivatives In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), 2016. 39 / 40

Future Works Robustness of Signal Regular Expressions New monitoring algorithms for SRE Integrate SRE with STL Formal verification using regular expressions 40 / 40