UNIVERSITY OF TRENTO SCUOLA SUPERIORE SANT ANNA

Transcription

1 UNIVERSITY OF TRENTO Department of Information Engineering and Computer Science SCUOLA SUPERIORE SANT ANNA Institute of Communication, Information and Perception Technologies Joint Double Degree course in Information Science and Technology Final thesis Learning from Probabilities: System-Aware Execution Time and Dependence Analysis within Real-Time Systems Supervisor: Prof. Giuseppe Lipari Graduant: Alessandra Melani Academic year: 2012/2013

2

3 Acknowledgments This thesis grew out of a complex journey which allowed me to gain a series of extremely fruitful experiences. I would like to thank all the people that in many ways contributed to guide me until here. First of all, my deep gratitude goes to Professor Giuseppe Lipari, who allowed me to spend five months learning and working at ONERA, in Toulouse, and has always been a great mentor and an excellent advisor for this thesis. His wisdom and experience also helped me to face the important choices characterizing this fundamental step of my student career: I am really thankful for all the time and the support he dedicated to me. I would also like to acknowledge Luca Santinelli and Eric Noulard for their encouraging support and continuous feedbacks which shaped and guided the direction of my work: they introduced me in the world of research and their great attitude in the field inspired me to become a member of the academic family. I am grateful to the whole ONERA team, which provided me a rich and fertile environment where to learn and grow. My sincere appreciation goes to both University of Trento and Scuola Superiore Sant Anna, which jointly contributed to shape my knowledge and develop my learning abilities. For these reasons, I want to thank all the professors that I encountered during the last two years. The continuous support of many student colleagues and friends has been indispensable. I would like particularly to thank Alice, Fabio, Alessia and all the friends that I met in Trento, Pisa and Toulouse, with whom I shared struggles, interests and successes. I would not have contemplated this road if not for my dear parents, Marzia and Mauro, who instilled within me the strength to pursue my efforts and the love of learning, with continuous support and dedication. They gave me wings to face the challenges of life and realize myself.

4 Contents Abstract IV 1 Introduction A notion of predictability Timing analysis to the WCET From probabilities to predictability Probabilistic Timing Analysis From observations to worst-cases Aim of the thesis Overview The probabilistic natural behavior A probabilistic model for real-time systems Handling dependences Handling dependences within real-time systems Dependence Composability Characterization of dependences Probabilistic distance analysis The probabilistic WCET Extreme Value Theory Steps in the application of EVT Observing real-time systems System insight Requirements The SchedMCore framework What is SchedMCore? A measuring tool: LTTng Architecture Hardware features System settings II

5 Contents III 3.6 Use case Experiments and results Real-time measurements Reference benchmarks EVT Applicability Experiments jfdctint cnt nsichneu dijkstra ns adpcm Cache analysis Conclusions 57 A Numerical results 59 Bibliography I

6 List of Figures 1.1 Execution Time Profiles and WCET Histogram representation of a task execution times CDF and 1-CDF representation of a task execution times System-aware execution time perspective Task model representation Hardware architecture Linear dependence with qqplot for the cnt benchmark Linear dependence with qqplot for the nsichneu benchmark jfdctint: isolation, non-preemptive and preemptive case comparison jfdctint: preemptive case, burn configuration jfdctint: scheduling policy comparison jfdctint: number of preemptions comparison jfdctint: number of preemptions comparison cnt: different isolations compared cnt: and system compared. Upper-bound ( shifted) from distribution cnt: execution time comparison cnt: extreme value theory of the execution time cnt: different preemption conditions compared. Zoom on the tail of the execution time profiles cnt: different preemption conditions compared. Single and nested preemptions cnt: execution time with preemptions and offsets comparison. Execution time and cache misses distributions cnt: Cache effects with multi-cores and preemptions combined nsichneu: ET comparison for the three configurations I

7 List of Figures II 4.17 nsichneu configuration 3: upper-bound and extreme value theory nsichneu configuration 3: extreme value theory for isolation and upper-bound dijkstra: and system compared. Upper-bound ( shifted) from distribution dijkstra: preemption execution time and cache misses distribution comparison dijkstra comparison dijkstra 3 cores and preemption configuration upper-bounded from the isolation case ns: and system compared. Upper-bound ( shifted) from distribution ns: preemption execution time and cache misses distribution comparison ns execution time comparisons ns execution time distribution zoom at the tail ns: Execution time and cache misses with multi-cores configurations adpcm: and system compared. Upper-bound ( shifted) from distribution adpcm: preemption ET, DC, IC and L3 distrubution comparison adpcm: Execution time and Cache misses with multi-cores configurations adpcm execution time comparison with zoom at the distribution tail cnt Execution time EVT comparison cnt L3 cache EVT comparison

8 List of Tables 4.1 Benchmark characterization:.text,.data,.bss and total size (bytes), size of source code (bytes), lines of code, single- or multi-path p-values for the i.i.d. and linear correlation tests. jfdctint in isolation, cnt in isolation, nsichneu in isolation and nsichneu with preemptions are reported A.1 Benchmarks and configurations comparison for systemic effects. Maximum and average measured execution time together with the distance A.2 Benchmarks and configurations comparison for systemic effects: cache distances A.3 Benchmark and configurations comparison. Maximum and average measured execution time together with the distances. 60 A.4 Benchmark and configurations comparison for multi-core. Maximum and average measured execution time with the distance III

9 Abstract Real-time systems elements affect task execution behaviors in multiple directions. Modeling and analyzing such effects allows the reduction of the pessimism which lies behind the notion of worst-case execution time. Through a combination of measurement-based approaches and probabilistic real-time analysis, it is possible to enhance the knowledge on the interactions that take place within a complex system, thus obtaining more accurate worst-case execution times. The effects of system settings, preemptions and multi-core architectures are studied, focusing on the way they exercize the cache memory hierarchy and affect task executions. An ad-hoc probabilistic model is conceived to study such dependences and their effect on execution time; then the developed theoretical framework is applied to relevant case studies. IV

10 Chapter 1 Introduction 1.1 A notion of predictability The main purpose of real-time systems is achieving predictability, i.e. asserting the correct behavior of systems which have stringent timing constraints and whose failures could cause serious damages to the environment. Predictability is one of the most important properties that a real-time system should have: in particular, it is fundamental to ensure in advance that all the critical timing constraints will be met during execution. This obviously depends, among the others, on the information associated to each task, on the underlying hardware architecture, on the kernel features and on the interdependence among system components. The importance of guaranteeing timing constraints is linked to the consequences of deadline misses. In [8] this represents the criterion to distinguish two classes of real-time tasks: Hard real-time tasks: a task is said to be hard if a deadline miss can cause catastrophic consequences on the system. In this case, any instance of the task should be guaranteed a priori in the worst-case scenario; Soft real-time tasks: a task is said to be soft if missing its deadlines decreases the performance of the system but does not jeopardize its correct behavior. The classical deterministic analysis employs the notion of Worst-case Execution Time to produce a bound on the amount of time a task could execute, then used to perform schedulability analysis. 1

11 Chapter 1. Introduction 2 Definition 1 (Worst Case Execution Time (WCET)) The Worst-case Execution Time of a computational task is the maximum execution time the task could take to run on a specific hardware platform. A real-time system becomes predictable by always accounting for the worst-case at every task execution. Real-time schedulability analysis applies WCETs in order to verify system schedulability in the worst possible conditions. As the input space for the task code is finite and the hardware behavior is assumed to be deterministic, it is reasonable to argue about the exact worst-case execution time, W CET exact. In realistic situations, we aim to produce a safe estimation C of the WCET as an upper-bound for any possible execution behavior of the code. Unfortunately, this deterministic bound can be potentially very pessimistic and sometimes even unreliable: in fact, the classical deterministic analysis is not always able to cope with the great level of variability introduced by modern computing systems. In many cases, these unpredictabilities are mostly caused by the same components which are aimed to improve the average performance of the processor, such as prefetch, branch-prediction, cache memory hierarchy, pipelining, multiple cores, and also some characteristics of the real-time kernel, for example synchonization mechanisms, memory management policy, interrupt handling mechanisms [8]. Software elements may experience unpredictabilities due to input variability, unclear modeling and interruptions that occur whenever they are most inappropriate [15, 6, 21]. The system environment can be diverse and dynamic, with multiple possible evolutions in time [26]. Moreover, unpredictabilities are emphasized by the dependences which arise in complex systems because of the interferences and interactions that system components create among each other. These features are responsible for the introduction of nondeterminism and even for a degradation in the worst-case behavior. Consequently, an exact WCET is in general unknown and potentially unknowable. A systemic view for the WCET estimation has to be developed, accounting for the effects of hardware and software elements on the task execution. Since most of the times there is not the possibility to master all the configurations that an environment can assume, the notion of predictability we would like to define can be considered as an emerging behavior, achieved step by step by trying to quantify the influence of each subcomponent on the system [7].

12 Chapter 1. Introduction Timing analysis to the WCET Many approaches exist to the WCET computation problem. They can be distinguished into two main categories: Analytical methods Measurement-based metods The first class of methods relies on the knowledge of the code of a real-time task. Exploiting this knowledge, it is possible to perform a static analysis, trying to retrieve all the possible control flow paths through the code implementing the task [42]. This information is then combined with some (abstract) model of the underlying hardware, allowing to compute a W CET static estimation. Static timing analysis is also currently approaching systemic effects to the WCET estimation [42]: caches, memory and busses are often taken in account while analytically estimating WCET [2, 28, 11, 10]. Nevertheless, the complexity of this approach could increase unacceptably as we try to produce more and more accurate models of the hardware. In fact, the combinations of its states would grow to a point where the computation of an acceptable upper-bound on execution time would become unfeasible [13]. The second class of methods relies instead on the direct observation of task executions and represents a valid alternative to the static approach. The measurement-based analysis relies on a simpler model which is subject to an extensive testing, obtaining a sample of observed execution times [44]. The W CET measured is the maximum value observed during measurements, but in general W CET measured W CET exact, due to the not assured execution condition coverage. For this reason, usually a safety margin is added to the W CET measured, in order to take in account unpredictable behaviors which contribute to the worst-case, but are not guaranteed to be observed at testing level. However, this procedure can lead to very wrong assumptions on the estimation of WCET, because we have no guarantee about the validity of the margin. 1.3 From probabilities to predictability The problems formerly introduced can be overcome by introducing a probabilistic approach. Probabilities are becoming a consistent way of representing the time evolution of systems. They are able to model indeterminacy and unpredictabilities that systems have, since a probabilistic representation

13 Chapter 1. Introduction 4 captures multiple behaviors and their frequency of happening. Such a more fine-grained representation can reduce the pessimism associated with a deterministic view in representing the time evolution of the system, where only the worst-case estimation is taken into account. In this sense the probabilistic approach offers a real alternative to the deterministic timing analysis. The challenging goal in this area is to make predictability out of probabilities [7, 13], in such a way to describe the dynamics and dependences that realistic real-time systems have in terms of a probabilistic model through statistical analysis. Exploiting probabilities in different system conditions, it is possible to characterize at a fine level the behavior of a system; a deep awareness of all system contributions is necessary to produce more optimistic WCET estimations. The objective of the probabilistic approach is to determine a distribution function that expresses the probability for the execution time to exceed a given value. In this way, the concept of deterministic WCET is replaced by a probabilistic WCET, which is no longer a scalar value but a distribution: for each value of execution time, we can get the probability for that value to be exceeded, so that any desired acceptance threshold (in terms of probability) can be fixed and used for the worst-case execution time computation Probabilistic Timing Analysis Many different analytical approaches exist in the context of Probabilistic Timing Analysis (PTA). They usually employ some form of extreme value statistical analysis [16, 17, 27] to construct a predicted WCET value, W CET predicted. In particular, recent works have re-formalized the application of Extreme Value Theory (EVT) to the WCET problem [13, 9], by making use of an ad-hoc probabilistic hardware architecture. With measurements they are able to guarantee accurate WCET estimations from execution time distributions. However, such approach does not consider the dependence among system elements and the effect they have on task execution. Another weak point of probabilistic measurement-based approaches concerns the coverage. Assuring exhaustive execution condition coverage requires deep knowledge of the system [4]. Even if the worst-case execution time is computed through test generation technique which ensure feasible path coverage, usually assumptions are made to negate variability [15, 43]. Our approach tries to combine probabilities with measurements, considering realistic hardware architectures instead of artifact models, so to develop an extensive system awareness when performing probabilistic WCET estimations.

14 Chapter 1. Introduction From observations to worst-cases The empirical distribution obtained by measurements is called Execution Time Profile [37]. Obviously, there is no notion of worst-case in the amount of information we can collect by running a specific application on a target hardware. In fact, there is absolutely no guarantee of including the worst case execution time among the observed samples, due to the not assured input and execution condition coverage. For this reason, it is necessary to elaborate the set of observations with a technique which is aimed to predict the probability of occurrence of extreme values, known to be rare events. In our case, we will consider the Extreme Value Theory (EVT) as the mathematical method which allows to estimate the maximal values of a set of observations [13, 27]. This concept is clarified by Figure 1.1, which illustrates the interval of possible values of execution time (possible), whose maximum value is the W CET exact. Moreover, it represents a measured profile (measurements) and one possible overestimation of the WCET (safe). EVT will be examined in detail in Section 2.5. measurements exact safe possible Figure 1.1: Execution Time Profiles and WCET 1.4 Aim of the thesis The aim of this work is to improve the knowledge of real-time systems by characterizing in probabilistic terms the dependences that relate system elements, in order to reduce the pessimism of their model and increase the accuracy of the timing analysis. This will avoid the need of overlaying pessimistic safe bounds and the consequent waste of resources, still guaranteeing the safety of the WCET prediction. For this purpose, we will produce a probabilistic model which is able to include and quantify dependences. Then we

15 Chapter 1. Introduction 6 will apply it to realistic benchmarks, focusing on the effect that cache memories have on the task execution times. Combining a measurement-based approach with probabilistic analysis, we can derive accurate task execution time probability distributions. The cache is exercized by playing with operating system interferences (e.g. prefetch and isolation), task interferences (e.g. preemptions) and hardware interferences (i.e. multi-core and the memory resource hierarchy). Building systemic awareness will allow to cover worst-case execution conditions; the grey-box approach that we intend to develop will reduce the pessimism of actual probabilistic timing models without increasing their complexity. 1.5 Overview Chapter 2 will introduce the probabilistic model that will be applied to characterize tasks execution times and their worst-cases. Chapter 3 will present the necessary requirements to observe and monitor a realistic real-time system and will introduce our specific environmental setup. It will also expose the system elements and the configurations we aim to investigate in the experimental stages. Chapter 4 will describe the experiments and the obtained results, while the last chapter will be for the conclusions.

16 Chapter 2 The probabilistic natural behavior Due to variability and unpredictability of modern real-time systems, what emerges as their behavior is close random processes 1. In this chapter we will clarify how probabilities come into play in such complex situations, proposing a theoretical framework which allows to characterize complex behaviors and dependence effects. 2.1 A probabilistic model for real-time systems A probabilistic real-time system is a real-time system where at least one of its parameters is described by a probability distribution function. In this work we consider the execution time as the parameter with a probabilistic representation. In particular, we assume the execution time of a task τ i to be a discrete random variable, since the execution time of a program can only be a multiple of the processor clock cycle. We will refer to C i 2 as the random variable describing the execution time of the task. The mathematical function which describes the possible values of a random variable and their associated probabilities is known as a Cumulative Distribution Function (CDF), defined as: F Ci (c) = P(C i c). 1 A random process is a sequence of random variables describing a process, whose outcomes do not follow a deterministic pattern, but follow an evolution described by probability distributions. 2 With calligraphic letters we refer to probabilistic distributions. 7

17 Chapter 2. The probabilistic natural behavior 8 Figure 2.1: Histogram representation of a task execution times. Figure 2.2: CDF and 1-CDF representation of a task execution times. An alternative characterization of a discrete random variable is in terms of probability mass function (pmf), denoted as f Ci, where ( ) c 0 f Ci = i = c min i c 1 i c k i i = c max i f Ci (c min i ) f Ci (c 1 i ) f Ci (c max (2.1) i ) with the property k i j=0 f C i (c j i ) = 1. The relation which connects the CDF with the respective pmf is F Ci (c) = c x=0 f C i (x). Besides, we can define the inverse Cumulative Distribution Function (1-CDF) as F C i (c) = 1 c x=0 f C i (x). A representation in terms of 1-CDF can outline exceedence thresholds as P {C i c}. Figures 2.1 and 2.2 show examples of execution time distribution, first through its histogram representation and then through the corresponding CDF and 1-CDF. Finally, in order to express the composition between random variables, we need to define the joint probability as the probability of two events happening

18 Chapter 2. The probabilistic natural behavior 9 conjointly: P{X = x and Y = y} given by P{Y = y X = x} P{X = x} or equivalently P{X = x Y = y} P{Y = y}. In a real-time perspective, we can interpret this probabilistic model considering a task τ i which generates a number of subsequent jobs τ i,j, with j = 1,...,. Each one of these jobs has an execution requirement described by C i, and, for each possible value c k i, the pmf f Ci (c k i ) describes its probability of occurrence within the execution streamline. The considered task model is τ i = (Φ i, C i, T i, D i ), with Φ i being the task release time, C i the distribution of possible execution times and associated probabilities, T i the maximum inter-arrival time and D i its relative deadline. The probability distribution of execution time obtained through exhaustive measurements of a task τ i (i.e. the empirical distribution) is called Execution Time Profile (ETP) [37], and will be indicated as C i Handling dependences In order to model unpredictabilities applying a probabilistic analysis, we want to propose a probabilistic representation to real-time parameters, introducing some preliminary notions first. An important concept is that of independence and dependence between random variables. Definition 2 (Independent random variables) Two random variables X and Y are said to be independent if they describe two events such that the occurrence of one event does not have any impact on the occurrence of the other event. Definition 3 (Dependence ) Two random variables X and Y are dependent (X Y, or equivalently Y X ) every time there is not independence, thus F X Y (x y) F X (x). These definitions can be enriched including that, in case of independence, the joint probability of X and Y (defined in Section 2.1) has a CDF which is given by the product of the individual CDFs: F X,Y = F X F Y. (2.2) In case of dependence, instead, it remains in its generic form: F X,Y (x, y) = F X Y (x y)f Y (y) = F Y X (x y)f X (x). (2.3)

19 Chapter 2. The probabilistic natural behavior Handling dependences within real-time systems The dynamic environment that characterizes realistic real-time systems is responsible for multiple dependences between system components: in such situations, the ETPs result to be affected by various effects (or causes), such as cache interference, scheduling policy, preemptions caused by concurrently executing tasks etc. Consequently, the worst case, both in deterministic and probabilistic terms, is modified. Our objective is to study how the execution time distributions vary in presence or absence of such effects. A real-time system can be seen as a set of events O = {O 1, O 2,... O n }, where each O j contributes to determine the configurations the system may assume during its evolution. The execution of a task mainly depends on the code that implements it. Aside that, but not marginally, there are other system elements which influence the task execution and consequently its execution time. An event O j affects C i if it is capable of modifying the execution time of the task and its corresponding distribution. Narrowing the system view to comprehend only the effects influencing C i, the system representation becomes O i = {O i 1, O i 2,...}, (2.4) where O i is the system view with respect to C i and its effect over C i. In the following, we will simplify the notation, referring to O and O j to indicate respectively the system representation from τ i and the effects affecting C i. For two events such as C i (the task execution time) and O j, the effect of O j on C i is described by the conditional probability P(C i O j ) = P(C i O j ), (2.5) P(O j ) as the probability of having the execution time C i once O j happens. def def We define C i O j = C i,oj and C i O j = C i,oj, in order to put together the task execution time and the effect of O j on it. In general, C i,oj C i, due to the dependence relationship between C i and O j. The following lemma states an important relation of independence which links the execution time conditional distribution with any system event. Lemma 1 (Independence of the CDF) Assuming two dependent execution time random variables C i and C k, the random variable C i C k is independent from C k and F Ck = F Ck C i C k.

20 Chapter 2. The probabilistic natural behavior 11 Proof: With C i and C k dependent, it is: F Ci,C k = F Ci C k F Ck = F Ck C i F Ci. Considering the execution time random variable C i C k, being F Ci C k its CDF representation, it is: F Ci C k,c k = F Ci C k C k F Ck = F Ck C i C k F Ci C k. Since F Ci C k C k = F Ci C k, F Ci C k,c k = F Ci C k F Ck, stating the independence between C i C k and C k. Furthermore, F Ci C k C k F Ck = F Ck C i C k F Ci C k = F Ci C k F Ck, (2.6) so that F Ck = F Ck C i C k. The lemma follows. With the improved description C i,oj of the task execution, it is also possible to conclude about the independence of the execution time conditional distribution with respect to any system event O j. As an example, thanks to Lemma 1, we have that, with C i C k, C i,ck is independent from C k. In fact, considering an event O j, once its effects are included inside the conditional distribution of execution times of the main task C i,oj, they will no longer affect it, i.e. the main task will run independently from that event. Obviously, Lemma 1 can be generalized to consider any kind of distribution, not only execution time distributions. It is worth noting that a very wide variety of events can fit in the set O: this mostly depends on the abstraction level we may want to consider. For instance, the presence of another task τ j running together with the main one could be represented at a finer level of detail as the distribution of execution times it generates. The main consequence of this reasoning is that any event could have an associated probability distribution function, even trivial, such as a binary random variable. What matters to ensure the applicability of Lemma 1 is the existence of that distribution, regardless of its shape Dependence Composability The execution time of a task τ i, C i,o, takes in account the effects due to all the events O j. Equations (2.4) and (2.5) suggest how it is possible to decompose the dependence problem into sub-problems, according to the desired level of granularity. Within the set O, the events may be related either in terms of inclusion, with O j O j+1, or in terms of partitions, where O j happens independently from O j+1. In our analysis we consider the first one as the most common case, and we will focus on such kind of composition relation while performing our experiments.

21 Chapter 2. The probabilistic natural behavior 12 Then, in the probabilistic framework we have created, we can state the following definitions of isolation and not-isolation. Definition 4 (Isolation) The isolation is the system configuration where the task τ i experiences no effects from the rest of the system, i.e. the set O is empty, and the corresponding ETP does not receive any interference by other system elements. Definition 5 (Not-isolation) A system is said to be not-isolated if the ETP of task τ i is influenced by other system elements, i.e. O = {O 1, O 2,...}. In Section we will see how to instantiate the isolation and the notisolation for realistic real-time systems, where running in isolation does not mean necessarily that the set O is empty, but that it contains the fewest possible system elements. In Section 3.1 a classification of the events that can affect task execution will be proposed. 2.3 Characterization of dependences The probabilistic approach to dependences usually employs the concept of copulas [3]. In this work, we re-formulate the basics of copulas for characterizing dependences in complex real-time systems. In a first approximation, a measure of dependence can be seen as a distance between two random variables. Then, a difference i between two execution time distributions C i and C i could be represented in terms of 1- CDF as i = max x F C i (x) F C i(x). A meaningful difference among 1-CDFs would be the one that considers the shift of a distribution, i,oj def = min{ c F C i (c ) F C i,oj (c)}. (2.7) i,oj is the shift that must be applied to a distribution obtained when O j is not present, in order to upper-bound a distribution influenced by the happening of O j. This quantity is able to characterize the effect that O j has on τ i and its execution time; hence it is a measure of the dependence between O j and C i, quantifying the effect of O j on the execution time of τ i. In other words, it can be seen as the cost to create independence between C i and O j. Exploiting this distance, it is possible to create a distribution which upper-bounds the dependence between C i and any of the events O j composing O. Such distribution C i,o j can be defined in terms of 1-CDF as F up C i,o j (c) def = F C i (c i,oj ). (2.8)

22 Chapter 2. The probabilistic natural behavior 13 We have now a distribution able to bound the independent case C i O j. The following theorem states the existence and the validity of such upper bound. Theorem 1 (Independence Bound) Let C i and O j be two events within a system, with the execution time distribution C i without the event O j and a second execution time distribution C i,oj when O j happens. Then, the distribution C i,o j, such that F up C (c) = F i,o C i (c i,oj ) with i,oj = min { c F C i (c ) j F C i,oj (c)}, represents an upper-bound to the independence case between C i and O j, C i O j. Proof: To prove the theorem, we distinguish three cases. 1. i,oj = 0: in this case C i is independent from O j, c F C i O j (c) = F C i (c), and no influence comes from the event O j to the task execution time C i. Then c, F up C i,o j (c) = F C,O j (c) = F C i (c), upper-bounding such independence case. 2. i,oj > 0: in this case c, F C i O j (c) > F C i (c) and F C i (c) < F C i O j (c) F up C i,oj (c). (2.9) The second inequality holds because by definition F up C (c) def = F i,o j C i (c i,oj ), and since i,oj is the minimum such that F C i (c ) F C i,oj (c), then also F C i (c i,oj ) F C i,oj (c), c. 3. i,oj < 0: in this case c F C i,oj (c) < F C i (c), and F C i O j (c) F up C i O j (c) F C i (c), (2.10) where the first inequality holds by combining Equations (2.7) and (2.8), and the second one comes from the fact that i,oj < 0. Equation (2.10) suggests that F C i (c) is a safe, although pessimistic, bound to the independence case C i O j, while F up C i O (c) is a still safe but more accurate j upper-bound. The importance of Theorem 1 lies in the fact that it allows to reconstruct the independence case between tasks by accounting for all the effects a task can suffer in a realistic system. With such bound, it is possible to safely carry on the probabilistic real-time analyses preserving the independence hypothesis, as in [32, 39].

23 Chapter 2. The probabilistic natural behavior Probabilistic distance analysis The i,oj defined in Equation (2.7) is the result of a complex combination of effects that take place within a system, so it could be seen as a function of all the system interferences it is subdue to: i,oj = c(cache) + b(bus) + m(memory) + o(others) (2.11) This implies that as i,oj has been defined for the execution time distributions, analogous quantities can defined for all the other empirical distributions which are contributing to influence the ETP, with the aim of correlating the observed profiles. By refining Equation 2.7, we define correspondent metrics for the cost of certain events on the cache misses profiles, at any cache level we would consider: cache i,o j represents the cost produced by the event O j on cache misses, and can be found as the composition of many contributions. Assuming L 1,i to be the measured profile of cache misses in L1 cache for task τ i, whenever O j is active it is: L1 i,o j def = min{ l F L 1,i (l ) F L 1,i,Oj (l)} (2.12) L 1,i is is the L1 misses distribution for the isolation case (the reference case) and L 1,i,Oj is the L1 misses distribution every time O j is active. Similarly, and L2 i,o j def = min{ l F L 2,i (l ) F L 2,i,Oj (l)} (2.13) L3 i,o j def = min{ l F L 3,i (l ) F L 3,i,Oj (l)}, (2.14) being L 2,i and L 3,i, respectively, the cache misses profiles for L2 and L3 cache memories. Whenever there are multiple configurations to be compared, the upperbound distribution results as an envelope among the upper-bounds for specific configurations: F up def i,o (x) = max{ O j O F up i,o j (x)} (2.15) The envelope F up i,o (x) upper-bounds the distributions from any single event O j O. The bound is valid only restricting to those specific cases and is able to bound the independence from the events belonging to O. The idea behind system-awareness is to enrich O with effective system events.

24 Chapter 2. The probabilistic natural behavior The probabilistic WCET Within a probabilistic scenario, the concept of deterministic WCET is replaced by the notion of probabilistic Worst-Case Execution Time (pwcet). The main difference between the two concepts is that the latter is a distribution, not a scalar value. In particular, we want to provide a probability distribution which gives the exceedence value of execution time that has a probability of occurrence lower than a required acceptance threshold. To formalize the concept of pwcet, we introduce a preliminary definition. Definition 6 (Greater than or equal to, [32]) Let X and Y be two random variables. Y is greater than or equal to X (or alternatively, X is less than or equal to Y), denoted by Y X (alternatively, Y X ), if P {Y v} P {X v} for any v (alternatively P {Y v} P {X v} for any v). In terms of the CDF representation, Y X (Y X ) if F Y F X (F Y F X ). With the 1-CDF representation, Y X (Y X ) if F Y F X (F Y F X ). Definition 6 establishes a partial ordering between random variables, and will allow to make a comparison between the ETP and the pwcet distribution: in particular, the pwcet must always be greater than or equal to any ETP, ensuring its safety. A formal definition of the exact pwcet C is the following. Definition 7 (probabilistic Worst-Case Execution Time Ci ) The pwcet Ci of a task τ i is defined as the least upper bound on all the distributions C j i of the execution time of τ i, where C j i is generated for every possible combination j of input data to the program by running it an infinite number of times. Thus j, Ci C j i. In terms of CDF, F Ci (x) = min j(f C j(x)) and the 1-CDF i is F C (x) = max j(f (x)). i C j i C i is guaranteed to exist knowing all the possible ETPs C j i for the task τ i. As in the deterministic world, measurement approaches cannot ensure complete coverage of all the execution conditions. Hence, we must produce a safe upper bound C i to the exact pwcet C i. The counterpart of this reasoning applied to the deterministic WCET computation envisions the classical notions of W CET exact, which is in general unknown, and W CET predicted, which is a safe bound to the exact WCET, determined either statically [2, 28, 11, 10] or with some extreme value probabilistic approaches [16, 17, 27].

25 Chapter 2. The probabilistic natural behavior 16 The advantage of the probabilistic setting is that it is extremely easy to compute distributions C i from appropriate ETP measurements, and the safety of such estimations can be ensured through analytical techniques such as the rare events theory. The forthcoming section explains how to get a safe pwcet estimation starting from measurements. 2.5 Extreme Value Theory The Extreme Value Theory (EVT) is a branch of statistics which aims to analyze the tail-end behavior of a distribution, also when there are not enough samples to actually observe those rare events [27]. Applied to pwcet estimation, EVT can compute the probability of occurrence of extreme values of execution times [16], in particular exceedence probabilities which are smaller than 10 n, where n is the required level of confidence. More precisely, it allows to predict the distribution function for the maximal (worst-case) or minimal (best-case) values of a set of n observations, which are modeled as random variables [13]. EVT is analogous to the Central Limit Theorem [23], but it estimates the extremes instead of the average [19]. The main result of EVT is provided by Theorem 2. Theorem 2 [23] Let {X 1, X 2,..., X n } be a sequence of i.i.d. random variables and let M n = max{x 1, X 2,..., X n }. If F is a non degenerate distribution function and there exists a sequence of pairs of real numbers (a n, b n ) such that a n 0 and lim n P ( Mn bn a n x) = F (x), then F belongs to either the Gumbel, the Frechet or the Weibull family. In order to apply Theorem 2, the two hypotheses that must be verified are: that the n random variables are independent and identically distributed (i.i.d.) the existence of the sequence of real numbers (a n, b n ). The Gumbel, Frechet and Weibull families are particular cases of the Generalized Extreme Value (GEV) distribution, whose CDF can be expressed as: { x µ (1+ξ e σ ) 1 ξ ξ 0 F ξ (x) = x µ e σ e ξ = 0

26 Chapter 2. The probabilistic natural behavior 17 Determining the three parameters that characterize this CDF, i.e. the shape (ξ), the scale (σ) and the location (µ), it is possible to prove the existence of the sequence of real numbers (a n, b n ). As regards the i.i.d. hypothesis, we detail the meaning of the two notions with Definitions 8 and 9. Definition 8 (Independent Random Variables) Two random variables X and Y are independent if they describe two events such that the occurrence of one event does not have any impact on the occurrence of the other event. Definition 9 (Identically distributed Random Variables) A sequence of random variables is identically distributed if all random variables have the same probability distribution. The hypothesis of independence (i.) is the most difficult to guarantee while working with realistic platforms. The presence of any dependence in a series of execution times influences the extremal behavior of the execution time. Nonetheless, it is possible to relax this hypothesis when dealing with processes satisfying a weak mixing condition [38]: in such cases the degree of long-term dependence at extreme levels is limited, and EVT still constitutes a safe bound. The identical distribution (i.d.) hypotesis states that consecutive observations must produce empirical distributions which repeat identically, in such a way that EVT can have enough information to model accurately the expected rare events from the measurements. This constraint is far weaker than the usual deterministic repeatability [40]. Identical measurement-based distributions may be identical even if every single measure is not repeatable: this constitutes a key advantage of pwcet over deterministic WCET. In this analysis we use the Gumbel distribution to describe the tail behavior of execution times, because previous applications of EVT, such as [27, 16], have shown that the Gumbel distribution fits well the problem of WCET estimation. For this kind of distribution we have that the shape parameter ξ = 0. Obviously, to ensure the validity of the selected CDF, a goodness-of-fit test is needed. Such validation is performed applying the Exponential Tail (ET) test [22] Steps in the application of EVT Once the hypotheses for the correct application of EVT are verified, two steps are performed in order to apply it:

27 Chapter 2. The probabilistic natural behavior 18 Grouping: from the original distribution, we must collect the values which fit the tail. For this purpose, the measured values are grouped into blocks of equal size, according to the block maxima method. Within each block, we consider the maximum observed value as a good candidate for describing the tail-end behavior of the distribution, so to build a new sample, called block maximum series. The size of the block must be determined as a tradeoff between the precision in fitting the tail and the number of values in the final sample: on one hand, having a large block size allows a better fitting; on the other hand, it results in a smaller number of values in the distribution of block maximum values. Moreover, Theorem 3 gives a theoretical basis to the application of the block maxima method. Theorem 3 [17] [Grouping] Let X be a random variable. If the n variables X 1, X 2,..., X n obtained from X fit a GEV, then the maxima of any blocks obtained from X fit the same GEV. Distribution fitting: if the ET test validates the fitting to a Gumbel distribution, we can estimate the parameters µ and σ using the block maxima series. This can be done applying linear regression to the quantile plot (QQplot) of our data [18]. A QQ-plot relates empirical quantile values with quantiles of the standard form of a target distribution. When the fitting is verified, the points on the QQ-plot form a straight line. The parameters µ and σ are then retrieved as the slope and intercept of the best-fit line through these points.

28 Chapter 3 Observing real-time systems This chapter will introduce the methodology and the requirements needed to appropriately execute and measure real-time tasks. Besides, we will expose how the requirements have been satisfied in our specific case, illustrating the main tools that have been adopted for this work, together with the motivations underlying our choices. 3.1 System insight We classify the elements that concur in modifying the execution profile of a task in three categories: 1. Systemic, i.e. all the elements deriving from the system settings, such as enabling of prefetch, presence or absence of system interrupts and all the other features that concur to create the specific notion of isolation; 2. Interactions, including all the contributions which create interferences between concurrently running tasks, mostly preemptions. We also have to take in account when and how preemptions can take place: they can be nested, displaced by different offsets, more or less consistent, single or repeated during a period etc.; 3. Resources, i.e. architectural components such as the presence of different cores, their shared memory, etc. The ETPs we obtain when running a task multiple times are influenced by the composition of the mentioned causes. We aim to study what is the behavior of a system when such elements interact and combine through the cache memory. Cache misses penalties are 19

29 Chapter 3. Observing real-time systems 20 HW multi cores,... System Cache Interactions preemptions,... ET isolation, prefetch,... Figure 3.1: System-aware execution time perspective. in fact one of the main sources of overhead for task execution [14, 1]: due to concurrent activities, the content of the cache memory can be altered over time. The cost of a cache miss is quantifiable with the time needed to replace the cache memory space with useful information. Of course, cache memory is not the only way tasks can interact, because in order to have a complete view we should include other components, such as the bus contention, the context switch overhead etc., but in this work we will concentrate on the interactions that can take place within the system due to the presence of the cache. Figure 3.1 sketches a conceptual representation of such interactions. The three mentioned categories of elements and their effects have been examined in detail through appropriate experiments, whose outcome is exposed in Chapter Requirements Trying to measure and observe a real-time system requires to set up an appropriate environment which allows to execute various real-time task-sets and to collect information about their behavior at each iteration. In particular, we need a real-time environment which ensures usual realtime execution with appropriate scheduling policies, memory locking, core affinity, etc. Moreover, it is required to have a tracing tool which allows to collect timestamped traces containing execution information, such as execution time, response time, number of cache misses, etc. The objective of our setup is to address any kind of real execution target. Thus, we cannot rely on simulation, neither a cycle accurate one, nor any hardware-specific assisted observation, because they may be not available. The best option is to perform instrumented real execution, often called tracing, ensuring a minimal perturbation due to real-time measurements. Performance monitoring tools usually adhere to one of the following paradigms: static tracing, which is usually performed by adding logging statements

30 Chapter 3. Observing real-time systems 21 to the code and compile them with the program, or through preprocessor directives or compiler command-line options; dynamic tracing, which envisions runtime patching of the code, modifying assembly instructions to add jumps to trace code. The second approach, involving a costly breakpoint interrupt, seems to be unsuitable to support real-time critical systems. Hence, a static instrumentation appears to be the best choice to achieve low overhead, at the only payback of a small increase in the code size. In order to conduct low-level performance analysis or tuning, retrieving counts of hardware-related activities, hardware performance counters are a common choice, since they are available in most of modern processors, also in the embedded computing area. Hardware performance counters are a set of special-purpose registers built into modern microprocessors that track lowlevel operations or events within the processor accurately and with minimal overhead. The number of available hardware counters in a processor is limited (4 in our case), and each CPU model could have a lot of different events that we would like to measure. Each counter can be programmed with the index of an event type to be monitored, like a L1-instruction cache miss or a branch misprediction. Compared to software profilers, such as gprof and Valgrind, hardware counters provide low-overhead access to a wealth of detailed performance information related to CPU s functional units, cache memories, main memory etc. Besides, in general no source code modifications are needed, although the types and the meaning of counters may vary depending on the architecture, due to the different hardware organizations. Finally, it is very important to have a precise characterization of the benchmark set we wish to adopt, in order to quantify in advance the runtime program behavior. To this aim we explored MICA 1 [29], a PIN tool 2 which allows to collect program characteristics independently from the microarchitecture on which the measurements are done. The following Sections will instantiate for our specific case the previously described requirements: SchedMCore will be used as real-time environment, and LTTng as tracing tool. Moreover, an overview of the specific hardware architecture and the explored configurations will be given

31 Chapter 3. Observing real-time systems The SchedMCore framework What is SchedMCore? SchedMCore [12] 3 is an open source integrated development framework for critical embedded systems, made by ONERA 4. It is provided as a full operational toolset with two main components: the SchedMCore Converter the SchedMCore Runner The SchedMCore Converter The SchedMCore Converter takes as input a task model and generates a formally analyzable model in C or UPPAAL, which encodes as a finite automaton all the possible execution sequences of the system. Then, either using UPPAAL model checker or the obtained custom C program, the automaton is explored in order to retrieve the executions which violate the system constraints. This automaton can also be used in order to generate off-line schedules, which can even satisfy optimality requirements for fixed-priority assigments. The scheduling policies which SchedMCore implements are able to cope with multiprocessor platforms and consider preemptive policies accepting (full) migration, which means that a suspended task can also be resumed on another processor. In particular, the implemented policies are: FP (Fixed Priority): tasks are assigned offline a specific priority; gedf (global Earliest Deadline First): multiprocessor extension of EDF, where priorities are assigned dynamically considering absolute deadlines; gllf (global Least Laxity First): multiprocessor extension of LLF, where priorities are assigned based on the remaining execution time; LLREF (Largest Local Remaining Execution First), which mixes fair execution and local laxity. According to the specific policy and the given architecture, SchedMCore Converter performs offline schedulability analysis, determining whether the desired taskset can be executed respecting the imposed real-time constraints. It has been used to ensure that the used task-set was theoretically schedulable

32 Chapter 3. Observing real-time systems 23 The SchedMCore Runner The SchedMCore Runner constitutes the execution environment (or runtime) tool which is capable of running a set of real-time tasks, with the previously mentioned scheduling policies. It is realized as a pluggable scheduling framework which envisions precise real-time execution on a multicore target architecture. With this tool, all the required real-time support can be provided: memory locking, switching to real-time scheduling policy, core affinity and so on. The SchedMCore Runner requires as inputs: The taskset description, which can be either a simple descriptive text file or a dynamic library produced by the compilation of a C file generated by Prelude [36]; The number of cores to be used, and, possibly, a core affinity mask; The scheduling policy to be used: besides the ones listed above, the user is allowed to create his own customized policy, pluggable into the runtime system. The SchedMCore runtime, used by the SchedMCore Runner, is implemented as a user-space library, in order to avoid the burden of going inside the OS kernel. In particular, its evolution does not need to follow the underlying operating system, allowing easy maintenance and portability, without the need of patching the Linux kernel. The same approach has been used in the past [30], and even more recently [35] it has been proven efficient. The runtime architecture consists of two management threads: The scheduler skeleton, which handles the communication with the OS and task accesses to system resources. Its activity is controlled by a semaphore, which allows the task to preempt other tasks. It is the thread which performs the necessary scheduling actions, invoking three mandatory functions: initialize(), schedule(), finalize(). The implementation of these functions is provided as a dynamic library, thus it is very easy to provide new pluggable schedulers; The time manager, entity which is periodically woken up by the operating system and unlocks the scheduler skeleton semaphore to trigger the scheduling operations. The design of SchedMCore requires at least five OS priority levels. The first is assigned to the time manager task, the second one is reserved to the generic scheduler task, the third is corresponding to the thread in execution,

33 Chapter 3. Observing real-time systems 24 D D τ 0 τ 1 C C Φ T T Figure 3.2: Task model representation. while the idle tasks 5 are assigned the fourth level. Finally, the last one is used by the main program. The SchedMCore task model SchedMCore adopts the classical periodic task model as a standard adaptation from [31], with a task set S = {τ i } i=1,...,n, where each task τ i has four real-time parameters (Φ i, C i, T i, D i ), with Φ being the task release time, C i the task worst case execution time estimation, T i the maximum task interarrival time and D i its relative deadline, where D i T i. Figure 3.2 clarifies the meaning of such parameters. The k th instance of τ i, defined as a job, is denoted with τ i,k. Since we deal with periodic tasks, its release time will be φ i,k = Φ i + kt i. We can also define the hyperperiod as the least common multiple of all the task periods, i.e. H = lcm 1 i n {T i }. Besides, it is possible to impose precedence constraints relating the execution order of tasks: a simple precedence constraint τ i τ j assumes that τ i and τ j have the same period and imposes that each job of τ i must be executed before one job of τ j ; an extended precedence constraint τ i,n τ j,m relates, still in terms of precedence, the n th instance of τ i with the m th instance of τ j, so in a sense it can be seen as a generalization of the previous case [36]. In order to specify the involved tasks, their real-time parameters, code and relations, the two SchedMCore main tools take as input the same file formats: either a simple textual description or a Prelude library [36]. In this work, we will use textual descriptions as inputs for the tools. In particular, the SchedMCore task model has been extended in order to associate to a task any kind of C function, which implements the real-time task [33]. 5 A task can be idle for different reasons: it may be blocked by some precedence constraint, not released yet, or, even if ready to execute, all the cores may be already occupied.

34 Chapter 3. Observing real-time systems A measuring tool: LTTng The performance monitoring tool we have chosen, LTTng (Linux Trace Toolkit new generation) [20] is a Linux framework which allows static collection of timestamped traces, for both kernel and user events. It adheres to the paradigm of static tracing, guaranteeing low overhead and good performance in supporting critical activities. LTTng also provides a new powerful feature, namely the possibility of appending to trace events Perf performance monitor counters [41]. This feature allows to monitor with great precision the behavior of the cache. In particular, in our measurements we use hardware performance counters for counting cache misses, and we try to limit as much as possible the amount of collected data, in order to minimize perturbation and system disturbance. 3.5 Architecture Hardware features All the experiments have been run on a machine having two Intel R Xeon R E GHz sockets, each one with four cores and three levels of cache. The first two levels (L1 and L2) are per core, while the last level (L3, or, equivalently, LLC) is per socket, hence shared among each group of four cores. The L1 cache is partitioned into 32 KB data cache and 32 KB instruction cache, while L2 and L3 (respectively 256 KB and 12 MB) are unified in terms of data and instructions. Figure 3.3 shows in detail the organization of the hardware platform System settings Since we want to address realistic real-time systems, the notion of isolation that we gave in section with Definition 4 is instantiated so to eliminate all the system elements that may influence the task execution, but those which are strictly necessary to ensure the correct behavior of the system. The isolation case represents the the reference against which all the other profiles are compared. However, the notion of isolation itself is subject to variability, because many are the configurations that can be regarded as isolation, although being very different one from the other. This means that, for the isolation case, the set O of events affecting the task execution time could be not empty, which is a common case for realistic systems, where it is nearly impossible to eliminate all the causes that could affect task execution.

35 Chapter 3. Observing real-time systems 26 Figure 3.3: Hardware architecture There are some system components which cannot be eliminated (such as non-maskable or timing interrupts), so they must be embedded inside the notion of isolation. Some other architectural components can be either kept active or disabled. It is the case of prefetch, general system hardware interrupts and soft IRQs. Our framework can easily treat the different cases, because including some events O 1,..., O n in the definition of isolation ensures, by Theorem 1, that the execution of the tasks becomes independent from those effects. Before fixing a specific notion of isolation, we tested different possible configurations: pref : smallest possible set of interrupts and system effects, enabling of prefetch, no other functional task running on the same core; : same as before, but prefetch is disabled; this will be our reference configuration; irq: same as, but the real-time priority of soft IRQs is raised to maximum level;