Efficient Simulation of Network Performance by Importance Sampling

Transcription

1 Efficient Simulation of Network Performance by Importance Sampling Poul E. Heegaard Norwegian University of Science and Technology Trondheim, Norway Outline Motivation Main contributions SYSTEM SIMULATION MODEL Rare event simulation Importance sampling Adaptive parameter biasing Network simulations Closing remarks Presentation of thesis,

2 Motivation Task: evaluation of system which are characterized by - being large and complex - being distributed with tight logical couplings - have strict quality of service SYSTEM SIMULATION MODEL Evaluation means: - analytic (efficient, but needs oversimplification) - simulation (flexible, but inefficient) - measurements (efficient, but expensive and inflexible) Presentation of thesis,

3 Motivation => Simulation is flexible means for system evaluation, => Speed-up is required, => Importance sampling is efficient, => Optimal/good parameters are essential. e-09 e-09 Insensitive to changes in f * (s) Exact, = 4.90e-09 property of interest 4e-09 e-09 Simulation results e-09 Optimal change of measure Error bars e-09 change of measure This thesis: Can importance sampling be applied in network simulations? Presentation of thesis,

4 Why is network simulation with rare events a problem? - model size and complexity 4 link failures users direct simulation of properties dependent on rare events is very inefficient Restrictive quality of service requirements Speed-up gain [log] e-07 e-0 e Exact blocking [log] Domain of interest (<e-0)! Presentation of thesis,

5 Main contributions - Adaptive biasing for importance sampling in performance simulations of networks - Flexible modelling framework - Network simulation for feasibility demonstrations e-07 blocking probabilities e-08 e-09 e-0 simulation results with error bars exact results e generators - Heuristics importance sampling experiments - Combination of speed-up techniques - Comparisons of rare event techniques - Application of importance sampling to MPEG and ATM Presentation of thesis,

6 Speed-up techniques PARALLEL AND DISTRIBUTED SIMULATIONHYBRID TECHNIQUES COMBINES Parallel and distributed N ANALYTIC SOLUTIONS WITH SIMULAT simulation (PADS) - decomposition in time or space x+ less than N X x x- T SPEED-UPS? N T T T 4 T close to N Parallel independent replicated simulation N X 0 X =x - conditional sampling SYSTEM SIMULATION MODEL VARIANCE MINIMIZATION EXPLOITS RARE EVENT PROVOKING TECHNIQUE KNOWN OR INTRODUCED CORRELATION CHANGE THE SAMPLING DISTRIBUTIO X and Y Y X control variable - antithetic variates - common random number RESTART with 4 levels Original distribution Importance sampling 0 0 Presentation of thesis,

7 Rare event simulation techniques Objective: Change the underlying sampling distribution to provoke rare events of interest to occur more often. Known approaches: - RESTART - Importance sampling Presentation of thesis,

8 Importance sampling f(x)=original model Importance sampling: P f ( gx ( ) = ) «P f ( gx ( ) = ) f * (X)=stressed model Rare event: P f ( gx ( ) = ) «y x Observation: g( X) = Ix ( y), e.g.: -> overflow of MPEG cells in a multiplexer -> blocked calls Relation: E f ( gx ( )) = E f ( gx ( ) ( X) ) where fx ( ) ( X) = the likelihood ratio between f( X) and f ( X) f ( X) n Estimator: X = -- ( X n i ) gx ( i ) i = where X i are samples from f () x Presentation of thesis,

9 Importance sampling heuristics () - Use likelihood ratio as indication of goodness of simulation results - no analytic solution available - no direct simulation results - use knowledge of E( L) = - Two observations from experiments: (i) L and rel.error( L) «=> ˆIS is good if its relative error rel.error( ˆIS )«(ii) L «or rel.error( L) : => ˆIS is poor even if the rel.error( ˆIS ) «. Presentation of thesis,

10 Importance sampling heuristics () - The sampling distribution is heavy tailed under too strong biasing (infinite variance) ^ running mean of IS true value E[ ] R Presentation of thesis,

11 Modelling framework Model feasibilities - different resource requirements - different quality of service requirements - pre-emptive priorities - alternative routing on overload and failures 4 link failures user type A priority= 4 priority= user type B primary route user type B secondary route user type A primary route user type A Objectives: assessment of - blocking probability - rerouting probability - disconnection probability - consequence of pre-emption - consequence of failures Presentation of thesis,

12 Mapping to state space model () Generator Resource pool 0 Presentation of thesis,

13 Mapping to state space model () Generator Resource pool 0 current state generator blocking pool blocking pool 0 => ARGET SUBSPACE 0,0,,,0,,,,0,,,,4 0,0 0, 0, 0, 0,4 generator blocking pool Presentation of thesis,

14 State space model TARGET SUBSPACE (resource constraint, boundary, barrier EVENTS arrival of entity from gen. departure of entity from gen. #entities gen., #entities gen. STATE Presentation of thesis,

15 Importance sampling in network simulations Challenges Multidimensional state models Gen Gen 0 Balanced dimensioning, i.e. no bottlenecks BARRIER 0 Gen Previous solutions to change of measure Gen STATE SPACE MODEL Identify a bottleneck Drift towards the bottleneck barrier Identify the bottleneck and change the measure according to this 0 Fixed change of measure If no single bottleneck => inefficient solution! Gen Presentation of thesis,

16 Adaptive change of measure A new, adaptive approach Towards the most important barrier at current state. State dependent change. A good estimate on the current importance of all barrier is required. Gen importance 0 Choose a barrier direction at random, and change the measure toward this. 0 0 importance 0 Algorithm At each state: (i) Estimate the current importance of all barriers (ii) Choose a direction (iii) Induce drift in the chosen direction. Step (i)-(iii) are repeated for every state. Gen Gen Approached barrier : Choose the barrier direction again according to the new relative importance estimates importance 0 0 Approached barrier 0: Choose the barrier direction again according to the new relative importance estimates Presentation of thesis,

17 Estimation of target importance - Requirements:. Sufficiently accurate. Robust. Efficient - Simplification: - only the relative importance is of interest - use the greatest importance contribution, H j ( ) => must identify the sub-path from current state a state in the target sub-space j. to Presentation of thesis,

18 Efficient search for the sub-path - find the sub-path with the largest contribution to H j ( ) - exploit the Markov properties 0,,., 4,, 0,,,, 0,0,0,0,0 x = ,0,0,0,,, 4,, x c k j k ( ) x c k j k ( ) x x x c k j c k j x c k j k ( + ) k number of resources allocated x c k j k ( + ) k Presentation of thesis,

19 Network example - No priority nor alternative routing e-07 blocking probabilities e-08 e-09 simulation results with error bars exact results e-0 e generators - Compared with exact results - Simulation more efficient than numerical calculations Presentation of thesis,

20 Network example - Improving the quality of service by rerouting (a) With primary route only e-04 blocking probabilities e-0 e-0 e-07 e-08 Upper bounds of blocking e-09 e-0 Simulated blocking probability of generator generators - Simulation results close to rough blocking estimates - Mean likelihood ratio close to with low relative error - Significant speedup over direct simulation observed Presentation of thesis,

21 Network example - Disturbing low priority traffic e-0 e-04 blocking probability e-0 e-0 CASE.: low priority traffic only CASE.: mixed with high priority traffic CASE.: mixed with high priority traffic and exposed to link failures e-07 gen. gen. gen. gen. gen. gen. - Best results for generator and in accordance to the biasing setup of importance sampling - No speed-up compared to direct simulation (loss probability in order of The mean likelihood ratio is 0.74 for case. => overbiased? Presentation of thesis,

22 Closing remarks - importance sampling in performance simulation of telecom networks with: - balanced utilisation of resources - users with different quality of service requirements - preemptive priority and rerouting - link and node failures - new, adaptive importance sampling biasing proposed - flexible modelling framework applied - feasibility demonstrated - heuristics for importance sampling experiments - combination of speed-up techniques - importance sampling in other applications - further development of fundament required before inclusion of rare event techniques in simulation tool Presentation of thesis,