Elementary Landscape Decomposition of the Test Suite Minimization Problem

Transcription

1 Elementary Decomposition of the Test Suite Minimization Problem Francisco Chicano, Javier Ferrer and Enrique Alba SSBSE 2, Szeged, Hungary, September -2 / 26

2 Problem Formalization Test Suite Minimization Given: Ø A set of test cases T = {t, t 2,..., t n } Ø A set of program elements to be covered (e.g., branches) M= {m, m 2,..., m k } Ø A coverage matrix T= Find a subset of tests X T Binary representation: t t 2 t 3... t n m m 2 m k maximizing coverage and minimizing the testing cost SSBSE 2, Szeged, Hungary, September -2 2 / 26

3 Definition Elementary s decomposition Binary Space Definition A landscape is a triple (X,N, f) where Ø X is the solution space Ø N is the neighbourhood operator Ø f is the objective function The pair (X,N) is called configuration space The neighbourhood operator is a function N: X P(X) Solution y is neighbour of x if y N(x) Regular and symmetric neighbourhoods d= N(x) x X y N(x) x N(y) 2 s s7 s2 s 3 4 s6 5 s3 2 s4 s5 Objective function f: X R (or N, Z, Q) 6 s8 s9 7 SSBSE 2, Szeged, Hungary, September -2 3 / 26

4 Definition Elementary s decomposition Binary Space Elementary s: Formal Definition An elementary function is an eigenvector of the graph Laplacian (plus constant) Adjacency matrix Degree matrix Graph Laplacian: s Depends on the configuration space s s7 s2 s3 s5 Elementary function: eigenvector of Δ (plus constant) s6 s4 s8 s9 Eigenvalue SSBSE 2, Szeged, Hungary, September -2 4 / 26

5 Definition Elementary s decomposition Binary Space Elementary s: Characterizations An elementary landscape is a landscape for which Depend on the problem/instance where def Linear relationship Grover s wave equation Eigenvalue SSBSE 2, Szeged, Hungary, September -2 5 / 26

6 Definition Elementary s decomposition Binary Space Decomposition What if the landscape is not elementary? Any landscape can be written as the sum of elementary landscapes f < e,f > < e 2,f > X X X There exists a set of eigenfunctions of Δ that form a basis of the function space (Fourier basis) e 2 Non-elementary function Elementary functions (from the Fourier basis) < e 2,f > f Elementary components of f < e,f > e SSBSE 2, Szeged, Hungary, September -2 6 / 26

7 Definition Elementary s decomposition Binary Space Examples Elementary s Problem Neighbourhood d k Symmetric TSP 2-opt n(n-3)/2 n- swap two cities n(n-)/2 2(n-) Graph α-coloring recolor vertex (α-)n 2α Max Cut one-change n 4 Weight Partition one-change n 4 Sum of elementary s Problem Neighbourhood d Components General TSP inversions n(n-)/2 2 swap two cities n(n-)/2 2 Subset Sum Problem one-change n 2 MAX k-sat one-change n k QAP swap two elements n(n-)/2 3 Test suite minimization one-change n max v i SSBSE 2, Szeged, Hungary, September -2 7 / 26

8 8 / 26 SSBSE 2, Szeged, Hungary, September -2 Regression The set of solutions X is the set of binary strings with length n Neighborhood used in the proof of our main result: one-change neighborhood Ø Two solutions x and y are neighbors iff Hamming(x,y)= Binary Search Space Definition Elementary s decomposition Binary Space

9 Definition Elementary s decomposition Binary Space Spheres around a Solution If f is elementary, the average of f in any sphere and ball of any size around x is a linear expression of f(x)!!! Σ f(y ) = λ 2 f(x) n non-null possible values Sutton H=2 H= Σ f(y ) = λ f(x) Whitley Langdon H=3 Σ f(y ) = λ 3 f(x) SSBSE 2, Szeged, Hungary, September -2 9 / 26

10 Definition Elementary s decomposition Binary Space Bit-flip Mutation: Elementary s Analysis of the expected fitness p= fitness does not change When f(x) > p= the same fitness j Sutton Whitley When f(x) < j Howe Chicano Alba j j p=/2 start from scratch p= flip around SSBSE 2, Szeged, Hungary, September -2 / 26

11 Definition Elementary s decomposition Binary Space Bit-flip Mutation: General Case Analysis of the expected fitness Example (j=,2,3): p.23 maximum expectation p=/2 start from scratch The traditinal p=/n could be around here SSBSE 2, Szeged, Hungary, September -2 / 26

12 ELD of f ELD of f 2 Elementary Decomposition of f The elementary landscape decomposition of Computable in O(nk) is Tests that cover m i constant expression Krawtchouk matrix Tests in the solution that cover m i SSBSE 2, Szeged, Hungary, September -2 2 / 26

13 ELD of f ELD of f 2 Elementary Decomposition of f 2 The elementary landscape decomposition of f 2 is SSBSE 2, Szeged, Hungary, September -2 3 / 26

14 ELD of f ELD of f 2 Elementary Decomposition of f 2 The elementary landscape decomposition of f 2 is SSBSE 2, Szeged, Hungary, September -2 4 / 26

15 ELD of f ELD of f 2 Elementary Decomposition of f 2 The elementary landscape decomposition of f 2 is Computable in O(nk 2 ) Number of tests that cover m i or m i Number of tests in the solution that cover m i or m i SSBSE 2, Szeged, Hungary, September -2 5 / 26

16 Selection Mutation GLS Selection Operator Selection operator Mutation Expectation Expectation Minimizing Mutation X We can design a selection operator selecting the individuals according to the expected fitness value after the mutation SSBSE 2, Szeged, Hungary, September -2 6 / 26

17 Selection Mutation GLS Mutation Operator Mutation operator Given one individual x, we can compute the expectation against p. Take the probability p for which the expectation is maximum 2. Use this probability to mutate the individual If this operator is used the expected improvement is maximum in one step (Sutton, Whitley and Howe in GECCO 2) SSBSE 2, Szeged, Hungary, September -2 7 / 26

18 Selection Mutation GLS Guarded Local Search With the Elementary Decomposition (ELD) we can compute: With the ELD of f and f 2 we can compute for any sphere and ball around a solution: : the average : the standard deviation Distribution of values around the average Chebyshev inequality Best At least 75% of the samples are in the interval Apply local search f SSBSE 2, Szeged, Hungary, September -2 8 / 26

19 Selection Mutation GLS Guarded Local Search With the Elementary Decomposition (ELD) we can compute: With the ELD of f and f 2 we can compute for any sphere and ball around a solution: : the average : the standard deviation Distribution of values around the average Chebyshev inequality Best f Don t apply local search At least 75% of the samples are in the interval SSBSE 2, Szeged, Hungary, September -2 9 / 26

20 Selection Mutation GLS Guarded Local Search: Experimental Setting Steady state genetic algorithm: bit-flip (p=.), one-point crossover, elitist replacement GA (no local search) GLSr (guarded local search up to radius r) LSr (always local search in a ball of radius r) Instances from the Software-artifact Infrastructure Repository (SIR) printtokens printtokens2 schedule schedule2 totinfo replace Oracle cost c=..5 n= test cases k=-2 items to cover independent runs SSBSE 2, Szeged, Hungary, September -2 2 / 26

21 Selection Mutation GLS Guarded Local Search: s c= c=3 c=5 printtokens schedule SSBSE 2, Szeged, Hungary, September -2 2 / 26

22 Selection Mutation GLS Guarded Local Search: s c= c=3 c=5 SSBSE 2, Szeged, Hungary, September / 26

23 Selection Mutation GLS Guarded Local Search: s c= c=3 c=5 SSBSE 2, Szeged, Hungary, September / 26

24 Selection Mutation GLS Guarded Local Search: s Time (secs.) SSBSE 2, Szeged, Hungary, September / 26

25 theory provides a promising technique to analyze SBSE problems We give the elementary landscape decomposition of the test suite minimization problem Using the ELD we can efficiently compute statistics in the neighbourhood of a solution We provide a proof-of-concept by proposing a Guarded Local Search operator using the information gained with the ELD Future Work The main drawback of the GLD is runtime: parallelize computation with GPUs Expressions for higher order moments (ELD of f c ) Remove the current constraint on the oracle cost Connection with moments of MAX-SAT SSBSE 2, Szeged, Hungary, September / 26

26 Elementary Decomposition of the Test Suite Minimization Problem Thanks for your attention!!! SSBSE 2, Szeged, Hungary, September / 26