Covering Arrays. Lucia Moura School of Electrical Engineering and Computer Science University of Ottawa

Transcription

1 School of Electrical Engineering and Computer Science University of Ottawa Conference on Combinatorics and its Applications (CJC65), Singapore, July 2018

2 In the late 90 s covering arrays started gaining attention in software testing and combinatorial design circles: 1996: Neil Sloane wrote an article Covering arrays and intersecting codes : AETG software and several articles by Cohen, Dalal, Fredman, Patton. 1998: PhD thesis of Brett Stevens Transversal packings and covers 2000: PhD thesis of Marc Chateauneuf Covering arrays Excellent survey by Colbourn (2004): Combinatorial aspects of covering arrays

3 Orthogonal arrays Definition: Orthogonal Array An orthogonal array of strength t, k columns, v symbols (and index 1) denoted by OA(t, k, v), is a v t k array with symbols from {0, 1,..., v 1} such that in every t N subarray, every t-tuple of {0, 1,..., v 1} t appears exactly once as a row. Example: t = 3; v = 2 symbols; k = 4 columns

4 Covering arrays Strength t = 3; v = 2 symbols; k = 10 columns; N = 13 rows Definition: Covering Array A covering array of strength t, k columns, v symbols and size N, denoted by CA(N; t, k, v), is an N k array with symbols from {0, 1,..., v 1} such that in every t N subarray, every t-tuple of {0, 1,..., v 1} t appears at least once as a row. Each l-set of columns with l-tuple of values for those columns is called an interaction.

5 Covering arrays Strength t = 3; v = 2 symbols; k = 10 columns; N = 13 rows Definition: Covering Array A covering array of strength t, k columns, v symbols and size N, denoted by CA(N; t, k, v), is an N k array with symbols from {0, 1,..., v 1} such that in every t N subarray, every t-tuple of {0, 1,..., v 1} t appears at least once as a row. Each l-set of columns with l-tuple of values for those columns is called an interaction.

6 CAs: applications in software and hardware testing

7 CAs: generalizations useful for applications mixed alphabets: different columns/parameters have different number of values Ex: 3 values for OS, 4 values for browser, etc. forbidden interactions: some combinations of values are invalid. Ex: If OS is mac then browser can t be internet explorer. variable strength: Ex: some interactions need not be tested; some sets of parameters require higher strength/coverage. error location: Ex: stronger types of covering arrays with more redundancies that allow us to know which interactions caused the errors

8 Application example: mixed alphabets (picture from Martinez, Moura, Panario, Stevens 2008)

9 Application example: variable strength a NAND x b XNOR y c AND d XOR z NOT AND a x e b y c d z e a b c d e test test test test test test test test see Raaphorst, Moura and Stevens (2018)

10 Covering arrays number and lower bounds covering array number CAN(t, k, v) = min{n : CA(N; t, k, v) exists}. Trivial lower bound CAN(t, k, v) v t CAN(t, k, v) = v t if and only if there exists an OA(t, k, v). Non existence of OA(t, k, v) implies CAN(t, k, v) > v t. Examples: If k > v + 1, CAN(2, k, v) > v 2. If k > v + 2, CAN(3, k, v) > v 3. Other lower bounds - Stevens, Moura and Mendelsohn (1998) CAN(2, k, v) v log k 2 CAN(2, k, v) min{n : k 1 ( 2 n ) v +δt,g 2 } n v

11 : asymptotic growth covering array number CAN(t, k, v) = min{n : CA(N; t, k, v) exists}. As k, CAN(2, k, v) = v 2 log k(1 + o(1)). (Gargano, Korner and Vaccaro 1994) CAN(t, k, v) v t (t 1) ln k(1 + o(1)) (Godbole, Skipper and Sunley 1996) Therefore, logarithmic growth For fixed v and t, letting k, CAN(t, k, v) = O(log k)

12 Density-based Greedy Algorithm greedy algorithms; can handle mixed alphabets, variable strength. For t = 2 Cohen, Dalal, Fredman and Patton (1997) propose a greedy algorithm with logarithmic guarantee on the size of the array (basis for their AETG software). It uses O(log k) steps but each step requires to solve an NP-complete problem which they approximate with a heuristic. So they either sacrifice the logarithmic guarantee or the polynomial time. Colbourn, Cohen and Turban (2004) introduced the concept of density and give a polynomial time algorithm with logarithmic guarantee for t = 2. Bryce and Coulbourn (2008) generalize this algorithm for general t. (polytime; logarithmic guarantee). Raaphorst, Moura and Stevens (2012) adapt algorithm to work with variable strength. (polytime; logarithmic guarantee)

13 Density algorithm: experiments for variable strength Strength Hypergraph = Steiner triple system of order k k # g CA(2, k, g) N m N M CA(3, k, g)

14 Constructions Colbourn, Combinatorial Aspects of, Colbourn, Covering arrays in the Handbook Comb Des, Colbourn, Covering array tables, online. CAN(2, k, 2) = min{n : k ( N 1) } N 2 Construction: each column is the incidence vector of a N 2 -subset of {1,... N} not containing one fixed element any OA(t, k, v) gives CAN(t, k, v) = v t Direct constructions: algebraic, using CPHF, etc Recursive constructions: product, etc.

15 Covering array tables by Colbourn t = 2,..., 6, v = 2,..., 25 (picture from Colbourn et al. 2005)

16 Mixed alphabets t = 2 Moura, Stardom, Stevens, Williams (2003) solve mixed alphabets for k 4; generalize several constructions including the basic product construction. Colbourn, Martirosyan, Mullen, Shasha, Sherwood, Yucas (2005) optimize product construction by taking advantage of don t care symbol.

17 Mixed alphabets t = 2 (cont d) (picture from Coulbourn et al. 2005)

18 Error location Colbourn and McClary (2008) is a fundamental paper about error location. They define locating arrays and detecting arrays. detecting arrays An array A is (d, t)-detecting if whenever I is a t-way interaction and I 1 is a set of t-way interactions with I 1 = d, it holds that ρ(a, I) ρ(a, I 1 ) I I 1 An array A is (d, t)-detecting if we allow interaction sizes t and I 1 d. CAs and MCAs with higher strength are detecting arrays Every CA(N; t + d, k, v) with d < v is a (d, t)-detecting array. Every MCA(N, t + d, v 1 v 2... v k ) with d < v 1 v 2... v k is a (d, t)-detecting array.

19 Error location (cont d) For a (d, t)-detecting array to exist we need d < v, because there exist non-locatable sets of errors when d v. Martinez, Moura, Panario and Stevens (2008) generalize detecting arrays by considering specific classes of error hypergraphs. locatable hypegraph Let H t,k,v be a k-partite graph with k parts of size v. A hypergraph H = H t,k,v is t-locatable if every t-way interaction I is locatable with respect to H, that is: for each I = {(f 1, a 1 ),..., (f t, a t )} there exist a k-tuple (T 1,..., T k ) that agrees with I and avoids H. error locating arrays An error locating array for hypergraph H = H t,k,v, denoted ELA(n, t, H), is an n k array A such that each t-way interactions is located by some row of A.

20 Error location (cont d) error locating arrays An error locating array for hypergraph H = H t,k,v, denoted ELA(n, t, H), is an n k array A such that each t-way interactions is located by some row of A. error locating arrays for a class of hypergraphs An error locating array for a class of hypergraphs H = H t,k,v, denoted ELA(n, t, H), is an n k array A that is an ELA(n, t, H) for each H H.

21 ELAs error locating arrays for a class of hypergraphs An error locating array for a class of hypergraphs H = H t,k,v, denoted ELA(n, t, H), is an n k array A that is an ELA(n, t, H) for each H H. Let H = H t,k,v be a class of hypergraphs with edge cardinality t. if all hypergraphs in H have d edges, an ELA(n, t, H) is the same as (d, t)-detecting. if all hypergraphs in H are locatable and t(d + 1) k, then any CA(n; t(d + 1), k, v) is an ELA(n, t, H). if all hypergraphs in H have safe values and t + d k, then any CA(n; t + d, k, v) is an ELA(n, t, H).

22 ELAs and important relations to CAFEs Locating a hypergraph is closely related to CAs with forbiden interactions. If H is locatable then ELAN(n, t, H) = CAF EN(n, t, H) E(H). For hypergraphs with safe values both ELAs and CAFEs can be constructed via combinatorial group testing for complexes: 1 Build a covering array CA(n; t, k, v), say A. 2 For each row R of A that covers an edge, run the adaptive algorithm by Chodoriwski and Moura (2015) for group testing for t-way defectives, interpreting the element e i to be interaction (i, R i ) and absence of element e i in a set to be interaction (i, s i ) with s i safe value for i. Substitute row R by all k-tuples corresponding to tests performed by that algorithm. 3 The array A obtained is an ELA(N, t, H). To obtain a CAF E(N E(H), t, H) remove the E(H) rows from A that locate each edge of H. If d = E(H) then the number of rows is n = O(d t (log k) 2 ).

23 Thank you!