Covering Arrays and Generalizations

Transcription

1 School of Information Technology and Engineering University of Ottawa UPC Seminar, November 6

2 Covering array examples array on alphabet {,..., g }; k columns; find covering array with minimum n. N= Examples: g = 3, k = 4 K = N= K = 3 4 (OAs, k MOLS, lower bound)

3 Covering array definition Definition: Covering Array A covering array with k factors, g levels for each factor and size n, denoted by CA(n; k, g), is an n k array with symbols from [, g ] such that for every pair of columns, every ordered pair in [, g ] is covered at least once. Objective: given k and g find a covering array with mininum size n. CAN(k, g) = min{n : there exists a CA(n; k, g)}. Example: g =, k = 4: K = N= 5 Is this optimal?

4 Application: component interaction testing Testing pairwise interaction of factors. components of a hardware system factors influencing publication sucess Operating System ( linux /windows ) Topic ( superconnectivity / Cayley graphs ) Web browser ( netscape / mozilla) Professor ( Anna Llado / Oriol Serra ) File format ( pdf / postscript ) Printer ( hp/ epson ) Student ( Jordi Moragas / Amanda Montejano Meeting place ( UPC / tapas bar ) ) inputs to software or combinatorial circuits test test test3 test4 test5 Factors: test test test3 test4 test5 Operating System Web browser File format Printer binary outcome: PASS/FAIL PASS/FAIL PASS/FAIL PASS/FAIL PASS/FAIL Factors: paper paper paper3 paper4 paper5 Topic Professor Student Meeting place binary outcome: ACCEPTED/REJECTED ACCEPTED/REJECTED ACCEPTED/REJECTED ACCEPTED/REJECTED ACCEPTED/REJECTED Correct/Incorrect Output covering array: exhaustive testing: ^4=6 tests pairwise testing: 5 tests Assumption: failures come from the interaction of or less factors. All way interactions are covered. May not detect bad 3 way interactions example:. Oriol Serra Jordi Moragas tapas bar

5 Advertisement: buy covering arrays! AETG (Telcordia): (web service) price per year: US$6, - US$6, TestCover.com: (web service) license price per year: US$, CaseMaker: (GUI software) price not in their web page Pro-test (SigmaZone): (GUI software) license: US$399 Other tools: IBM Intelligent Test Case Handler, CATS, OATS, IPO, TConfig, TCG (NASA), AllPairs, Jenny, ReduceArray, DDA, Test Vector Generator, OA, CTE-XL, PICT (Microsoft), rdexpert.

6 Covering array generalizations important for applications mixed alphabet sizes: factors may have different number of possible values example: OSs, 3 browses, 4 file formats, printers.

7 Covering array generalizations important for applications mixed alphabet sizes: factors may have different number of possible values example: OSs, 3 browses, 4 file formats, printers. forbiden configurations: some combinations may be forbidden (not supported) example: (linux,internet explorer) not allowed.

8 Covering array generalizations important for applications mixed alphabet sizes: factors may have different number of possible values example: OSs, 3 browses, 4 file formats, printers. forbiden configurations: some combinations may be forbidden (not supported) example: (linux,internet explorer) not allowed. higher strength: want to test 3-way interaction of factors example: failures may occur due to bad 3-way combinations (Oriol, Jordi, Tapas bar)

9 Covering array generalizations important for applications mixed alphabet sizes: factors may have different number of possible values example: OSs, 3 browses, 4 file formats, printers. forbiden configurations: some combinations may be forbidden (not supported) example: (linux,internet explorer) not allowed. higher strength: want to test 3-way interaction of factors example: failures may occur due to bad 3-way combinations (Oriol, Jordi, Tapas bar) mixed strength: certain factors need higher srength example: all pairwise + (student,professor,meeting place)

10 Covering array generalizations important for applications mixed alphabet sizes: factors may have different number of possible values example: OSs, 3 browses, 4 file formats, printers. forbiden configurations: some combinations may be forbidden (not supported) example: (linux,internet explorer) not allowed. higher strength: want to test 3-way interaction of factors example: failures may occur due to bad 3-way combinations (Oriol, Jordi, Tapas bar) mixed strength: certain factors need higher srength example: all pairwise + (student,professor,meeting place) covering arrays on graphs: certain combinations don t need interactions tested example: all pairwise interactions except (topic, professor), (meeting place, student)

11 Covering array: summary of results general lower bound: CAN(k, g) g

12 Covering array: summary of results general lower bound: CAN(k, g) g CAN(k, g) = g iff OA(k, g, λ = ) iff (k ) MOLS of order g. Only possible for k g +.

13 Covering array: summary of results general lower bound: CAN(k, g) g CAN(k, g) = g iff OA(k, g, λ = ) iff (k ) MOLS of order g. Only possible for k g +. CAN(k, g) CAN(k +, g)

14 Covering array: summary of results general lower bound: CAN(k, g) g CAN(k, g) = g iff OA(k, g, λ = ) iff (k ) MOLS of order g. Only possible for k g +. CAN(k, g) CAN(k +, g) CAN(k, g) CAN(k, g + )

15 Covering array: summary of results general lower bound: CAN(k, g) g CAN(k, g) = g iff OA(k, g, λ = ) iff (k ) MOLS of order g. Only possible for k g +. CAN(k, g) CAN(k +, g) CAN(k, g) CAN(k, g + ) CAN(k, g = ) has been solved.

16 Covering array: summary of results general lower bound: CAN(k, g) g CAN(k, g) = g iff OA(k, g, λ = ) iff (k ) MOLS of order g. Only possible for k g +. CAN(k, g) CAN(k +, g) CAN(k, g) CAN(k, g + ) CAN(k, g = ) has been solved. For general g: direct and recursive constructions.

17 Covering array: summary of results general lower bound: CAN(k, g) g CAN(k, g) = g iff OA(k, g, λ = ) iff (k ) MOLS of order g. Only possible for k g +. CAN(k, g) CAN(k +, g) CAN(k, g) CAN(k, g + ) CAN(k, g = ) has been solved. For general g: direct and recursive constructions. Non-constructive asymptotic result known for fixed g: CAN(k, g) g log k, as k

18 Covering array optimization questions Fix g. Minimizing n for fixed k (number of tests) CAN(k, g) = min{n : there exists a CA(n; k, g)}. Maximizing k for fixed n (number of factors) CAK(n, g) = max{k : there exists a CA(n; k, g)}. Relationship between min-max problems CAN(k, g) = min{n : CAK(n, g) k}.

19 Direct construction via Orthogonal arrays Definition: Orthogonal Array An orthogonal array with k factors, g levels for each factor, denoted by OA(k, g), is an g k array with symbols from a [, g ]G such that for every pair of columns, every ordered pair in [, g ] is appears at exactly once. For g a prime power, there exists g mutually orthogonal Latin squares which gives an OA(g +, g) orthogonal array. (equivalent to the existence of a projective plane of order g). Construction uses finite fields.

20 Direct construction via Orthogonal arrays Definition: Orthogonal Array An orthogonal array with k factors, g levels for each factor, denoted by OA(k, g), is an g k array with symbols from a [, g ]G such that for every pair of columns, every ordered pair in [, g ] is appears at exactly once. For g a prime power, there exists g mutually orthogonal Latin squares which gives an OA(g +, g) orthogonal array. (equivalent to the existence of a projective plane of order g). Construction uses finite fields. For g prime power, CAN(k, g) = g for all k g +.

21 Direct construction via Orthogonal arrays Definition: Orthogonal Array An orthogonal array with k factors, g levels for each factor, denoted by OA(k, g), is an g k array with symbols from a [, g ]G such that for every pair of columns, every ordered pair in [, g ] is appears at exactly once. For g a prime power, there exists g mutually orthogonal Latin squares which gives an OA(g +, g) orthogonal array. (equivalent to the existence of a projective plane of order g). Construction uses finite fields. For g prime power, CAN(k, g) = g for all k g +. For g not a prime power, use the larger knwon number of MOLS:

22 Direct construction via Orthogonal arrays Definition: Orthogonal Array An orthogonal array with k factors, g levels for each factor, denoted by OA(k, g), is an g k array with symbols from a [, g ]G such that for every pair of columns, every ordered pair in [, g ] is appears at exactly once. For g a prime power, there exists g mutually orthogonal Latin squares which gives an OA(g +, g) orthogonal array. (equivalent to the existence of a projective plane of order g). Construction uses finite fields. For g prime power, CAN(k, g) = g for all k g +. For g not a prime power, use the larger knwon number of MOLS: CAN(k, 6) = 36 for k =,, 3, but CAN(4, 6) > 36.

23 Direct construction via Orthogonal arrays Definition: Orthogonal Array An orthogonal array with k factors, g levels for each factor, denoted by OA(k, g), is an g k array with symbols from a [, g ]G such that for every pair of columns, every ordered pair in [, g ] is appears at exactly once. For g a prime power, there exists g mutually orthogonal Latin squares which gives an OA(g +, g) orthogonal array. (equivalent to the existence of a projective plane of order g). Construction uses finite fields. For g prime power, CAN(k, g) = g for all k g +. For g not a prime power, use the larger knwon number of MOLS: CAN(k, 6) = 36 for k =,, 3, but CAN(4, 6) > 36. CAN(k, ) = for k =,, 3, 4, but CAN(5, )? =.

24 Recursive construction: Blocksize recursive construction with 3 disjoint rows: CA(3,3) OD(3,3) size=6 size=9+6=5 CA(4,3) size= CA(=4*3,3) 9 CA(N,k,g)+OD(N,k,g)= CA(N+N,k*k,g)

25 Algorithmic construction: Greedy method Greedy method used in the AETG system (D. Cohen, Dalal, Fredman and Patton (997)): Choose one test at a time. At each stage select a test that covers the maximum number of uncovered pairs. Good news: for fixed g, CA size is proportional to log k. Bad news: to pick a test covering the maximum number of uncovered tests is NP-complete, so the authors use a heuristic for test selection which does not guarantee the logarithmic growth. The DDA (determinisitc density algorithm) by M. Cohen, Colbourn and Turban (4): greedy method that runs in polynomial time; for fixed g, CA size is proportional to log k; this is based on a guarantee that each selected test cover the average number of uncovered tests.

26 CAs with g= are extremal set systems set system S complement: C column column 3 4 column3 5 column4 {,} {,3} {,4} {,5} {3,4,5} {,4,5} {,3,5} {,3,4} base set = {,,3,4,5} S must be pairwise intersecting: pair (, ) is covered. C must be pairwise intersecting: pair (, ) is covered. each of S and C must have the Sperner property: pairs (, ) and (, ) covered.

27 Sperner theorem for set systems A system of subsets of an n-set has the Sperner property if no two subsets in the system are comparable. {,,3,4} {,,3} {,,4} {,3,4} {,3,4} {,} {,3} {,4} {,3} {,4} {3,4} {} {} {3} {4} {} Sperner s Theorem (98) If A has the Sperner property, then A ( n n ). The upper bound is only acchieved by the set of all ( n )-subsets of the n-set, or by its (subsetwise) complement.

28 Erdos-Ko-Rado theorem for set systems A system of subsets of an n-set is (pairwise) intersecting if every two subsets in the system have nonempty intersection. Examples: (n = 5) A = {{,, 3}, {, 4, 5}, {, 3, 4}, {, 4, 5}, {3, 4, 5}} (n = 6) B = {{,, 3}, {,, 4}, {,, 5}, {,, 6}, {, 3, 4}, {, 3, 5}, {, 3, 6}, {, 4, 5}, {, 4, 6}, {, 5, 6}} Erdos-Ko-Rado Theorem (96) Let A be an intersecting system of subsets of an n-set, such that each subset has cardinality at most k. If n k, then A ( n k ). Moreover, if n > k, then equality holds if and only if A is a k-uniform trivially intersecting system.

29 Optimal construction for binary alphabet Pick all n/ -subsets of [, n] that contain a common element. n even: n odd: Note: the arrays are transposed here (k n). Both A and A are intersecting and Sperner.

30 The binary covering array theorem Theorem (Katona 973, Kleitman and Spencer 973) CAK(n, t =, g = ) = ( n n/ ). Moreover, this bound is attained by a n/ -uniform trivially -intersecting set system. Proof: Let A be the set system corresponding to a CA. (Case ) n even.

31 The binary covering array theorem Theorem (Katona 973, Kleitman and Spencer 973) CAK(n, t =, g = ) = ( n n/ ). Moreover, this bound is attained by a n/ -uniform trivially -intersecting set system. Proof: Let A be the set system corresponding to a CA. (Case ) n even. A = {A, A : A A} is Sperner.

32 The binary covering array theorem Theorem (Katona 973, Kleitman and Spencer 973) CAK(n, t =, g = ) = ( n n/ ). Moreover, this bound is attained by a n/ -uniform trivially -intersecting set system. Proof: Let A be the set system corresponding to a CA. (Case ) n even. A = {A, A : A A} is Sperner. Sperner s theorem implies A ( n n/).

33 The binary covering array theorem Theorem (Katona 973, Kleitman and Spencer 973) CAK(n, t =, g = ) = ( n n/ ). Moreover, this bound is attained by a n/ -uniform trivially -intersecting set system. Proof: Let A be the set system corresponding to a CA. (Case ) n even. A = {A, A : A A} is Sperner. Sperner s theorem implies A ( n A A ( n ) ( n/ = n n/ ). n/).

34 The binary covering array theorem Theorem (Katona 973, Kleitman and Spencer 973) CAK(n, t =, g = ) = ( n n/ ). Moreover, this bound is attained by a n/ -uniform trivially -intersecting set system. Proof: Let A be the set system corresponding to a CA. (Case ) n even. A = {A, A : A A} is Sperner. Sperner s theorem implies A ( n A A ( n ) ( n/ = n n/ ). (Case ) n odd. n/).

35 The binary covering array theorem Theorem (Katona 973, Kleitman and Spencer 973) CAK(n, t =, g = ) = ( n n/ ). Moreover, this bound is attained by a n/ -uniform trivially -intersecting set system. Proof: Let A be the set system corresponding to a CA. (Case ) n even. A = {A, A : A A} is Sperner. Sperner s theorem implies A ( n A A ( n ) ( n/ = n n/ ). (Case ) n odd. n/). Wlog assume A n/, for all A A.

36 The binary covering array theorem Theorem (Katona 973, Kleitman and Spencer 973) CAK(n, t =, g = ) = ( n n/ ). Moreover, this bound is attained by a n/ -uniform trivially -intersecting set system. Proof: Let A be the set system corresponding to a CA. (Case ) n even. A = {A, A : A A} is Sperner. Sperner s theorem implies A ( n A A ( n ) ( n/ = n n/ ). (Case ) n odd. n/). Wlog assume A n/, for all A A. A is -intersecting, so by the EKR theorem, A ( n n/ ).

37 Covering arrays are systems of set partitions A covering array (strength ) is a system of set-partitions: column column column 3 column 4 {,,3,} {4,5,6} {,4,7,} {,5,8} {,5,9} {,6,7,} {,,6,8} {4,9,} {7,8,9} {3,6,9} {3,4,8} {3,5,7}

38 Covering arrays are systems of set partitions A covering array (strength ) is a system of set-partitions: column column column 3 column 4 {,,3,} {4,5,6} {,4,7,} {,5,8} {,5,9} {,6,7,} {,,6,8} {4,9,} {7,8,9} {3,6,9} {3,4,8} {3,5,7} Maximization problem: Given N, find a set partition system P with maximum P that is (pairwise) strongly intersecting: For all P, Q P we have for all P i P, Q j Q, P i Q j.

39 Strongly intersecting condition: upper bound via -parts Theorem (Stevens, Moura and Mendelsohn 998) CAK(n,, g) ( n g ). n g This theorem only uses the two smallest parts of each partition, and the following fact: Consider a pair of set systems, A, A,..., A k and B, B,..., B k, with A i + B i c and A i a c/, and such that A i B i =, and all other sets intersect. Then, k ( c a). It is possible to relabel symbols of the covering array so that P j n g and P j + P j n g

40 Stronly intersecting versus Sperner formulation Strongly intersecting formulation: Partitions P and Q corresponding to two columns of a covering array must satisfy: for all P i P, Q j Q, P i Q j. Strongly Sperner formulation: Partitions P and Q corresponding to two columns of a covering array must satisfy: for all P i P, Q j Q, P i Q j and P i Q j

41 Sperner s theorem for set-partition systems largest cardinality k of a system P of g-partitions of [, n] such that for all P i, P j P: P P i, P P j (P P andp P ). (Weakly) Sperner Theorem (Meagher, Moura and Stevens 5) Let g, n such that n = cg + r and r < g. Then, ( ) n N n (, ) (g r) + r(c+). c n Theorem (Meagher, Moura and Stevens 5) Let g, n such that g n. Then, N n (, ) = ( ) n n. Moreover, this g bound is met if and only if the g-partitions are uniform (all parts with cardinality n g ).

42 Example: weakly Sperner property n=g {,,3},{4,5,6} {,,4},{3,5,6} {,,5},{3,4,6} {,,6},{3,4,5} {,3,4},{,5,6} {,3,5},{,4,6} {,3,6},{,4,5} {,4,5},{,3,6} {,4,6},{,3,5} {,5,6},{,3,4} n=3g {,,3},{4,5,6},{7,8,9} {,,4},... {,,5},... {,,6}, {,7,8},... {,8,9},{,3,4}, {5,6,7}

43 Comparison of two bounds obtained Theorem (Stevens, Moura and Mendelsohn 998) CAK(n,, g) ( n g ). n g Theorem (Meagher, Moura and Stevens 5) If g n, then CAK(n,, g) ( ) n n. g if g >, g n, then ( n ) ( ) g n n < n g g

44 Erdos-Ko-Rado theorem for set-partition systems We are interested on a maximal partition system P such that: each partition of [, n] have g parts of size n g ; two partitions P, Q P are such that there exists P i P and Q j Q such that P i Q j p. Useful for bounds on anti-covering-arrays for certain uniform cases. Ex: n = g, p = Conjecture Suppose g n, and let c = n/g be the size of each part of the (uniform) partition system. P = ( n p c p) U(n c, g ).

45 P Q R Required property: g p c Conjecture:

46 P Q R Required property: g p c Conjecture has been proven for p = c: Conjecture:

47 P Q R Required property: g p c Conjecture has been proven for p = c: Theorem (Meagher and Moura 5) Conjecture: Let n g and let P Ug n be a maximal partition system in which every two partitions share at least one class. Let c = n/g. Then, P = U(n c, g )

48 Covering array on graphs factors influencing publication sucess with knwon SAFE INTERACTIONS: Topic ( superconnectivity / Cayley graphs ) Professor ( Anna Llado / Oriol Serra ) Student ( Jordi Moragas / Amanda Montejano Meeting place ( UPC / tapas bar ) ) Factors: paper paper paper3 paper4 Topic Professor Student Meeting place for complete graph min N = 5 For this graph min N = 4

49 Covering array on graphs: definition Definition: Covering Array A covering array on a graph G with alphabet size g and size n, denoted by CA(n; G, g), is an n k = V (G) array with symbols from [, g ] such that for every pair of columns corresponding to an edge of G, every ordered pair in [, g ] is covered at least once. Objective: given G and g find a covering array with mininum size n. CAN(G, g) = min{n : there exists a CA(n; G, g)}. Determining CAN(G, ) is NP-complete (Serousi and Bshouty) reduction to 3-COLOUR.

50 Graph homomorphisms Definition: graph homomorphism A graph homomorphism from graph G to graph H, denoted G H is a mapping from V (G) to V (H) that takes edges to edges. Vertex colouring = homomorphism to the complete graph with numberofcolours vertices. G H G H 3 3 See book Graphs and homomorphisms by Hell and Nesetril, 4.

51 Qualitative independence graph The qualitative independence graph QI(n, g) has: Vertex Set: all g-partitions of [, n] that have every class of size at least g. Edges: two vertices are connected if their partitions are qualitatively independent (P, Q are qualitatively independent if P i Q j for all i, j.)

52 Why QI(n, g) are interesting? Results by Meagher and Stevens (5): A k-clique in QI(n, g) corresponds to a CA(n, k, g); A CA(n, G, g) exists if and only if there exists a graph homomorphism G QI(n, g); CAN(G, g) = min{n : G QI(n, g)}.

53 Clique and chromatic bounds Corollary (Meagher and Stevens 5): If there exists a homomorphism G H, then CAN(G, g) CAN(H, g). It is well-known that there exists homomorphisms: K ω(g) G K χ(g). Therefore: CAN(ω(G), g) CAN(G, g) CAN(χ(G), g). This gives the colouring construction : k-colour the vertices of G. build a CA(n, k, g). pull back a CA(n, G, g).

54 Colouring construction k-colour the vertices of G. build a CA(n, k, g). pull back a CA(n, G, g). a b c d e 3 3 G a b d e c

55 Binary alphabet results ω(qi(n, )) = ( n n ) (Kleitman and Spencer, Katona 973).

56 Binary alphabet results ω(qi(n, )) = ( ) n n (Kleitman and Spencer, Katona 973). χ(qi(n, )) = ( n ) n (Meagher and Stevens 5)

57 Binary alphabet results ω(qi(n, )) = ( ) n n (Kleitman and Spencer, Katona 973). χ(qi(n, )) = ( n ) n (Meagher and Stevens 5) If CAN(G, ) n, then there exists a CAN(n, G, ) with n s per row (nearly balanced). (Meagher and Stevens 5)

58 Binary alphabet results ω(qi(n, )) = ( ) n n (Kleitman and Spencer, Katona 973). χ(qi(n, )) = ( n ) n (Meagher and Stevens 5) If CAN(G, ) n, then there exists a CAN(n, G, ) with n s per row (nearly balanced). (Meagher and Stevens 5)

59 Binary alphabet results ω(qi(n, )) = ( ) n n (Kleitman and Spencer, Katona 973). χ(qi(n, )) = ( n ) n (Meagher and Stevens 5) If CAN(G, ) n, then there exists a CAN(n, G, ) with n s per row (nearly balanced). (Meagher and Stevens 5) Conjecture for general g (Meagher) If CAN(G, g) n, then there exists a CAN(n, G, g) that is nearly balanced (each symbol appears either n g or n g times).

60 Binary alphabet results ω(qi(n, )) = ( ) n n (Kleitman and Spencer, Katona 973). χ(qi(n, )) = ( n ) n (Meagher and Stevens 5) If CAN(G, ) n, then there exists a CAN(n, G, ) with n s per row (nearly balanced). (Meagher and Stevens 5) Conjecture for general g (Meagher) If CAN(G, g) n, then there exists a CAN(n, G, g) that is nearly balanced (each symbol appears either n g or n g times). If true, we can concentrate on AUQI(n, g) (almost uniform qualitative independence graphs)!

61 Mixed covering arrays Different factors/parameters can have different alphabet sizes. gi = 3 References: Moura, Stardom, Stevens and Williams, Mixed covering arrays (3) Colbourn, Martirosian, Mullen, Shasha, Sherwood and Yucas, Pruducts of mixed covering arrays of strength two (6) Sherwood, A column expansion construction for optimal and near-optimal mixed covering arrays (preprint).

62 Mixed covering arrays on graphs Combine covering array on graphs with mixed alphabet sizes. Reference: Meagher, Moura, Zekaoui, Mixed covering arrays on graphs,(to appear): generalize graph homomorphism results; give optimal constructions for special classes of graphs. Cheng, The Test Suite Generation Problem: Optimal Instances and Their Implications, preprint. Give optimal constructions for special classes of graphs and for hypertrees.

63 Higher strength (t 3) t=3 k=5 g= References: Chateauneuf and Kreher, On the state of covering arrays of strength three,. Colbourn, Martirosyan, Trung, and Walker, Roux-type Constructions for Covering Arrays of Strengths Three and Four (6). etc.

64 Locating arrays We not only want to detect that an error exists, but we want to know which t-interaction caused the error. Related to design of experiments and combinatorial group testing. mixed covering array *3*3*3*3*3 d= t= C T Reference: Colbourn and McClary, Locating and detecting arrays: existence and minimization, preprint. Work in progress by myself with Martinez, Panario and Stevens on the adaptive problem.

65 References R. Ahlswede, N. Cai and Z. Zhang. Higher level extremal problems. J. Combin. Inform. System Sci., :85, 996. M. Chateauneuf and D. L. Kreher. On the state of strength-three covering arrays J. Combin. Des., :7-38,. 3 C. Cheng. The Test Suite Generation Problem: Optimal Instances and Their Implications, preprint. 4 C.J. Colbourn, M.B. Cohen, and R.C. Turban. A Deterministic Density Algorithm for Pairwise Interaction Coverage. Proceedings of the International Conference on Software Engineering (SE 4), Innsbruck, Austria, pp. 45-5, February 4. 5 D.M. Cohen, S.R. Dalal, M.L. Fredman and G.C.Patton. The AETG system: an approach to testing based on combinatorial design. IEEE Trans. Soft. Engin., 3: , C.J. Colbourn, S.S. Martirosyan, G.L. Mullen, D.E. Shasha, G.B. Sherwood, and J.L. Yucas. Products of Mixed Covering Arrays of Strength Two. J. Combin. Des., 4:4 38, 6. 7 C.J. Colbourn, S.S. Martirosyan, Tran Van Trung, and R.A. Walker II. Roux-type Constructions for Covering Arrays of Strengths Three and Four. Designs, Codes and Cryptography, 4:33 57, 6. 8 P. Erdos, C. Ko and R. Rado. Intersection theorems for systems of finite sets. Quart. J. Math. Oxford Ser. :33 3, G. Katona. Two applications (for search theory and truth functions) of Sperner type theorems. Period. Math. Hungar., 3:9 6, 973. D.J. Kleitman and J. Spencer. Families of k-independent sets. Discrete Math., 6:55 6, 973. K. Meagher. Covering arrays on graphs: qualitative independence graphs and extremal set-partition theory.ph.d. thesis, University of Ottawa, 77 pages, Sept. 5. K. Meagher and L. Moura. Erdos-Ko-Rado theorems for uniform set-partition systems. Electr. J. Combin., R4, pages, 5. 3 K. Meagher, L. Moura and B. Stevens. A Sperner-type theorem for set-partition systems. Electr. J. Combin., N, 6 pages, 5. 4 K. Meagher and B. Stevens. Covering arrays on graphs. Journal of Combinatorial Theory B, 95():34 5, 5.

66 References continued 5 L. Moura, J. Stardom, B. Stevens and A. Williams. Covering arrays with mixed alphabet sizes. Journal of Combinatorial Designs, :43 43, 3. (3) 6 G. Sherwood,. A column expansion construction for optimal and near-optimal mixed covering arrays. preprint. 7 N. Sloane. Covering arrays and intersection codes. Journal of Combinatorial Designs, :5 63, E. Sperner. Ein Satz uber Untermengen einer endlichen Menge. Math. Z., 7: , B. Stevens, L. Moura and E. Mendelsohn. Lower bounds for transversal covers. Des., Codes and Cryptogr., 5:79-99, 998.