Supplementary Materials for Optimizing Two-level Supersaturated Designs by Particle Swarm Techniques

Transcription

1 Supplementary Materials for Optimizing Two-level Supersaturated Designs by Particle Swarm Techniques Frederick Kin Hing Phoa Institute of Statistical Science, Academia Sinica and Ray-Bing Chen Department of Statistics, National Cheng Kung University and Weichung Wang Institute of Applied Mathematical Sciences, National Taiwan University and Weng Kee Wong Department of Biostatistics, University of California at Los Angeles September 5, Numerical Studies for the choices of q LB and q GB in the MIX Operation We recommend q LB = m/3 or m/4 and q GB = m/6 in the MIX operation. This recommendation is based on a few simulation studies that used SIBSSD to find E(s 2 )-optimal SSDs. For space consideration, we describe one here where we consider finding an optimal SSD. For each combination of q LB and q GB, we ran each search for the LB and GB SSDs 100 times. The number of iterations and the approximate time for finding the Supported by the Career Development Award of Academia Sinica (Taiwan) grant number 103-CDA- M04 and National Science Council (Taiwan) grant number M MY3. Supported by the National Science Council (Taiwan) grant number M MY2 and the Mathematics Division of the National Center for Theoretical Sciences (South) in Taiwan. Supported by the National Science Council (Taiwan) and the Taida Institute of Mathematical Sciences. Supported by National Science Council, Mathematics Research Promotion Center grant number and the Ministry of Education (Taiwan) Top University Project to the National Cheng Kung University. 1

2 E(s 2 )-optimal SSD were recorded. As mentioned before, we assume that q LB > q GB to avoid premature convergence of all SSDs to the GB SSD. Table S.1 reports the simulation results. The four numbers in each box are the mean number of iterations, the standard deviation of the number of iterations required, the average time required in the 100 trials and its corresponding standard deviation. All simulations were done in R version 2.15 on a laptop equipped in Intel Core i7-2820qm CPU at 2.30GHz. The average time is obtained from the R function system.time. There are several observations from Table S.1. First, if the q s are set to be too small, more iterations are required to obtain the E(s 2 )-optimal SSD because information from the LB and GB SSDs are not available to the SSDs in the next iteration. In the extreme case, when we have q GB = q LB = 0, the whole procedure becomes a random search. Second, if the q s are set to be too large, extra time is required for the computational steps for the column exchanges. In addition, large values of q s may mean that the SSDs are more likely to be trapped in a local optimum and the lack of random columns replacement makes the escape from the local optimum more difficult. Simulation results shown in Table S.1 suggest the choices for q LB = 3 or 4 and q GB = 2 respectively, which match our recommendation. SIBSSD did well with q LB = [m/3] and q GB = [m/6] for the local and global searches of SSDs reported in the paper, where [x] is the positive integer as large as x. The average number of iterations required by SIBSSD shown in Table S.1 are not large, but the CP algorithm is faster than SIBSSD in terms of the time required for convergence. However, short convergence time of the CP algorithm does not mean that the final SSD is guaranteed to be optimal or near optimal either. This can be demonstrated in a simple simulation study replicated 100 times to find the E(s 2 )-optimal SSD. 2

3 Table S.1: Simulation results for different choices of q LB and q GB when m = 12 and N = 10. The 4 numbers in each cell are: the mean number of iterations, the standard deviation of the number of iterations, the average time (in seconds) and the standard deviation of the time (in seconds) required in the 100 trials. Choice of q GB for GB SSD Choice of q LB for LB SSD

4 2 SIBSSD-Generated Design Tables Tables S.3, S.4 and S.5 show the SIBSSD-generated SSDs, which are comparable to those reported in Bulutoglu and Cheng (2004) for N = 10, 12, 14, 16. These tables include the dimension of SSDs, their E r -efficiencies calculated from Bulutoglu and Cheng (2004) and the column indexes of these SSDs. The last column in the three tabless is the column indexes of our best SSDs obtained via SIBSSD. Each number in a column index is the decimal representation of a binary (two-level) column. Table S.2 demonstrates the correspondence between the column index and the column level settings of the SSD. To transform a decimal number (column index) 203 into its binary representation (two-level factor settings), we first find the binary number of 203, which is Then we map every (0, 1) digit into ( 1, +1) entry and obtain a binary column ( 1, 1, +1, +1, 1, 1, +1, 1, +1, +1) shown in the first column of Table S.2. Table S.2: SSD listed in Table 3. Column Index SSD Columns Practical Uses in Screening Experiments The SIBSSD-generated SSDs can provide good alternatives to some commonly used SSDs in scientific research and in practice. Consider, for example, the well known SSD D Lin proposed by Lin (1993) in Table S.6 and the SIBSSD-generated E(s 2 )-optimal SSD D P shown in Table S.4. The value of E(s 2 ) for D P is , which is slightly smaller than the value of from D Lin. It is instructive to compare designs based on additional characteristics and in this case, we further compare these two designs based on their ability to identify active factors. For this purpose, we set up a simulation study to ascertain capabilities of these two designs to identify active factor in a screening experiment. Among all variable selection methods for SSDs available in the literature, we choose to use the Stepwise Response Refinement Screener (SRRS) proposed by Phoa (2014) to identify active factors because it is one of the efficient and consistent methods for selecting active factors using SSDs. The simulation results in Phoa (2014) showed that the SRRS method performs well in SSD variable selection when compared to several commonly used methods in the literature. We set up our simulation similar to Example 4 of Phoa (2014) using ideas from Marley and Woods (2010) who suggested that the number of runs should be at least three times 4

5 Table S.3: SIBSSD-generated SSDs and their Er-efficiencies calculated from Bulutoglu and Cheng (2004) for N = 10 and 12. N m Er Column Indexes of SSDs found by the SIBSSD algorithm % (47, 179, 236, 286, 369, 436, 454, 661, 805, 905, 930) % (94, 199, 403, 412, 535, 601, 653, 692, 714, 782, 837, 976) % (920, 286, 838, 422, 810, 307, 466, 820, 535, 124, 397, 865, 341) % (203, 218, 271, 453, 468, 535, 604, 665, 710, 737, 842, 849, 908, 914) % (117, 218, 279, 398, 454, 527, 598, 690, 709, 812, 835, 856, 866, 913, 916) % (87, 124, 174, 434, 460, 481, 551, 618, 661, 665, 740, 752, 809, 820, 850, 902) % (117, 158, 167, 205, 285, 316, 331, 342, 422, 466, 535, 570, 590, 724, 775, 868, 913) % (91, 107, 182, 248, 286, 348, 358, 426, 496, 570, 572, 614, 716, 722, 817, 850, 872, 906) % (721, 355, 794, 376, 395, 569, 341, 94, 838, 434, 472, 812, 992, 234, 397, 841, 716, 109, 436) % (614, 316, 174, 662, 434, 355, 218, 484, 460, 810, 744, 395, 122, 782, 55, 79, 342, 167, 930, 451) % (866, 542, 835, 527, 682, 214, 626, 103, 283, 902, 364, 844, 203, 689, 601, 314, 968, 557, 234, 425, 220) % (803, 722, 117, 849, 179, 696, 203, 992, 376, 589, 569, 614, 744, 107, 302, 677, 118, 542, 611, 244, 818, 602) % (618, 484, 908, 738, 364, 597, 451, 199, 331, 472, 117, 310, 186, 782, 865, 710, 852, 850, 728, 929, 205, 557, 913) % (604, 810, 47, 661, 905, 283, 844, 301, 357, 91, 121, 460, 817, 182, 572, 790, 440, 682, 611, 310, 527, 632, 157, 677) % (908, 696, 299, 182, 61, 602, 880, 286, 614, 535, 818, 932, 837, 103, 682, 403, 899, 466, 362, 206, 782, 555, 601, 316, 662) % (796, 422, 211, 914, 206, 481, 692, 618, 121, 397, 55, 849, 372, 709, 364, 740, 803, 838, 434, 936, 342, 186, 472, 752, 714, 482) % (677, 279, 539, 117, 457, 961, 299, 468, 880, 541, 812, 331, 789, 109, 179, 412, 929, 824, 614, 185, 728, 589, 647, 398, 236, 602, 369) % (110, 419, 211, 721, 841, 866, 852, 229, 570, 557, 809, 654, 122, 440, 696, 205, 458, 436, 992, 902, 714, 236, 348, 628, 611, 283, 662, 728) % (426, 211, 410, 976, 327, 737, 803, 213, 662, 121, 236, 626, 454, 618, 539, 376, 174, 242, 94, 203, 614, 58, 955, 372, 542, 692, 714, 185, 167) % (710, 213, 59, 653, 422, 787, 342, 234, 930, 976, 472, 566, 651, 428, 218, 334, 395, 620, 227, 181, 542, 333, 920, 188, 214, 632, 654, 31, 314, 436) % (854, 873, 1251, 1944, 2265, 2362, 2508, 2605, 2800, 2947, 3409, 3658, 4000) % (423, 633, 670, 1111, 1227, 1340, 1992, 2284, 2382, 2457, 2898, 3130, 3284, 3597) % (441, 629, 795, 854, 1115, 1492, 1677, 1714, 1896, 2110, 2499, 2776, 3185, 3356, 3857) % (311, 858, 963, 1295, 1464, 1477, 1579, 1644, 2203, 2409, 2854, 2956, 3158, 3402, 3490, 3857) % (380, 485, 603, 867, 918, 1079, 1142, 1258, 1295, 1490, 1848, 2234, 2355, 2382, 3409, 3668, 3874) % (119, 500, 729, 754, 873, 1258, 1457, 1596, 1827, 2107, 2412, 2488, 2499, 2725, 2866, 3185, 3808, 3920) % (571, 742, 977, 1197, 2165, 2227, 2254, 2410, 2473, 2620, 2695, 2761, 2844, 3226, 3312, 3335, 3657, 3682, 4000) % (411, 627, 718, 853, 934, 1103, 1268, 1386, 1677, 1746, 1814, 2172, 2262, 2499, 2599, 2892, 3143, 3350, 3610, 4032) % (245, 619, 726, 828, 934, 965, 1118, 1319, 1506, 1512, 1678, 1805, 1873, 1874, 2375, 2382, 2866, 3016, 3476, 3684, 3715) % (231, 287, 629, 903, 996, 1244, 1325, 1458, 1475, 1713, 1813, 1874, 2358, 2417, 2513, 2710, 2953, 3143, 3221, 3492, 3619, 3908) % (543, 2857, 2103, 2769, 882, 3672, 715, 2978, 869, 700, 1638, 1713, 1940, 2758, 237, 3628, 2396, 3653, 2292, 2837, 2702, 634, 3123) % (3682, 685, 2529, 3207, 3404, 1272, 715, 2474, 359, 1307, 1954, 1992, 814, 1841, 3972, 2889, 2620, 3369, 2458, 3595, 1134, 854, 1477, 2914) % (603, 741, 1382, 3521, 2892, 2095, 3192, 1678, 3668, 2851, 3143, 2837, 3010, 3626, 1355, 1649, 1805, 3745, 237, 2457, 2635, 1833, 2412, 3213, 2792) % (374, 1198, 3622, 3162, 1203, 1295, 2858, 1938, 1859, 2247, 730, 3552, 2611, 1641, 490, 1638, 2347, 3356, 1145, 700, 3211, 95, 3186, 2762, 1818, 435) % (3118, 2375, 3858, 1241, 543, 1738, 630, 1127, 3276, 1925, 2199, 1310, 1460, 3668, 474, 3250, 686, 1478, 2172, 3040, 1649, 3732, 2259, 437, 860, 1394, 3353) % (3850, 2953, 1830, 2898, 3476, 2757, 2343, 3633, 1179, 2620, 1678, 444, 2738, 1000, 3496, 1641, 1925, 3180, 3612, 1841, 1748, 698, 2725, 797, 1520, 3225, 2904, 3808) % (207, 3164, 413, 2835, 3126, 3717, 1637, 1457, 3609, 599, 1746, 1355, 3722, 1055, 2829, 1806, 952, 732, 2665, 2394, 1873, 3225, 2755, 3524, 1195, 2199, 3521, 1926, 1752) % (3012, 238, 2364, 3238, 3342, 1381, 3636, 606, 2155, 685, 807, 3808, 2261, 2883, 498, 3688, 2954, 2866, 2412, 3158, 1520, 1932, 3459, 1890, 2758, 1358, 2520, 3242, 438, 1690) % (3347, 2860, 3976, 3605, 1938, 3619, 1086, 1477, 3312, 441, 423, 1716, 854, 3428, 1309, 2206, 3214, 2964, 2418, 1834, 1446, 1372, 2481, 1731, 3466, 1209, 1000, 2266, 795, 694, 207) % (3721, 3366, 854, 694, 2508, 2885, 1880, 2172, 3666, 1637, 2247, 2992, 1445, 915, 1309, 2617, 867, 3846, 1131, 629, 1678, 287, 1836, 811, 748, 1685, 1266, 1520, 2103, 3636, 686, 1118) % (1611, 3115, 1268, 2738, 1457, 3640, 573, 2898, 3591, 2417, 1817, 1427, 3605, 2203, 938, 1746, 1421, 3171, 3350, 1086, 2446, 4000, 679, 1251, 1896, 3748, 807, 2761, 725, 1690, 819, 3969, 2233) % (2460, 2788, 190, 3621, 1229, 3095, 423, 3718, 2390, 2095, 821, 2381, 599, 2699, 2530, 1131, 1806, 903, 1268, 574, 3366, 2737, 2666, 1836, 3377, 2652, 882, 365, 3908, 741, 2984, 1986, 2278, 1212) % (492, 1738, 1904, 854, 2788, 3522, 1272, 3681, 2738, 2142, 725, 371, 1127, 3880, 3221, 1925, 1148, 1458, 2473, 2868, 1817, 3397, 1436, 497, 1764, 573, 3672, 2265, 3024, 3252, 1683, 3126, 3412, 798, 1422) % (2649, 1889, 980, 952, 238, 2343, 459, 2892, 2977, 2457, 1818, 573, 2890, 1706, 783, 371, 2707, 2674, 2454, 2755, 249, 2732, 543, 2410, 966, 2866, 3226, 2000, 2396, 3465, 3653, 2292, 2984, 3380, 3688, 3718) 5

6 Table S.4: SIBSSD-generated SSDs for N = 14 and their Er-efficiencies as measured by the lower bounds from Bulutoglu and Cheng (2004). N m Er Column Indexes of SSDs found from the SIBSSD algorithm % (2425, 4813, 7076, 7632, 8677, 9628, 9968, 10838, 11913, 12467, 13160, 14396, 14538, 15121, 15429) % (9107, 10271, 13016, 14898, 15633, 4879, 6371, 5302, 3733, 5211, 2522, 4537, 7818, 6996, 2681, 14533) % (1453, 2419, 5415, 5745, 6819, 6956, 8933, 9138, 9771, 10567, 11444, 12128, 12776, 13077, 13414, 14393, 15745) % (823, 1772, 3952, 4565, 5167, 5436, 6019, 6761, 7073, 7253, 8569, 10053, 11429, 12902, 12980, 13792, 13849, 15116) % (1459, 1739, 2279, 3358, 3670, 4219, 4566, 5484, 7309, 7842, 8892, 9263, 9429, 10208, 10698, 13063, 13414, 13466, 15441) % (830, 1149, 1521, 1750, 2255, 3498, 4715, 5017, 5415, 5517, 6082, 6323, 7701, 8539, 8599, 8871, 9897, 12035, 12772, 13466) % (1486, 2477, 3865, 4159, 5042, 6734, 7273, 8917, 9068, 9338, 10467, 10550, 11019, 11420, 12633, 12941, 13575, 13736, 14490, 14676, 15908) % (1822, 2207, 2801, 2918, 3881, 4663, 5003, 5229, 5242, 6826, 8427, 8828, 9035, 9779, 9900, 10299, 11430, 12583, 13232, 14648, 14861, 15938) % (1207, 1479, 1964, 2426, 2774, 3181, 4726, 5041, 5275, 5368, 5678, 6439, 6556, 7876, 8682, 8847, 9588, 10428, 11825, 12381, 12517, 13590, 15522) % (1815, 3018, 4596, 4717, 5347, 6471, 7331, 8622, 8819, 9125, 9543, 9929, 10299, 10453, 11492, 12649, 13010, 13187, 13454, 13656, 14737, 14861, 14918, 16160) % (13861, 8072, 15170, 14561, 15506, 11545, 5489, 699, 11880, 5722, 3875, 6964, 12590, 6685, 9107, 6443, 10302, 4600, 1359, 10152, 12875, 7307, 12685, 5798, 13369) % (5421, 7702, 14382, 9912, 5964, 3685, 13081, 9373, 6825, 5827, 13988, 3560, 3230, 10829, 9126, 3896, 15745, 9333, 11555, 13418, 15696, 12164, 10058, 14960, 1843) % (942, 1207, 1630, 2020, 2485, 2669, 3613, 3875, 4455, 4473, 5048, 5182, 5771, 5901, 6762, 6934, 8889, 8990, 9043, 9515, 9580, 10414, 12839, 12997, 13876, 14605, 16160) % (251, 1516, 1749, 2936, 3251, 3662, 3785, 3847, 5003, 5074, 5181, 5678, 6247, 6826, 8798, 8871, 9571, 9786, 10121, 10383, 10857, 11172, 11546, 12621, 14048, 14492, 15554, 15889) % (501, 1359, 1402, 1756, 2703, 3041, 4318, 5073, 5347, 6082, 6550, 6986, 7592, 9099, 9126, 9625, 9960, 10482, 10586, 11459, 12100, 12587, 12748, 12908, 13137, 13383, 13970, 14537, 15748) % (956, 1199, 2921, 3290, 4222, 4579, 5930, 6429, 6791, 6962, 7648, 8654, 8821, 9123, 9656, 10287, 10542, 10649, 11204, 11315, 12521, 12660, 12691, 12954, 13581, 13897, 14086, 14666, 15016, 15524) % (10026, 8678, 11057, 13589, 10697, 15524, 3988, 11845, 8855, 1837, 13912, 5795, 12972, 3215, 9625, 6938, 7073, 4461, 8892, 1990, 13961, 14730, 11564, 5616, 15395, 3816, 14086, 12224, 14924, 5061, 9059) % (8636, 1679, 5548, 5872, 3992, 13081, 9433, 6805, 6539, 13962, 5179, 12065, 5089, 14506, 14441, 14870, 13637, 7632, 463, 4954, 7197, 6956, 11596, 15746, 3379, 3659, 14089, 7756, 10984, 11141, 5797, 6361) % (2143, 15875, 1849, 10865, 6857, 1869, 3878, 7764, 9884, 11370, 12839, 1987, 2742, 9457, 9188, 10637, 7333, 12629, 12908, 11202, 9367, 5478, 635, 1765, 4766, 9295, 4523, 14104, 5362, 7880, 11436, 2428, 8938) % (10905, 9188, 6926, 15528, 3796, 14104, 10446, 12950, 14915, 14396, 954, 11814, 9613, 15233, 5864, 12629, 11184, 3223, 8068, 13762, 13861, 5779, 11504, 8909, 13528, 4285, 6772, 6625, 3928, 6857, 14738, 8819, 11541, 11787) % (3306, 10806, 5830, 5179, 6550, 14444, 15010, 8861, 4794, 1852, 7333, 9359, 7694, 7576, 4947, 15619, 1327, 4999, 3017, 11787, 14120, 11924, 10178, 10650, 4001, 12716, 9650, 7768, 14216, 13521, 10574, 13652, 1246, 3861, 2020) % (10673, 2846, 12035, 11480, 7976, 6741, 6475, 1437, 12636, 14505, 6064, 12698, 9271, 13666, 4339, 9158, 11922, 702, 2363, 6866, 3430, 3537, 10872, 13402, 15500, 7284, 9080, 1770, 14128, 10724, 7446, 6081, 15236, 14883, 9045, 13837) % (9326, 3482, 9020, 6733, 9877, 10724, 16064, 1001, 3638, 12920, 14086, 9557, 3629, 11043, 12168, 8910, 5030, 3028, 6000, 11075, 1898, 13729, 9811, 1703, 15460, 5451, 3909, 10467, 2422, 9676, 11321, 14862, 12567, 16160, 9005, 6429, 3470) % (9422, 5785, 1882, 12815, 14930, 3909, 10523, 14785, 4503, 7562, 4476, 13609, 10680, 6251, 9123, 13521, 14988, 223, 3761, 8921, 4835, 13106, 7060, 9188, 4969, 13650, 13876, 8128, 2746, 11377, 6268, 1821, 11084, 15881, 3367, 13484, 14613, 14855) % (14876, 9515, 10693, 7073, 15497, 11938, 2796, 6000, 3877, 4207, 13094, 7219, 1214, 8825, 7693, 10556, 13540, 4537, 13194, 9877, 2231, 1933, 8751, 11576, 10835, 3862, 15397, 8599, 2414, 15010, 13638, 3659, 7310, 6423, 14665, 3523, 997, 11492, 11146) % (3642, 2477, 5287, 10767, 14993, 12068, 2802, 9785, 9626, 6460, 8429, 847, 1756, 3159, 7310, 11403, 11349, 8814, 8606, 5895, 3980, 8629, 14489, 11604, 1270, 985, 2359, 13078, 15044, 4033, 6520, 5213, 2591, 11594, 10595, 2515, 5481, 14426, 10003, 12742) % (1694, 3537, 13736, 8351, 2510, 5972, 15044, 13187, 9698, 10550, 4533, 998, 13398, 10992, 9052, 7718, 13389, 14599, 13492, 10115, 10901, 14224, 11858, 3753, 4915, 1821, 9068, 4826, 4837, 8315, 1214, 6938, 2517, 9005, 14744, 11660, 4600, 12491, 6798, 1479, 7497) % (9123, 4507, 889, 9786, 7400, 8909, 4207, 10481, 2223, 15619, 1483, 3373, 2518, 7009, 12724, 1143, 12445, 10541, 10457, 5036, 1510, 15010, 3733, 9031, 13351, 3467, 4339, 10302, 6486, 2789, 11462, 15464, 3635, 5301, 6671, 2746, 7701, 9649, 9870, 12379, 14979, 10550) 6

7 Table S.5: SIBSSD-generated SSDs for N = 16 and their Er-efficiencies as measured by the lower bounds from Bulutoglu and Cheng (2004). N m Er Column Indexes of SSDs found from the SIBSSD algorithm % (891, 5295, 10429, 11667, 14280, 25693, 28914, 29033, 34292, 39129, 41445, 41671, 44138, 45342, 46641, 50571, 58040) % (1455, 3167, 7020, 12955, 14398, 18041, 21686, 22937, 27427, 30026, 36634, 37235, 41421, 48169, 51434, 53775, 56581, 57660) % (12647, 11506, 30372, 5694, 14923, 3953, 43798, 52263, 43685, 48424, 61491, 46481, 6333, 40134, 9615, 42108, 27677, 34475, 38485) % (1515, 6494, 13005, 16164, 18374, 20791, 25724, 27089, 34645, 35997, 38130, 41383, 41848, 45270, 51684, 51787, 53357, 54168, 58638, 62785) % (5246, 7822, 8891, 10744, 15784, 19683, 20903, 21464, 26782, 30249, 34740, 36123, 37323, 39093, 42189, 47714, 50424, 51882, 55353, 57646, 62610) % (17333, 21865, 18682, 39904, 59683, 29578, 11689, 10086, 12755, 52614, 59073, 34618, 32290, 58608, 12120, 3811, 24464, 52041, 38539, 47665, 38227, 46504) % (44357, 26403, 64096, 50886, 33769, 39207, 19285, 59794, 11180, 13134, 41623, 61485, 16005, 40714, 58652, 37812, 3135, 23193, 44600, 43211, 13808, 54665, 45843) % (2806, 7470, 11155, 12661, 14130, 18221, 21338, 25541, 26142, 27747, 27985, 31892, 36633, 36678, 41387, 42549, 43356, 46659, 50672, 51255, 53527, 57466, 58758, 64288) % (7725, 10938, 12761, 13030, 15893, 17207, 19700, 19770, 22742, 24386, 25209, 25743, 27459, 31004, 34927, 35733, 38490, 44115, 44960, 45366, 53937, 55946, 58902, 59980, 63523) % (24226, 26220, 45682, 40152, 52592, 27846, 22165, 54666, 48897, 42697, 29779, 34486, 51371, 1523, 53985, 31632, 43189, 4589, 10168, 57812, 54332, 6750, 49947, 46374, 28858, 27705) % (5606, 56389, 32385, 22830, 47938, 15629, 3631, 15272, 36554, 46644, 54826, 40280, 63622, 41197, 36772, 53673, 37767, 47212, 59148, 28002, 19164, 13470, 30432, 44083, 58762, 51636, 40120) % (759, 13669, 15507, 6861, 3485, 41055, 59557, 22649, 39380, 10744, 36405, 31260, 37054, 50129, 54476, 46617, 37771, 51479, 44107, 45357, 3886, 25485, 44740, 52393, 39777, 21831, 31363, 51790) % (2782, 42883, 27749, 52009, 23948, 10730, 12623, 49637, 44868, 27179, 36043, 62152, 59670, 31618, 26009, 18162, 59601, 7137, 10909, 32321, 11760, 22867, 17351, 12981, 25464, 1883, 25767, 26444, 20245) % (1901, 23244, 51847, 8081, 29781, 43293, 26266, 49401, 60864, 38285, 6379, 23732, 16855, 29577, 19755, 52056, 43685, 54180, 52785, 2550, 30983, 40770, 54318, 21299, 1950, 20045, 37463, 14908, 26468, 55580) % (46914, 34524, 21969, 12053, 15088, 8439, 3019, 39318, 5031, 51315, 15418, 1530, 24133, 17270, 39257, 59796, 39526, 3870, 36529, 11113, 13684, 23352, 7341, 45992, 29211, 37181, 21212, 23970, 50993, 26988, 14801) % (57525, 50858, 48004, 19180, 31257, 25330, 11578, 36438, 12910, 19239, 52284, 31842, 27783, 58697, 30500, 55776, 11224, 54042, 59266, 33135, 19251, 43689, 43378, 18891, 25020, 2033, 7354, 20894, 45227, 11662, 59470, 12503) % (36069, 15692, 51531, 38489, 5561, 46469, 57532, 29409, 39284, 53462, 40106, 50471, 4719, 47155, 9587, 19669, 41433, 42526, 63513, 13455, 41581, 27880, 34546, 54060, 39563, 61890, 54377, 48960, 43937, 4602, 62562, 22321, 14391) % (36409, 51681, 27733, 48736, 3452, 27818, 61740, 58800, 12785, 56082, 54421, 20420, 6390, 5927, 54498, 26919, 48452, 50515, 13722, 32385, 57910, 22709, 21611, 8112, 47379, 29304, 23582, 8919, 47245, 21337, 51418, 11379, 31554, 2987) % (6462, 13268, 61994, 58646, 43633, 9115, 26797, 25691, 38130, 10085, 11506, 18408, 34510, 34215, 33469, 53955, 58800, 51653, 40217, 43922, 3735, 20259, 49499, 61064, 44364, 33273, 46881, 51828, 43850, 23010, 17214, 8088, 20305, 10667, 36458) % (13901, 55905, 23864, 4671, 26741, 20058, 57689, 20935, 25398, 27726, 30596, 18845, 18289, 56341, 13651, 31378, 39254, 60163, 15117, 29034, 14521, 36139, 25779, 15824, 53466, 64515, 3811, 46196, 41689, 10726, 58540, 34199, 43574, 44824, 45355, 54240) % (29036, 43490, 43852, 22923, 17273, 52833, 12208, 25306, 62744, 38852, 2973, 3534, 55822, 37095, 32386, 14934, 7467, 43070, 29868, 37810, 47257, 1726, 15141, 52532, 26375, 44681, 23272, 55700, 57517, 13405, 45866, 58245, 38729, 14801, 42442, 5052, 15588) % (43802, 39115, 33526, 28974, 19578, 14580, 40241, 25049, 5789, 53596, 41083, 14940, 10216, 53865, 62546, 13155, 60041, 46504, 21051, 35692, 57653, 6585, 19783, 44085, 31184, 2003, 20088, 15969, 42329, 49389, 7501, 55136, 46158, 26141, 37717, 1470, 10599, 25829) % (59722, 45180, 26202, 31878, 13391, 47908, 6431, 38856, 55170, 3022, 16080, 10923, 9173, 37590, 59624, 52820, 7786, 22745, 17906, 55875, 29042, 53787, 15250, 39393, 42659, 43219, 28481, 17127, 3769, 52380, 58137, 51766, 36167, 44338, 62597, 31332, 41678, 38513, 44115) % (57939, 41446, 9208, 58728, 52044, 31536, 26837, 11882, 18906, 52402, 61656, 45740, 44316, 17981, 28294, 13430, 20783, 26797, 17022, 34517, 51572, 7068, 13653, 45214, 1998, 31302, 40026, 6269, 53961, 21899, 11497, 43279, 54380, 31940, 39139, 21988, 36532, 65032, 39749, 50207) % (2735, 35131, 57962, 29571, 22886, 14541, 48163, 54952, 46230, 17651, 39604, 57765, 49943, 31648, 1966, 41206, 29214, 28056, 26165, 52870, 59531, 25902, 53060, 45837, 33754, 9201, 27190, 51309, 20795, 45922, 11591, 42412, 44555, 53415, 59842, 40242, 61532, 6558, 43828, 59576, 14954) % (25818, 20039, 56468, 14423, 15026, 45646, 39033, 5100, 43661, 49819, 57447, 57588, 24168, 48394, 27430, 59992, 42801, 30221, 27081, 55120, 14033, 36294, 31169, 9692, 62632, 47810, 11498, 31124, 44692, 43864, 2301, 44449, 28204, 14049, 3739, 40013, 13374, 60528, 55466, 50886, 3954, 31763) % (51140, 11925, 3691, 9565, 21395, 59730, 40049, 49758, 23768, 8020, 47816, 32259, 36111, 6630, 63557, 27820, 9846, 38700, 12495, 35257, 38101, 45621, 13161, 47630, 46258, 51261, 5021, 42267, 8446, 7478, 21647, 35271, 44696, 2803, 27045, 14746, 42886, 38122, 41953, 49846, 36547, 62499, 35375) % (23654, 14684, 24216, 4066, 59500, 36532, 27442, 28914, 42450, 59208, 48170, 55746, 37745, 50492, 4542, 11605, 13737, 29477, 37455, 15080, 48177, 2427, 43430, 47892, 10895, 18677, 26166, 3948, 8926, 26053, 5974, 41956, 9339, 45808, 59971, 31512, 17323, 15969, 36753, 32130, 58584, 43291, 19320, 22741) % (37362, 28050, 3004, 48390, 52848, 31176, 36293, 9593, 40089, 3375, 15700, 29987, 38849, 5358, 59018, 59557, 20163, 42166, 48680, 22436, 6807, 54805, 10855, 14001, 25813, 9822, 11155, 7259, 17821, 21334, 30833, 3826, 14052, 24360, 42840, 13209, 38195, 10085, 45583, 16258, 14524, 43828, 41450, 48324, 35809) % (60962, 42094, 26827, 12943, 55466, 29462, 62849, 51981, 30249, 24260, 11676, 43729, 50830, 37834, 18918, 8100, 18392, 7323, 8638, 3939, 22861, 15060, 31306, 35389, 19607, 48141, 53303, 48240, 1631, 37101, 26277, 36046, 17657, 58012, 28759, 38291, 54868, 59717, 58712, 30904, 13676, 41953, 12043, 46275, 39706, 60678) % (36442, 25198, 18374, 14663, 23665, 27814, 44896, 13611, 20107, 15516, 38070, 3002, 31251, 13042, 56032, 35299, 9933, 16765, 58451, 62218, 6998, 41653, 5735, 26480, 59941, 63688, 21417, 18543, 37469, 49402, 18901, 23100, 11881, 50963, 41340, 25817, 10994, 43055, 12060, 29750, 41623, 40014, 40497, 52644, 5861, 14801, 3259) % (29154, 45387, 21390, 7373, 45972, 21811, 13839, 39067, 63749, 58507, 27271, 52524, 39142, 33598, 36802, 15637, 49621, 25560, 9461, 43625, 10670, 5093, 46504, 14430, 2875, 44866, 31336, 39841, 42662, 20300, 26979, 57901, 42317, 47512, 16033, 5993, 54318, 62021, 4559, 56520, 46194, 33499, 35175, 57769, 9708, 37500, 10429, 51444) 7

8 Table S.6: A two-level supersaturated design (Lin 1993). Run

9 the number of active factors. Since both D P and D Lin have 14 observations, the number of active factors in this experiment would not be more than 4. However, to be more liberal, we allow to include one more active factor. Thus we considered five cases i = 1,..., 5 with i active factors and for each case, we generated 500 models where the selection of active factors is random and without replacement. The signs of the active factors are also randomly selected from either positive or negative, and the magnitudes are randomly selected from 2 to 10 with replacement. For each model, we generated data 100 times and obtained the True Model Identified Rate (TMIR) and the average true model size, where we recall from Phoa (2014), TMIR is defined as the number of times (over 100 times) that SRRS correctly identifies all active factors and the average true model size is generally the average (over 100 times) of the final model size suggested by SRRS. In our simulations, we followed Phoa (2014) and fixed γ = 1, where γ is the threshold of noise level in SRRS described in the paper. Generally speaking, we choose γ to be approximately 5% 10% of the absolute magnitude of estimate of the first chosen active factor. Readers who are interested in repeating this simulation are encouraged to use the graphical user interface (GUI) package of SRRS that is available in the comprehensive R archive network. Phoa and Lin (2014) describes the implementation of SRRS with several self-explanatory illustrations. Table S.7 compares the simulation results from the two SSDs D P and D Lin. The performance of SRRS is similar using both D P and D Lin as the experimental plans. These distributions seem quite skewed both in TMIR and true model size, which means that the the SRRS performs well no matter if D P or D Lin is used as an experimental plan. Both SSDs are very effective in identifying 1 and 2 active factors in terms of TMIR (at least 96%) and average model sizes, and both are a bit less effective in identifying 3 active factors. The difference between the two SSDs is obvious when SRRS attempts to identify 4 or 5 active factors. When D P is used in cases IV and V, the mean TMIRs and the mean model sizes of SRRS are 4% 7% higher and about 0.1 closer to the true model sizes than the results when D Lin is used. Although the difference of the E(s 2 ) values between the two SSDs is small, there are still some differences in their analysis performance. The SRRS performs better when SSD has a smaller E(s 2 ) value than one with a larger E(s 2 ) value. There are two useful observations in this comparison. First, the above comparison shows that by using the same analysis method (like SRRS) and the same measure (like TMIR or mean model size), a SSD with smaller E(s 2 ) is likely to provide better accuracy. If SSDs are not easily accessible via catalogues or softwares, then SIBSSD becomes a useful and efficient tool for finding SSDs with low E(s 2 ). Second, two SSDs with similar E(s 2 ) values typically result in little changes in the analysis results. For example, in the above comparison, the difference in the SRRS performance between the two SSDs is small and so is their difference in E(s 2 ) values (about 6%). Figure 1 shows if the maximum number of iteration is set at 100, we have a SSD with E(s 2 ) = Is it worth spending the extra amount of time to obtain the E(s 2 )-optimal SSD? Figure 1 shows that the maximum number of iterations is set at step 52, we have a GB SSD with E(s 2 ) = and at step 141, we have a GB SSD with E(s 2 ) = ). This implies that we have to spend almost three times more the amount of time to bring about roughly a 6% improvement in the analysis performance. So our answer is no and this explains why we are satisfied with 9

10 Table S.7: Results from a simulation study to compare abilities of the two designs D P and D Lin for identifying active factors using the Stepwise Response Refinement Screener. Case SSD Min 1st Quartile Median Mean 3rd Quartile Max I TMIR D Lin 97% 99% 100% 99.49% 100% 100% D P 98% 99% 100% 99.31% 100% 100% Size D Lin D P II TMIR D Lin 96% 99% 100% 99.38% 100% 100% D P 97% 99% 100% 99.46% 100% 100% Size D Lin D P III TMIR D Lin 5% 99% 100% 96.96% 100% 100% D P 2% 99% 100% 97.54% 100% 100% Size D Lin D P IV TMIR D Lin 0% 93% 99% 84.33% 100% 100% D P 0% 97% 99% 88.21% 100% 100% Size D Lin D P V TMIR D Lin 0% 20% 86% 64.22% 100% 100% D P 0% 50% 93% 71.06% 99% 100% Size D Lin D P

11 most of our results in our design tables when E r > 90%. References Bulutoglu, D.A., Cheng, C.S. (2004). Construction of E(s 2 )-optimal supersaturated designs. Annals of Statistics 32, Lin, D.K.J. (1993). A new class of supersaturated designs. Technometrics 35, Marley, C.J., Woods, D.C. (2010). A comparison of design and model selection methods for supersaturated experiments. Computational Statistics and Data Analysis 54, Phoa, F.K.H. (2014). The Stepwise Response Refinement Screener (SRRS). Statistica Sinica 24, Phoa, F.K.H. (2014). A graphical user interface platform of the Stepwise Response Refinement Screener for screening experiments. Proceedings of COMPSTAT 2014,