Testing Computability by Width Two OBDDs

Transcription

1 Testing Computability by Width Two OBDDs Dana Ron School of EE Tel-Aviv University Ramat Aviv, Israel Gilad Tsur School of EE Tel-Aviv University Ramat Aviv, Israel Abstract Property testing is concerned with deciding whether an object (e.g. a graph or a function) has a certain property or is far (for some definition of far) from every object with that property. In this paper we give lower and upper bounds for testing functions for the property of being computable by a read-once width-2 Ordered Binary Decision Diagram (OBDD), also known as a branching program, where the order of the variables is known. Width-2 OBDDs generalize two classes of functions that have been studied in the context of property testing - linear functions (over GF (2)) and monomials. In both these cases membership can be tested in time that is linear in 1/ɛ. Interestingly, unlike either of these classes, in which the query complexity of the testing algorithm does not depend on the number, n, of variables in the tested function, we show that (one-sided error) testing for computability by a width-2 OBDD requires Ω(log(n)) queries, and give an algorithm (with one-sided error) that tests for this property and performs Õ(log(n)/ɛ) queries.

2 1 Introduction Property testing is concerned with deciding whether an object (e.g. a graph or a function) has a certain property or is far (for some definition of far) from every object with that property [RS96, GGR98]. Typical property testing algorithms are randomized, and perform queries regarding local properties of the object (e.g., the value of a function f on the input x), returning a correct answer with high probability. Generally, the distance parameter, denoted ɛ, is given as an input to the property testing algorithm and effects its running time. Property testing algorithms are designed to run in time that is sublinear in the size of the tested object or to perform a number of queries that is sublinear in it. Indeed, many of them use a number of queries that is independent of the object s size. Our Results. In this paper we give lower and upper bounds for testing functions for the property of being computable by a read-once width-2 Ordered Binary Decision Diagram (OBDD), also known as a read-once Oblivious Branching Program of width 2, where the order of the variables is known. Width-2 OBDDs generalize two classes of functions that have been studied in the context of property testing - linear functions (over GF (2)) [RS96] and monomials [PRS02]. In both these cases membership can be tested in time that is linear in 1/ɛ. Interestingly, unlike either of these classes, in which the query complexity of the testing algorithm does not depend on the number of variables in the tested function, we show that (one-sided error) testing for computability by a width- 2 OBDD requires Ω(log(n)) queries, and we give an algorithm (with one-sided error) that performs Õ(log(n)/ɛ) queries. 1 We note that it is open whether allowing two-sided error can decrease the complexity of the problem. Observe that the logarithmic dependence on n is still much lower than the linear dependence that is necessary for learning this family. We later shortly discuss the extensive research on OBDDs in the learning literature. Function classes for which property testing algorithms have been designed are usually characterized as either algebraic (e.g. [BLR93, RS96, AKK + 05, KR00, JPRZ04]) or non-algebraic (e.g., [PRS02, FKR + 04, DLM + 07]), though some results can be viewed as belonging to both categories. We view the family of functions we study as falling naturally into the second category, since it is described by a type of computational device and not by a type of algebraic formula. As opposed to many algorithms for algebraic families, algorithms for non-algebraic families generally rely on the fact that the functions in the family are close to juntas, that is, functions that depend on a small number of variables. This is true, by definition, for singletons [PRS02] and juntas [FKR + 04], but also for monomials, monotone DNF with a bounded number of terms [PRS02], general DNF, decision lists and many other function classes, studied in [DLM + 07]. In contrast, our algorithm tests for membership in a class of functions in which the function may depend (significantly) on many variables. Techniques. Variables in functions that are computable by width-2 OBDDs can be divided into two groups. Variables that the function is linear in, and all other variables. This distinction is made more precise in Section 2. Our algorithm attempts to locate the last (according to the fixed order) O(log(1/ɛ)) non-linear variables, and uses this structural information to determine whether a function is computable by a width-2 OBDD. This can be contrasted with the results in [DLM + 07] that cover a wide-range of non-algebraic function families. There, the algorithms detect a small number of subsets of variables, each containing a relevant variable. If the tested function belongs to the class in question then it depends almost entirely on this small set of relevant variables. Our 1 Here the notation Õ(T ) represents an upper bound that is linear in T up to a polylogarithmic factor. 1

3 algorithm learns something of the structure of the OBDD computing the function that relates to a small set of variables, yet many variables can have non-negligible influence. Since our algorithm rejects only when it finds evidence that the tested function is not computable by a width-2 OBDD, it immediately follows that it always accepts functions in this family. The core of the proof is in showing that if the function is ɛ-far from the family, then the algorithm rejects with high constant probability. More precisely, we prove the contrapositive statement. Since our algorithm works by verifying that various restrictions of the tested function are close to having certain properties, the difficulty is in proving that we can glue together these restrictions and obtain a single width-2 OBDD. Additional related work. We note that the type of question we ask differs from that studied by Newman [New02]. Newman shows that a property testing algorithm exists for any property decidable by a constant width branching program. In [New02] the property is defined with regard to a particular branching program, and the algorithm tests membership in a language decidable by that program. In contrast, in our result, the language we test for membership in is one where every word is the truth table of a width-2 branching program. OBDDs and, in particular, bounded width OBDDs have been studied in the machine learning context rather extensively. In particular it has been shown that width-2 OBDDs are PAC-learnable, while width-3 and wider OBDDs are as hard to learn as DNF formulas [EKR95]. These results were strengthened in [BTW98, BBTV97]. When membership queries are allowed and the underlying distribution is uniform, width-2 branching programs with a single 0 sink are efficiently learnable [BBTV97]. When both membership and equivalence queries are allowed then there are several positive results for other restricted types of branching programs [RW93, GG95, Nak00, Nak05, BBTV97]. Organization. In Section 2 we provide some basic definitions and present general claims regarding OBDDs and width-2 OBDDs in particular. In Section 3 we give a lower bound for one-sided error testing of computability by width-2 OBDDs. In Section 4 we give a one-sided error testing algorithm for computability by a width-2 OBDD. 2 Preliminaries 2.1 Basic Definitions Definition 1 The distance between a function f and a function g (both from domain X to range R), denoted d(f, g), is defined as Pr x [f(x) g(x)] where x is drawn uniformly at random from X. When d(f, g) > ɛ we say that f is ɛ-far from g. This definition extends to the distance of a function f from a family of functions G (also denoted d(f, G)). Here we have d(f, G) = min g G {d(f, g)}. When f is ɛ-far from all g G we say that f is ɛ-far from G. We may occasionally describe a function f as ɛ-close to g. This means that f is not ɛ-far from g. Definition 2 A property testing algorithm T for property P is given oracle access to an object in the form of a function f from domain X to range R and a distance parameter 0 < ɛ < If f P then T accepts with probability at least 2/3 (over its internal coin tosses); 2. If f is ɛ-far from P then T rejects with probability at least 2/3 (over its internal coin tosses). 2

4 A property testing algorithm is said to be a one-sided error property testing algorithm if it accepts with probability 1 when f P. Of course, if we wish to accept or reject incorrectly with probability at most δ we can repeat the application of a property testing algorithm Θ(log(1/δ)) times and take a majority vote. Later in this work we will routinely amplify probability of success as required. A property testing algorithm that we use as a basic building block in our algorithm is the linearity tester, proposed by Blum, Luby and Rubinfeld [BLR93]. In [BLR93] it is assumed that for a linear function f it holds that f(0 n ) = 0. For our purposes linearity allows for a single coefficient that doesn t depend on any variable, and the BLR algorithm is easily adapted for such a case. Definition 3 We say that f : {0, 1} n {0, 1} is a linear function if there exist coefficients b 0, b 1,..., b n {0, 1} such that for x = x 1,..., x n {0, 1} n, f(x) = b 0 + Σ n i=1 b ix i. Theorem 1 [BLR93] There exists a one-sided error testing algorithm for linearity. Its query complexity is O(1/ɛ). Definition 4 A Binary Decision Diagram (BDD), also known as a branching program, is an acyclic directed graph with a single source where sinks are labeled 0 or 1, and other nodes are labeled by a Boolean variable from X = {x 1,..., x n }. The Boolean function associated with a BDD is computed on a particular assignment to the variables in X by returning the label of the sink reached when these assignment values are used to trace a route through the graph. There are different definitions in the literature for Ordered Binary Decision Diagrams. results hold for the definition of a strict fixed width read-once binary decision diagram: Our Definition 5 A read-once Ordered Binary Decision Diagram (OBDD) is a BDD where each path from the root to a sink must pass through all variables in a fixed order (where each variable appears once). The width of an OBDD is the maximum number of nodes in a level of the rooted graph. Branching programs can, of course, compute functions from other, non-binary, finite domains and to other finite ranges. Indeed, it may sometimes be more convenient to consider such programs, and we do this in Claim 1. As each path from the root to a sink must pass through all variables in a given order, x i s level in the OBDD is well defined. Throughout the following text we refer to OBDD s over a fixed order of the variables x 1,..., x n that is known to us. Once the order of the variables has been fixed there exists an automaton corresponding to any OBDD, that is, the OBDD can be thought of as an automaton without loops and with accepting states corresponding to 1 sinks of the OBDD. 2.2 Properties of OBDDs Our first claim follows directly from the definition of OBDDs. Claim 1 A function f : {0, 1} n [k] (where [k] = {0,..., k 1}) is computable by a width-k OBDD if and only if f(x 1,..., x n ) = g n (f n 1 (x 1,..., x n 1 ), x n ) where f n 1 is a function computable by a width-k OBDD (over 0 variables if n = 1) and g n is a function from [k] {0, 1} to [k]. 3

5 Proof: We prove this by induction on the number of variables. In the base case we have a function of 1 variable and the claim is trivial. The induction step in each direction is simple too - if we have a function f on n 1 variables computable by a width k OBDD and a function g that accepts f s output and the n th variable, we can simulate it by placing the correct edges in an OBDD of length n and width k. Likewise, if we have an OBDD of length n and width k, the state on the n 1 th level is computable by a width k OBDD, and the final state of the OBDD is only a function of the state before last and of the n th bit. Definition 6 A set S is said to be an i-prefix Equivalence Class (or just an i-equivalence class) for a function f : {0, 1} n {0, 1} (where i n) when S is a maximal subset of {0, 1} i such that for all x, y S and for all z {0, 1} n i it holds that f(xz) = f(yz). As a corollary of Claim 1 we have: Corollary 1 A function f : {0, 1} n {0, 1} is computable by a width k OBDD if and only if i [1, n] there are at most k distinct i-prefix equivalence classes for f. Proof: If a function is computable by a width k OBDD then it has, for all i [1, n], at most k nodes on its i th level (denoted arbitrarily 0 to k 1 if k such nodes exist, otherwise denoted from 0 up). Thus, each prefix x 1,..., x i either belongs to the equivalence class of all prefixes reaching the node 0, to that of all prefixes reaching the node 1, and so on up to the k 1 th node. In the other direction, let f be a function of n variables where i [1, n] there are at most k distinct i-prefix equivalence classes for f. we prove by induction on n that f is computable by a width k OBDD. The base case is where f is a function of 0 variables, which can clearly be computed by a width k OBDD (for k 1, which we naturally assume). As the induction step, we consider f n 1 - the function telling us to which equivalence class in f each prefix of length n 1 belongs. As f is a function of f n 1 (which is computable by a width k OBDD by the induction hypothesis) and of x n, we have from Claim 1 that f n is computable by a width k OBDD. Definition 7 A string z {0, 1} n i is a distinguishing assignment for two i-prefix equivalence classes S 1, S 2 over a function f if for x S 1, y S 2 it holds that f(xz) f(yz). In describing our algorithm we routinely restrict a function over some of its input variables. We introduce a notation for such restrictions: Definition 8 Let f be a function of n boolean variables, and let w {0, 1} m. We define f i,j,w, where j = i + m 1, to be the function f with the variables x i,..., x i+m restricted to the values w 1,..., w m accordingly. This means that f i,j,w (x 1,..., x n ) f(x 1,..., x i 1, w 1,..., w m, x j+1,..., x n ). As the values x i,..., x j have no influence on f i,j,w we view it, interchangeably, as a function of n variables or as a function of n m variables. For a width-2 OBDD we can arbitrarily equate each of the (at most) 2 nodes in the i th level of the OBDD with a value in {0, 1}. We denote by f i : {0, 1} i {0, 1} the function that is given the string x 1,..., x i and returns the value of the node in the i + 1 th level reached from the source by traversing the OBDD according to them. Thus, for a distinguishing assignment w of length n i we have that either f i (x 1,..., x i ) = f i+1,n,w (x 1,..., x i ) for all x 1,..., x i, or that f i (x 1,..., x i ) = f i+1,n,w (x 1,..., x i ) for all x 1,..., x i. 4

6 2.2.1 Merging levels and bit influence Definition 9 An OBDD is said to be 0-merging (1-merging) on the i th level if in level i + 1 there exists a node with two incoming edges marked by 0 (1). For σ {0, 1}, if an OBDD isn t σ-merging on the i th level we say it is σ-non-merging on the i th level. Claim 2 Let M be a width-2 OBDD and let f i 1 be the function that maps the variables x 1,..., x i 1 to a node on the i th level. If layers i to n are 0, 1-non-merging, then M computes a linear function of f i 1 (x 1,..., x i 1 ) and of x i,..., x n. That is, there exists a linear function ˆf such that f(x 1,..., x n ) = ˆf i (f i 1 (x 1,..., x i 1 ), x i,..., x n ). Proof: We show this by induction on n i. For n i = 1 we have that each sink has one edge marked 1 and one edge marked 0 entering it. Clearly one of the following holds: 1. f n = f n 1 2. f n = f n f n = f n 1 x n 4. f n = f n 1 x n 1 For the induction step we simply note that by induction the claim is true for f n 1 and the transition step is similar to the base step, that is, a linear function of some linear function f(x 1,..., x i ) and of x i+1 is a linear function of x 1,..., x i+1. Definition 10 Let x i be the string x with the i th bit flipped. The bit influence of the i th bit in the function f, denoted inf f (i) or just inf(i) is Pr[f(x) f(x i )] when x is drawn from the uniform distribution. Claim 3 For a width-2 OBDD M where layer i is 0-merging or 1-merging, for j < i it holds that inf fi (j) 1 2 inf f i 1 (j). Proof: Consider, without loss of generality, a width-2 OBDD M where layer i is 0-merging. For j < i we have: and the proof is completed. inf fi (j) = Pr[f i (x) f i (x j )] (1) = 1 2 Pr[f i(x) f i (x j ) x i = 0] Pr[f i(x) f i (x j ) x i = 1] (2) = 1 2 Pr[f i(x) f i (x j ) x i = 1] 1 2 inf f i 1 (j) (3) Claim 4 For a function f computable by a width-2 OBDD M, the influence of the bits x 1,..., x i 1 in f i is no greater than their influence in f i 1. Proof: We note that if g i is 0, 1-non-merging than the influence of the variables stays the same. Otherwise, by Claim 3 it decreases. Claims 3 and 4 will later allow us to find several merging levels in width-2 OBDDs and to disregard all the variables coming before these levels. In Section 4 we will use the following corollary of these claims to show the correctness of our algorithm. 5

7 Claim 5 Let f be a function of the form f(x 1,..., x n ) = f (g(x 1,..., x n m ), x n m+1,..., x n ) where f is computable by a width-2 OBDD M. If M has at least k merging levels, then f is 2 k -close to the function f (0, x n m+1,..., x n ), that is computable by a width-2 OBDD. Proof: By Claim 3 the influence of the first bit of f is at most 2 k. The claim follows. 3 A Lower Bound Theorem 2 Any one-sided error tester for computability by a width-2 OBDD requires Ω(log(n)) queries. We prove Theorem 2 by noting that any one-sided error property testing algorithm that rejects a function f must have queried, for some i [1, n], several strings indicating that f has at least 3 different i-equivalence classes. If this were not the case, then by Claim 1, f may be computable by a width-2 OBDD. We describe a family of Θ(n) functions, all 1/4-far from any function computable by a width-2 OBDD, where each function is fully specified by a unique value i where the function has more than two equivalence classes. We note that any one-sided error property testing algorithm for width two OBDDs would, in fact, implicitly correspond to an exact learning algorithm for members of this family, and that due to the family size this would require at least Ω(log(n)) queries. The construction is as follows: We define the function γ j, for 1 < j < (n 1) as x 1 (x j x j+1 ) x n. Essentially, for all prefixes except for the j th this function has two equivalence classes, but it has more than two equivalence classes for the prefix x 1,..., x j, and it has a constant distance from any function computable by a width-2 OBDD. Claim 6 Let f be a function computable by a width-2 OBDD. For 1 < j < (n 1), it holds that d(γ j, f) 1/4. We prove this claim using the fact that for all j it holds that x 1 has influence 1 in γ j, and that all the variables have influence greater than or equal to 1/2 (all the variables have influence 1 except x j and x j+1, that have influence 1/2). Proof: Consider towards a contradiction a function f that is computable by a width-2 OBDD and isn t 1/4-far from the function γ j. We consider several cases: 1. If the influence of x 1 in f is less than 1, then, since f is computed by a width-2 OBDD we get by Claim 3 and Claim 4 that x 1 s influence in f must be at most 1/2. This is the case because non-merging levels do not reduce influence, and other levels reduce influence by at least half, and the first bit must either have 1 or 0 influence in f 1. If the influence of x 1 is indeed less than 1/2, then f is at least 1/4-far from γ j and we reach a contradiction, because for at least half of the values x it holds that f(x) = f(x 1 ), but γ j (x) γ j (x 1 ) for all values of x. 2. Otherwise, as the influence of x 1 is 1, again, by Claim 3 and Claim 4, all of the levels in an OBDD computing f are 0, 1-non-merging, and thus f is a linear function (by Claim 2). If f does not depend on one of its variables, then, again, it must be at least 1/4-far from γ j, as the influence of all the variables in γ j is at least 1/2. 6

8 3. We are left with the case where f is a parity of all its variables (or the negation of their parity), and all variables have influence 1 in f. Again, f must be at least 1/4-far from γ j, because the influence of the j th bit in f is 1 and in γ j it is 1/2. The following is a well known fact. Claim 7 A (possibly randomized) algorithm L that is given oracle access to a boolean function f F, performs queries on f and determines the identity of f with probability greater than 2/3 must perform at least Ω(log( F )) queries. Theorem 2 follows directly from Claims 6 and 7. 4 The Testing Algorithm Imagine that we are promised that a function f given to us as input is either computable by a width-2 OBDD that has no merging levels, or that it is far from any function computable by a width-2 OBDD. We could check which of the above is the case using BLR s linearity test on f, as a function computable by a width-2 OBDD that has no merging levels is a linear function. Now, imagine we are promised that f is either far from any function computable by a width-2 OBDD or that it is computable by a width-2 OBDD that has exactly one merging level, in the i th level, where i is known. We could check to see which of the cases above holds by going through the following procedure. First we d like to see if f has at most two i-equivalence classes. We cannot know this exactly, but we are able to tell if f is close to a function with 1, 2, or more i-equivalence classes using an algorithm we will describe below. If we only find one i-equivalence class for f it remains to check if f is a linear function of x i+1,..., x n. If it is then f is computable by a width-2 OBDD with one merging level, i (and we can accept). If f has more than two i-equivalence classes then it is clearly not computable by a width-2 OBDD (of any kind), and we can reject. Finally, if f has two i-equivalence classes we must check that the function f i 1 (the function that maps the variables x 1,..., x i 1 to (i 1)-equivalence classes) is linear, and that the function which maps the i-equivalence class and the variables x i+1,..., x n to f(x 1,..., x n ) is linear, as well. As a final hypothetical scenario, consider the following promise: f is either far from every function computable by a width-2 OBDD, or can be computed using a width-2 OBDD with a single unknown merging level. If we could locate the merging level, we know we could tell which of the two cases holds, as done in the previous paragraph. We note that as a consequence of Claim 3 any function that is computable by a width-2 OBDD and isn t linear is far from linear, so we would like to check f and see what parts of it are linear. We can do this by performing a binary search for a linear section. Begin by restricting the first n/2 variables to 0, and checking if the function computed on all the rest of the variables is (close to) linear. If it is, repeat the process with fewer variables restricted. If it isn t, repeat the process with more variables restricted. If we are, indeed, given a function that is computable by a width-2 OBDD that has only a single merging level, this process will allow us to detect the merging level with high probability. The property testing algorithm we suggest for computability by a width-2 OBDD is based on the observations made above and on those made in the previous sections. In particular, we note that, as a consequence of Claim 5, any function f computable by a width-2 OBDD is ɛ-close to a function g computable by a width-2 OBDD that has at most O(log(1/ɛ)) merging levels, where 7

9 the merging levels of g are all merging levels of f. When our algorithm is given as input a function computable by a width-2 OBDD, it will (with high probability) locate the last O(log(1/ɛ)) merging levels (if such merging levels exist, of course). Locating these levels will be done using a binary search technique reminiscent of the one suggested above. We will restrict the function on some of its bits (x 1,..., x j ) and test whether the restricted function is linear, using a version of the BLR linearity test. For any function f computable by a width-2 OBDD our algorithm will find the structure of a function g that is close to it, and for every function that passes our test, we will show that it is likely to be close to a function computable by a width-2 OBDD. A notion that is used repeatedly is that of a function f that can be computed by a width-2 OBDD that accepts as input the value of a function g(x 1,..., x t ) and the bits x t+1,..., x n (in that order) and outputs the value f(x 1,..., x n ). We define this formally: Definition 11 A function f : {0, 1} n {0, 1} is said to be a W2-function of g : {0, 1} t {0, 1} and of x t+1,..., x n if there exists a width-2 OBDD that accepts as input the value g(x 1,..., x t ) and the bits x t+1,..., x n (in that order) and outputs the value f(x 1,..., x n ). In Fig. 1 we present the testing algorithm for computability by a width-2 OBDD. In the algorithm we use the value ɛ, which intuitively stands for the amount of error we are willing to accumulate during each round of the algorithm. We set ɛ = ɛ/(4 log(1/ɛ)). The algorithm uses two sub-procedures, Get-linear-level and Count-equiv-classes, both described after the algorithm. Theorem 3 Algorithm Test-width-2 is a one-sided error testing algorithm for the property of being computable by a width-2 OBDD, that runs using Õ(log(n)/ɛ) queries. 4.1 Building Blocks We now proceed to discuss the probabilistic procedures used in Test-width-2. We later return to proving Theorem 3. We have already mentioned the BLR linearity test, one procedure that we will use as an internal building block in our own sub-procedures. We now turn to describe an additional building block - a procedure that is given access to a function f and a number i, and attempts to check whether f has 1, 2 or more i-equivalence classes. Despite the fact it only counts up to 2, or perhaps up to many, we dub this sub-procedure Count-equiv-classes. A precise description of the algorithm appears in Fig. 2. The straightforward approach to performing this task may be to take a set of prefixes of length i and compare each two (or all of them) on a set of suffixes, trying to find prefixes that belong to different equivalence classes. A simple analysis implies a procedure that performs Θ(1/ɛ 2 ) queries. The approach we take is slightly different. We start with the arbitrary string 0 i, which belongs to some equivalence class. To identify a second equivalence class we simply test the equality of f(x 1,..., x n ) with f 1,i,0 i(x i+1,..., x n ). If a second equivalence class is detected then we use a similar technique to try and find a third equivalence class (with a small adjustment). This approach leads to a Θ(1/ɛ) algorithm. Claim 8 The algorithm Count-equiv-classes, given oracle access to a function f acts as follows: 1. If f is ɛ-far from every function with one i-equivalence class, then with probability at least 1 δ Count-equiv-classes will return representatives of at least two equivalence classes. 2. If f is ɛ-far from every function with two i-equivalence classes, then with probability at least 1 δ Count-equiv-classes will return representatives of three equivalence classes. 8

10 Test-width-2 Input: Oracle access to a function f; Precision parameter ɛ. 1. Let f 0 = f. Let r = 1 and t 0 = n. The variable t will represent the number of variables of f that we haven t restricted, and r will be the number of the current round. The indexes on t and f will indicate the round r and help us keep track of different values for the analysis. 2. While t r 1 1 and r log(1/ɛ) + 2 (a) Locate linear section: Run Get-linear-level(f r 1, ɛ 1 /3, 6(log(1/ɛ)+2) ). This locates the last index j such that f r 1 is (ɛ /3)-close to a linear function of f r 1 (x1,..., xj) j+1,t r 1,w and of x j+1,..., x t r 1 for a distinguishing sequence w. (b) If Get-linear-level indicated the existence of more than 2 different i-equivalence classes on some level i, reject. (c) Otherwise, let j be the level returned by Get-linear-level and let w be the distinguishing sequence returned by it. (d) Let g r = f r 1 j+1,t r 1,w and let t r = j. (e) If j 1 i. Run Count-equiv-classes(g r, t r 1, ɛ /3, 1 12 log(1/2ɛ) ). This tells us whether the number of ( t r 1)-equivalence classes in g r is 1, 2 or more with precision ɛ /3. ii. If a single equivalence class is found, accept. iii. If more than 2 equivalence classes are found, reject. iv. Let w denote the distinguishing assignment (of size 1) between the 2 equivalence classes found (returned by Count-equiv-classes). Let f r = g r j,j,w and let tr = j 1. (f) Else, let f r = g r and let t r = t r. (g) r = r return accept. Figure 1: Algorithm Test-width In any case Count-equiv-classes does not indicate the existence of more than the number of i-equivalence classes of f. 4. Conditioned on Count-equiv-classes(f) returning the representatives of 2 different i- equivalence classes and a distinguishing assignment, with probability at least 1 δ it holds that f is ɛ-close to a function of f i+1,n,w (x 1,..., x i ) (where w is the distinguishing assignment) and of the variables x i+1,..., x n. ( ) The algorithm performs O log(1/δ) ɛ queries. Proof: If f is ɛ-far from any function with one i-equivalence class, then it is, in particular, ɛ-far from f 1,i,0 i(x i+1,..., x n ). We will therefore encounter a value such that f(x) f 1,i,0 i(x i+1,..., x n ) with probability at least 1 (δ/2) if we set m appropriately, and will thus return the representatives of at least two i-equivalence classes. Likewise, if f is ɛ-far from any function with at most two i- equivalence classes, it is in particular ɛ-far from the function it is compared to in Item 5 and we 9

11 Count-equiv-classes Input: Oracle access to a function f; Integer value 0 < i n; Precision parameter ɛ; Confidence parameter δ. 1. Select m = Θ(log(1/δ)/ɛ) strings x 1,..., x m from {0, 1} n. 2. If f(x) = f 1,i,0 i(x i+1,..., x n ) for all x {x 1,..., x m } output that 1 equivalence class was found. Otherwise, let y {0, 1} i, w {0, 1} n i be such that f(yw) f(0 i w). 3. Select m = Θ(log(1/δ)/ɛ) new strings z 1,..., z m {0, 1} i. 4. Define g(z j ) as 0 if f(0 i, w) = f(z j 1,..., zj i, w), and as 1 otherwise. Compute g(zj ) for all j. 5. For all j, if g(z j ) = 0 and f(z j ) f(0 i, z j i+1,..., zj n), then output representatives of 3 different i-equivalence classes (0 i, y and z j 1,..., zj i ) and distinguishing assignments for them (w and z j i+1,..., zj n). Do the same if g(z j ) = 1 and f(z j ) f(y, z j i+1, zj n). 6. Output the representatives of 2 equivalence classes (0 i and y) and a distinguishing assignment for them (w). Figure 2: Algorithm Count-equiv-classes. will encounter a value z such that g(z j ) = 0 and f(z j ) f(0 i, z j i+1, zj n) or such that g(z j ) = 1 and f(z j ) f(y, z j i+1, zj n), and we will thus output the representatives of 2 different i-equivalence classes. Under no circumstances does the algorithm indicate the existence of equivalence classes for which no witness was found. It remains to prove the last item. To this end we define the function h w (x) as follows: If g(x) = 0 then h w (x) = f(0 i, x i+1,..., x n ), and otherwise h w (x) = f(y 1,..., y i, x i+1,..., x n ). For the value w determined in Item 2 it holds that if f is ɛ-far from h w then we reject with probability at least 1 δ in Item 5 (which can be seen as testing identity between f and h w ). As h w is a function of f i+1,n,w (x 1,..., x i ) and of the variables x i+1,..., x n the conclusion follows. The query complexity is linear in m, which is O(log(1/δ)/ɛ). Before describing the algorithm Get-linear-level (in Fig. 4) and proving its correctness, we describe the algorithm Test-level-linearity (see Fig. 3) that it uses as a building block. Claim 9 When given oracle access to a function f and a value i, Test-level-linearity acts as follows: 1. If f is a linear function of the output of a boolean function g(x 1,..., x i ) and of the variables x i+1,..., x n, then Test-level-linearity accepts, and outputs a sequence w such that f i+1,n,w equals g. 2. If f is ɛ-far from any linear function of any function g(x 1,..., x i ) and of the variables x i+1,..., x n, then with probability at least 1 δ Test-level-linearity rejects, possibly identifying 3 different equivalence classes. The algorithm performs Θ(log(1/δ)/ɛ) queries. 10

12 Test-level-linearity Input: Oracle access to a function f; Integer value 0 < i n; Precision parameter ɛ; Confidence parameter δ. 1. Run Count-equiv-classes(i, ɛ = ɛ/4, δ = δ/4). 2. If Count-equiv-classes returns representatives of 3 different i-equivalence classes, output reject. 3. If Count-equiv-classes returns 1 equivalence class, run the BLR linearity test on f 1,i,0 i with precision parameter ɛ = ɛ/3 and with confidence δ = δ/4. If the test accepts - output accept and 0 i as a distinguishing assignment, otherwise, output reject. 4. If Count-equiv-classes returns representatives of 2 different i-equivalence classes, x and y, and a distinguishing assignment w, do the following: (a) Run the BLR linearity test on f 1,i,x and on f 1,i,y with precision ɛ = ɛ/4 and with confidence δ = δ/4. If either test rejects, reject. (b) Select m = Θ(log(1/δ)/ɛ) strings in {0, 1} n i, denoted w 1,..., w m. (c) If f 1,i,x (w j ) = f 1,i,y (w j ) for all j, or f 1,i,x (w j ) f 1,i,y (w j ) for all j, output accept and the distinguishing assignment w. Otherwise output reject. Figure 3: Algorithm Test-level-linearity. Proof: We start with Item 1. Test-level-linearity rejects in only 4 cases, none of which happen if f is a linear function of a function g(x 1,..., x i ) and of the variables x i+1,..., x n : 1. The algorithm encounters 3 different i-equivalence classes. This is clearly impossible as g(x 1,..., x i ) can only take (at most) 2 different values. 2. The algorithm encounters a single equivalence class, and restricted to that class f is not a linear function. As any restriction on the boolean variables of a linear function gives a linear function, if f is linear, so is, e.g., f 1,1,0. 3. The algorithm encounters two equivalence classes, and restricted to one of them f is not a linear function. Again, as any restriction on the boolean variables of a linear function gives a linear function, if f is linear, so is, e.g., f 1,1,0. 4. We have two values x, y such that f 1,i,x (w j ) = f 1,i,y (w j ) on some value w j, and f 1,i,x (w k ) f 1,i,y (w k ) on some value w k. This clearly doesn t happen in a linear function f (g(x 1,..., x i ), x i+1,..., x n ) where the bit g(x 1,..., x i ) has influence, and likewise doesn t happen in a linear function f (g(x 1,..., x i ), x i+1,..., x n ) where the bit g(x 1,..., x i ) has no influence. As every bit in a linear function has either influence 1 or 0, it holds that f is not a linear function. We show the correctness of Item 2 as follows. We assume none of the sub procedures used by Test-level-linearity fails, e.g., when a call is made to the BLR linearity test it accepts only if the function it is invoked upon is indeed (ɛ/4)-close to linear, and (despite this not being an explicit 11

13 sub procedure) we assume that in Item 4c of Test-level-linearity we accepted a function f where f 1,i,x (w j ) = f 1,i,y (w j ) on all but ɛ/4 of the values or that f 1,i,x (w j ) f 1,i,y (w j ) on all but ɛ/4 of the values. The cumulative probability of one of these sub procedures failing is at most δ, and thus the conclusion will follow once we show that such success and f passing the test implies that f is (ɛ)-close to a linear function of some function g(x 1,..., x i ) and of the variables x i+1,..., x n. Assuming all sub-procedures succeed as described above, if f accepts at Step 3 then on all but at most ɛ/4 of the inputs f is close to a function that doesn t depend on the variables x 1,..., x i (by the correctness of Count-equiv-classes), and on all but at most ɛ/3 of the inputs it equals a linear function of x i+1,..., x n. It follows that f is ɛ-close to a linear function of the variables x i+1,..., x n and of a constant function (of the variables x 1,..., x i ). If f accepts at Step 4 then (by the correctness of Count-equiv-classes) it is (ɛ/3)-close to a function of the variables x i+1,..., x n and of f i+1,n,w (x 1,..., x i ) where w is the distinguishing assignment returned by Count-equiv-classes. Step 4a ensures that f is (ɛ/4)-close to a linear function when f i+1,n,w (x 1,..., x i ) = 0, and Steps 4b and 4c ensure that when f i+1,n,w (x 1,..., x i ) = 0 and when f i+1,n,w (x 1,..., x i ) = 1 the function f is either close to being the same linear function of x i+1,..., x n (making f close to a linear function of x i+1,..., x n that disregards the values x 1,..., x i ) or is close to being an opposite function (making f close to a linear function of x i+1,..., x n and of f i+1,n,w (x 1,..., x i )), as required. Get-linear-level Input: Oracle access to a function f; Precision parameter ɛ; Confidence parameter δ. 1. Let min = 1 and max = n. 2. Let w be the empty string. 3. While min < max (a) Let mid = (max + min)/2 (b) Run Test-level-linearity(f, mid, ɛ, δ = δ/ log(n)). If Test-level-linearity finds 3 different mid-equivalence classes, reject. (c) If Test-level-linearity returns accept set max = mid and set w to be the distinguishing sequence. (d) Otherwise, set min = mid return mid and w. Figure 4: Algorithm Get-linear-level. Claim 10 When Get-linear-level is given oracle access to a function f (of n variables), a precision parameter ɛ and a confidence parameter δ it acts as follows. Get-linear-level rejects only if more than 2 different i-equivalence classes were located for some i. Otherwise, with probability greater or equal to 1 δ it returns a value 1 i n and a string w so that the following hold: 1. The function f is ɛ-close to a linear function of f j+1,n,w (x 1,..., x j ) and of the variables x j+1,..., x n. 12

14 2. If j 1, then the function f is not a linear function of f j,n,w (x 1,..., x j 1 ) and of the variables x j,..., x n for any 2 w. ) The algorithm performs O queries. ( log(n) log(log(n)/δ) ɛ Proof: We prove that both items hold conditioned on Test-level-linearity never failing (i.e., never accepting when in fact f is far from every linear function of some g(x 1,..., x i ) and of the variables x i+1,..., x n ). The probability of such a failure on each round is at most δ/ log(n), which cumulatively never exceeds δ in the (at most) log(n) rounds that the algorithm performs. The correctness of Item 1 when i n follows from the correctness of Test-level-linearity according to Claim 9 - f passed the subroutine Test-level-linearity on the value i and is thus ɛ-close to a linear function of f i+1,n,w (x 1,..., x i ) and of the variables x i+1,..., x n. When i = n the result is trivial as it requires f(x 1,..., x n ) to be computed by a width-2 OBDD that gets the value of f(x 1,..., x n ) as input. The correctness of Item 2 again follows from the correctness of Test-level-linearity. Unless j = 1 the function Test-level-linearity rejected when invoked on the j 1 th level of f (by the structure of binary search), and thus f is not a linear function of f i,n,w and of the variables x i,..., x n for any value w. The query complexity follows directly from the query complexity of Test-level-linearity, which is invoked at most log(n) times with the parameters ɛ and δ = δ/ log(n). 4.2 Wrapping it all up Before proving Theorem 3 we prove a small claim that will assist us in the proof. In both the claim and the proof of the claim we describe a situation where none of the (probabilistic) sub-procedures used by Test-width-2 fail. By Procedures not failing we mean, e.g., that if the BLR test accepts, then the function is indeed ɛ-close to a linear function. Claim 11 Assuming none of the sub-procedures used by it fail, at the end of the r th round of Test-width-2, the function f r 1 is ɛ -close to a W2 function of f r (x 1,..., x t r) and the variables x t r +1,..., x t r 1. The relationship between f r 1 and f r is demonstrated in Fig Proof: By Claim 10 we have that at the end of Item 2d of Test-width-2 in the r th round, the function f r 1 is (ɛ /3)-close to a W2 function of g r (x 1,..., x j ) and the variables x j+1,..., x t r. When j = 1 this suffices (as we set f r = g r ). When j > 1 we have by Claim 8 that at Item 2(e)iv f r is set to a function ɛ /3-close to a W2 function of g r and of the variable x t r. This function is surely a W2 function and thus f r 1 is 2ɛ /3-close to a W2 function of f r (x 1,..., x t r) and the variables x t r +1,..., x t r 1, as required. Proof of Theorem 3: We prove Theorem 3 in three stages. We first show the correctness of the algorithm assuming that none of the probabilistic procedures it performed erred in any way. We follow this by bounding the probability of error for the different probabilistic procedures, and finally, we analyze the query complexity, concluding the proof. Correctness assuming the success of sub-tests involves proving the following: 2 Note that this is true with probability 1 due to the one-sided rejection criteria of Test-level-linearity, but the claim as is suffices 13

15 f r 1 (x 1,..., x t r 1 ) β r 1 f r (x 1,..., x t r ) ψ r β r 1 β r α r 1 α r Figure 5: An illustration for the construction of α r and β r 1. Completeness: Test-width-2, given oracle access to a function computable by a width-2 OBDD, accepts. 2. Soundness: Test-width-2, given oracle access to a function ɛ-far from any function computable by a width-2 OBDD, rejects with probability at least 2/3. Proof of the completeness condition is straightforward: Rejection by Test-width-2 occurs only when 3 different i-equivalence classes are detected. By Corollary 1 this never happens in a function computable by a width-2 OBDD. As Test-width-2 always terminates either by accepting a function or rejecting it, the completeness condition holds. We prove the soundness condition by proving the contrapositive - any function that passes the tester with probability greater than 1/3 is ɛ-close to a function computable by a width-2 OBDD. To this end we assume that all the sub-procedures performed by Test-width-2 succeed and show that in such a case any function passing the test is, indeed, ɛ-close to a function computable by a width-2 OBDD. We later prove that the cumulative probability of the sub-procedures failing is less than 2/3, thus ensuring that Test-width-2 is indeed a one-sided error property testing algorithm. We define the function α 0 to be f. We next construct for every round r of the algorithm a function α r that has the following properties: 1. The function α r is close to the function α r 1. In particular d(α r, α r 1 ) ɛ = ɛ/(4 log(1/ɛ)). 2. The function α r is a W2 function of f r (x 1,..., x t r) and of x t r +1,..., x n, and has at least r 1 merging levels. The W2 function that accepts as input the values f r (x 1,..., x t r) and x t r +1,..., x n is denoted β r. We construct α r based on α r 1 as follows: By Claim 11, at the end of the r th round the function f r 1 is ɛ -close to a W2 function, which we denote ψ r, of f r (x 1,..., x t r) and the variables x t r +1,..., x t r 1. Let α r = β r 1 (ψ r (f r (x 1,..., x t r), x t r +1,..., x t r 1), x t r 1 +1, x n ). As we wish to view α r as equivalent to β r (f r (x 1,..., x t r), x t r +1), we define β r accordingly (see Fig. 4.2). We have that d(α r, α r 1 ) ɛ. We note that β r is computable by a width-2 OBDD by a straightforward construction, and that unless j = 1 on the r th round, the new width-2 OBDD constructed by this procedure (that computes β r ) has one more merging level than the one on the r 1 th round. 14

16 Denoting the last round of Test-width-2 as s we now note that α s is (ɛ/4)-close to a function computable by a width-2 OBDD (assuming f passed the test). There are three ways the test can terminate successfully: 1. The test reaches the (log(1/ɛ) + 2) th round. In such a case α s is ɛ close to a W2 function (that accepts f s (x 1,..., x t s) and x t s +1,..., x n as input) with log(1/ɛ) + 2 merging levels in the OBDD computing it, and by Claim 5 is (ɛ/4)-close to a function computable by a width-2 OBDD. 2. The test terminates because t s = 1. In such a case, by Claim 11, at the end of the s th round the function f s is a function of 0 variables (a constant function), surely computable by a width-2 OBDD. 3. The test terminates because a single equivalence class was found in Item 2e. In such a case f s is (ɛ /3)-close to a constant function, as above. Let h be a function computable by a width-2 OBDD that s (ɛ/2)-close to α s, and let W 2 be the set of functions computable by width 2 OBDDs. We have as required. d(f, W 2) d(f, h) (4) d(f, α 1 ) + d(α 1, α 2 ) + + d(α s 1, α s ) + ɛ/2 (5) ɛ/2 + (log(1/ɛ) + 2)(ɛ/(4 log(1/ɛ))) (6) ɛ (7) The probability of any sub-test failing. The cumulative probability of any sub-procedure used by Test-width-2 of failing during Test-width-2 s execution is less than 2/3. This is due to the fact that in each of at most log(1/ɛ) + 2 rounds the algorithm performs two probabilistic sub-procedures, each with a probability of failure of at most 1 6(log(1/ɛ)+2). Using a simple union bound we get a total probability of failure of at most 2(log(1/ɛ) + 2) 1 6(log(1/ɛ)+2) = 1/3. The query complexity. It remains to analyze the query complexity of the algorithm. The tester repeats the outer loop at most log(1/ɛ) + 2 times, and performs queries in two Items - 2a and 2e, where the number of queries in Item 2a is by far the larger and sums up to Θ(log(n)(log(log(n)/δ )/ɛ )) where ɛ = ɛ/(12 log(1/ɛ)) and δ 1 = 6(log(1/ɛ)+2), giving us a total number of queries of ( ) log(n) log(log(n) log(1/ɛ)) log(1/ɛ) Θ ɛ and the proof is complete. References [AKK + 05] N. Alon, M. Krivelevich, T. Kaufman, S. Litsyn, and D. Ron. Testing Reed-Muller codes. IEEE Transactions on Information Theory, 51(11): ,

17 [BBTV97] F. Bergadano, N. Bshouty, C. Tamon, and S. Varricchio. On learning branching programs and small depth circuits. In COLT 97, pages , [BLR93] M. Blum, M. Luby, and R. Rubinfeld. Self-testing/correcting with applications to numerical problems. Journal of the ACM, 47: , [BTW98] N. Bshouty, C. Tamon, and D. Wilson. On learning width two branching programs. Information Processing Letters, 65: , [DLM + 07] I. Diakonikolas, H. K. Lee, K. Matulef, K. Onak, R. Rubinfeld, R. A. Servedio, and A. Wan. Testing for concise representations. In FOCS 07, pages , [EKR95] F. Ergün, R. S. Kumar, and R. Rubinfeld. On learning bounded-width branching programs. In COLT 95, pages , [FKR + 04] E. Fischer, G. Kindler, D. Ron, S. Safra, and S. Samorodnitsky. Testing juntas. Journal of Computer and System Sciences, 68(4): , [GG95] R. Gavalda and D. Guijarro. Learning ordered binary decision diagrams. In ALT 95, pages , [GGR98] [JPRZ04] [KR00] [Nak00] O. Goldreich, S. Goldwasser, and D. Ron. Property testing and its connection to learning and approximation. Journal of the ACM, 45(4): , C. S. Jutla, A. C. Patthak, A. Rudra, and D. Zuckerman. Testing low-degree polynomials over prime fields. In FOCS 04, M. Kearns and D. Ron. Testing problems with sub-learning sample complexity. Journal of Computer and System Sciences, 61(3): , A. Nakamura. Query learning of bounded-width OBDDs. Theoretical Computer Science, 241:83 114, [Nak05] A. Nakamura. An efficient query learning algorithm for OBDDs. Information and Computation, 201: , [New02] I. Newman. Testing membership in languages that have small width branching programs. SIAM Journal on Computing, 31(5): , [PRS02] M. Parnas, D. Ron, and A. Samorodnitsky. Testing basic boolean formulae. SIAM Journal on Discrete Math, 16(1):20 46, [RS96] R. Rubinfeld and M. Sudan. Robust characterization of polynomials with applications to program testing. SIAM Journal on Computing, 25(2): , [RW93] V. RagHavan and D. Wilkins. Learning branching programs with queries. In COLT 93, pages 27 36,