Exploiting Correlation in Stochastic Circuit Design

Transcription

1 Eploiting Correlation in Stochastic Circuit Design Armin Alaghi and John P. Haes Advanced Computer Architecture Laborator Department of Electrical Engineering and Computer Science Universit of Michigan, Ann Arbor, MI, 4809, USA {alaghi, Abstract Stochastic computing (SC) is a re-emerging computing paradigm which enables ultra-low power and massive parallelism in important applications like real-time image processing. It is characteried b its use of pseudo-random numbers implemented b 0- sequences called stochastic numbers (SNs) and interpreted as probabilities. Accurac is usuall assumed to depend on the interacting SNs being highl independent or uncorrelated in a loosel specified wa. This paper introduces a new and rigorous SC correlation (SCC) measure for SNs, and shows that, contrar to intuition, correlation can be eploited as a resource in SC design. We propose a general framework for analing and designing combinational circuits with correlated inputs, and demonstrate that such circuits can be significantl more efficient and more accurate than traditional SC circuits. We also provide a method of analing stochastic sequential circuits, which tend to have inherentl correlated state variables and have proven ver hard to anale. Kewords Stochastic computing; signal correlation; low-power design; stochastic circuit design. I. INTRODUCTION Stochastic computing (SC) was introduced in the 960s as a wa to implement comple computing tasks at low hardware cost [2] [6] [7]. Its ke feature is the use of long random bitstreams called stochastic numbers (SNs) to represent the data being processed. Fig. a shows how a single AND gate can multipl two n-bit SNs in n clock ccles. Here,, denote logic functions, and the 8-bit sequences X, Y, Z denote the corresponding SNs with their probabilit values,, in parenthesis. The probabilit of appearing on is equal to the probabilit of on, multiplied b the probabilit of a on, assuming that the input signals are independent or uncorrelated. SC has seen little use in practice because it involves comple and poorl understood trade-offs among circuit sie, computation time, and accurac. There has been a recent resurgence of interest in SC as it has been shown to be ver cost-effective in some useful applications, notabl lowdensit parit check (LDPC) decoding [9] [6] and image processing [3] [3]. Stochastic circuits have other nice properties such as ver low power and high error tolerance. On the other hand, SC has several drawbacks that need to be addressed to obtain practical circuits. In particular, independent inputs have been seen as necessar for SC because correlated inputs can produce inaccurate results [2]. This is illustrated b Fig. b where the bit-streams X and Y are identical, and so are maimall correlated. The output Z now has the value instead of the desired product. X = (4/8) Y = (4/8) X = (4/8) Y = (4/8) Z = (2/8) Z = (4/8) Figure. Stochastic multiplication; accurate result with uncorrelated inputs; inaccurate result due to correlated inputs. Fig. 2 shows some basic components for constructing stochastic circuits. Addition is implemented b a two-wa multipleer in the scaled form ( + )/2, which ensures that the sum of two SNs lies in the probabilit interval [0,]. Note that the scaling factor is supplied b an independent SN W of fied value 0.5, i.e., a purel random bit-stream. The remaining two circuits convert numbers between ordinar binar form N and stochastic form X with = N/2 k. The (pseudo) random number generator in Fig. 2c is tpicall implemented b a linear feedback shift register (LFSR) [0]. Signed numbers can be handled b bipolar notation, which maps the SN range from [0,] to [,]. If X is an SN with (unipolar) value, its corresponding bipolar value is. The circuits of Fig. 2 require ver minor modification to handle bipolar numbers. For eample, an XNOR gate performs the bipolar multiplication, while the multipleer of Fig. 2b continues to serve as a scaled adder for bipolar SNs. Most SC circuits designed so far do not behave satisfactoril when their inputs are even moderatel correlated. The general solution to this problem has been to avoid correlation as much as possible, either b using independent sources for all input SNs, or else b selectivel re-randomiing correlated SNs. The latter involves first 0 AND Multipleer Random number W (c) (d) generator Clock Binar number N Stochastic number X k k Clock A B Binar counter Comparator A<B k Stochastic number X Binar number N Figure 2. Unipolar SC components: multiplier scaled adder (c) binar-to-stochastic converter (d) stochastic-to-binar converter.

2 w w (4/8) (4/8) 0 0 (6/8) (4/8) (4/8) 0 0 (6/8) (5/8) (5/8) Figure 3. Stochastic addition showing accurate results with uncorrelated and correlated inputs X and Y. converting an SN to binar form and then converting it back to stochastic form via circuits like those of Fig. 2c-d. These steps come at high cost, imposing as much as 80% area overhead on one stochastic image-processing circuit [8]. In an iterative LPDC decoder [6], SNs must be re-randomied in ever iteration to avoid deadlocks. Correlation is poorl understood in the SC contet, in contrast with areas such as communications theor [0]. The onl relevant stud we know of is due to Jeavons et al. [], who do not provide an measure of correlation among bitstreams or its impact on SC behavior. The define SNs X and Y to be independent if, where is the SN obtained b ANDing X and Y. This, in effect, sas that a stochastic multiplier s inputs are uncorrelated if the output is accurate. It gives no hint of what happens with correlation present. Ma et al. [5] discuss the propagation of inaccurac through stochastic circuits under the assumption of independent inputs, and do not anale correlated inputs. This paper introduces a rigorous measure of correlation among SNs, and shows how to anale circuits with correlated inputs. It leads to an interesting conclusion: contrar to general belief, correlation is not alwas harmful in SC. In fact, it can even be eploited to design better circuits! Fig. 3 gives a motivating eample. The multipleer implements which, with = 0.5, is the scaled sum. If all inputs are independent as in Fig. 3a, the circuit performs the add operation accuratel. In Fig. 3b, ever 0 of X coincides with a from Y, impling that X and Y are (negativel) correlated, but the circuit still computes accuratel. This will be eplained in Sec. II. The net eample illustrates a situation where, counter to most intuition, correlation is actuall beneficial. The absolutevalued subtraction function is useful in image processing [3]. If it is implemented under the usual assumption of independent inputs, a large stochastic circuit is needed [3]. However, it can also be implemented b a single XOR gate fed with highl correlated inputs in which there is maimum overlap of s and 0s between the two input SNs. In that case, the probabilit of getting two s (or two 0s) on and is (or ), impling that the probabilit of different values on and becomes if (5/8) (4/8) (/8) Figure 4. XOR gate with correlated inputs implementing absolutevalued subtraction., as in Fig. 4, or else if. In other words, the output represents. Note that an XOR fed with independent SNs X and Y implements an entirel different function, namel,. These eamples demonstrate correlation s significance in SC, and suggest that advantage can be taken of correlation in certain cases. We introduce here a measure of correlation for SC and a method of analing SC circuits based on the probabilistic transfer matri (PTM) methodolog [2]. We also develop a method of snthesiing combinational circuits that work in the presence of correlation. Finall, we discuss the analsis of sequential stochastic circuits, which is ver poorl understood at present. Prior work is limited to a few special cases with regular structures [4] [4]. General sequential circuits are difficult to anale because of correlation among the state variables. The correlation analsis of this paper provides a wa to tackle these circuits. The main contributions of this paper are A new correlation measure SCC for SNs Use of PTMs to anale stochastic circuits A stud of combinational circuits with correlated inputs and their application to some useful SC designs Analsis of general sequential stochastic circuits The paper is organied as follows. Sec. II discusses PTMs and their role in SC. A rigorous analsis of correlation in the SC contet is presented in Sec. III. Then Sec. IV deals with combinational stochastic circuits, while Sec. V addresses sequential circuits. Some conclusions are drawn in Sec.VI. II. PROBABILISTIC TRANSFER MATRICES The probabilistic transfer matri (PTM) [2] has proven valuable in the probabilistic analsis of logic circuits, such as circuits subject to soft errors. Here we show that PTMs are also useful for analing correlation in stochastic circuits. In the usual SC scenario, a circuit with m inputs,, 2,, m is analed using the probabilit values of m bit-streams applied to its input lines, assuming that these bit-streams are independent. In the PTM formulation, the same input data is represented b a stochastic vector V of sie 2 m. The elements of V are the probabilities of the possible input combinations of 2 m, which can var from 00 0 to. For eample, V = [/2 0 0 /2] is the PTM corresponding to two inputs and, when the probabilit of = 00 and = is /2 and the other probabilities are ero. The PTM has more information than conditional probabilit values it also implicitl contains correlation information. For eample, V = [/2 0 0 /2] indicates that SNs X and Y are highl correlated because the probabilit of having different values on and is

3 alwas ero. The vector V 2 = [/4 /4 /4 /4], on the other hand, represents two completel uncorrelated SNs. Ever logic circuit has a PTM representing its error-free function. This is a matri of sie 2 m 2 l, where m and l are the numbers of inputs and outputs of the circuit, respectivel. The PTM of an AND gate implementing = is The entr p i,j is the (conditional) probabilit of input i producing output j. Multipling an input PTM b a circuit PTM ields an output PTM, which for a single-output gate is a vector [o 0 o ] where o 0 and o are the probabilities of 0 and, respectivel, appearing at the output. For eample, the SC operation performed b the XOR circuit of Fig. 4 is described b the PTM calculation [ ] [ ] [ ] It was stated earlier that the adder of Fig. 3 has input SNs X and Y which can be correlated without causing inaccurac. We now prove this via PTM analsis. The input vector has the form I = [i 0 i i 2 i 3 ], in which i 0, i, i 2 and i 3 denote the probabilit of being 00, 0, 0 and, respectivel. The input SN W is a constant of value /2 and so is omitted from the PTM. The adder s PTM is therefore [ ] The corresponding output vector is given b [ ] [ ] [ ] Note that and alwas holds, whether X and Y are independent or correlated. Hence, ( ). The input vectors [/8 3/8 /8 3/8] and [0 /2 /4 /4] corresponding to the inputs X and Y of Figs. 3a and 3b, respectivel, both lead to the same output vector [3/8 5/8]. III. CORRELATION OF STOCHASTIC NUMBERS Correlation refers to statistical similarit between two phenomena. As discussed in detail in [0], the correlation of two sequences (bit-streams) is measured b some form of covariance or dot-product operation. With appropriate normaliation, a correlation value of + means maimum similarit, a correlation value means minimum similarit (maimum difference), and a correlation of 0 means the sequences are uncorrelated. While man measures of similarit eist [5], the are not ver useful in the SC contet. The standard definition of correlation (also known as the Pearson correlation [5]), in particular, is unsuitable because it imposes constraints on the epected value of the bit-streams. For eample, implies that the bit-streams must be identical. A suitable similarit measure should be independent of, or orthogonal to, the data values; in other words, it should not impose constraints on the data. We therefore propose a new correlation measure defined as follows. Definition : The SC correlation SCC(X, Y) of two SNs X and Y is given b { The starting point in constructing SCC(X, Y) is obtaining the bit-wise AND function (a kind of dot-product) of the SNs, that is, finding. This is then centralied b the uncorrelated value ielding Finall, the centralied value is normalied b dividing it b the maimum possible values. The centraliation makes SCC consistent with the definition of independence in [] when, i.e., when. The normaliation guarantees that for two maimall similar (or different) SNs X and Y, we get (or ). Unlike the standard correlation measure, SCC does not var with the SN values. All the intermediate values of SCC are linearl interpolated between the independent case and the maimum similarit (or difference) case. For eample, means that is half-wa between, i.e., the independent case, and, i.e., the maimum overlap case. We can also define SCC using the notation of [5], which allows eas comparison with other correlation concepts. For two n-bit SNs X and Y, denote the number of overlapping s b a, the number of overlapping s of X and 0s of Y b b, the number of overlapping 0s of X and s of Y b c, and the number of overlapping 0s on both SNs b d. Clearl,. We then have the following definition which is equivalent to Def. : { The numerator ad bc is common to man similarit measures including Pearson correlation and it captures the overlap of 0s and s in the two bit-streams. The denominator, on the other hand, is simpl a normaliation factor. While Pearson correlation is normalied b the variance of the bit-streams, SCC is normalied so that the bitstreams with maimum (minimum) overlap of s and 0s lead to SCC = + ( ), independent of the values of the SNs. Table I shows eamples of bit-streams with their and SCC values. Note that and SCC are the same for independ-

4 TABLE I. SOME SNS WITH THEIR SCC AND STANDARD CORRELATION VALUES Stochastic numbers SC correlation Standard correlation X = 0000 Y = X = 0000 Y = X = 0000 Y = 0000 X = 00 Y = X = 00 Y = X = 00 Y = X = Y = ent SNs, and for SNs with equal values. When the SNs have different values, SCC consistentl gives the value + (or ) for maimum (minimum) overlap of s and 0s between the bit-streams, while gives different values. This shows that, unlike, SCC is not affected b the values of the bit-streams. As mentioned earlier, the function of a stochastic circuit can effectivel be changed b enforcing correlations among its inputs. The XOR gate of Fig. 4 illustrates this. Fig. 5a shows the stochastic functions implemented b the same XOR gate at different levels of SCC. In all cases, the output of the function remains the same at the four corners, but the function changes greatl for the intermediate values. Fig. 5b shows the same function for various fied values of and SCC. IV. COMBINATIONAL CIRCUITS Ever stochastic circuit implements a real-valued function F, which is interpreted as its stochastic behavior. For eample, the AND gate of Fig. implements the multiplication function = F(, ) =, assuming. The inputs of F are the values of the SNs and. Hence, to obtain the stochastic behavior of a logic circuit, we need to determine its corresponding probabilit function F. We saw earlier that a circuit s functionalit can change in the presence of correlation. The AND gate, for eample, implements = F(, ) = min(, ) if. Table II. FUNCTIONS FOR THE PTM ELEMENTS OF A TWO-INPUT CIRCUIT (, ) SCC = 0 i 0 (, ) SCC = (, ) SCC = + ma min i min ma i 2 min ma i 3 ma min Based on Def. and the subsequent discussion, the SC function of the AND gate for the case of, sa, is half-wa between its SC function for and. In general, we can epress a circuit s functionalit as a linear combination of its functions at and or. Hence for an SCC, can be written as { where, and are the functions of the same circuit at and +, respectivel. For the AND gate eample, we have,, and. In order to derive the stochastic behavior of a circuit C with correlated inputs, we use the PTM tools discussed in Sec. II. If C has two input bit-streams X and Y with correlation SCC, construct the input PTM I = [i 0 i i 2 i 3 ], in which the i k s are epressed in the form of Eq. (3). The s, s, and s corresponding to each i k are defined in Table II. From these, we can etract C s stochastic behavior b multipling the vector I b the circuit PTM. For instance, if C is an XOR gate, [ ] [ ] ields the functions illustrated in Fig. 5a for some representative values of SCC. (3) p X p X SCC = SCC = 0.5 SCC = 0 SCC = 0.5 SCC = SCC= SCC= /2 SCC= SCC=+ SCC=0 +/2 SCC=+ SCC=+ = 0 = 0.25 = 0.5 = 0.75 = Figure 5. Functions implemented b an XOR gate with different input SCC values, and with fied values of.

5 t 0 t t 2 t Figure 6. High-level structure of a snthesied two-input circuit with correlated inputs, prior to simplification. Onl correlation corresponding to the special case has been considered in the literature. As we have just seen, other cases such as lead to useful results. The PTM formulation of two-input SC circuits discussed above points to a method for snthesiing stochastic circuits with correlated inputs. In this approach, a target function F(, ) is approimated b a function = (, ) defined b [ ] [ ] [ ] (4) in which the t k s are the parameters of F to be determined and the i k s are epressed in the form of Eq. (3) and Table II. The process of approimation is to find the best t k s and the best SCC for which the following error function is minimied. ( ) (5) This is done b adjusting the t k s and the SCC using standard optimiation techniques. Because of the limited number of parameters and the well-behaved error function, this problem is relativel eas to solve. Once the parameters are found, can be realied b a logic circuit with the overall multipleer-stle structure shown in Fig. 6, The constant SNs needed for the four t k s can be generated b up to four copies of the circuit in Fig. 2c. The resulting circuit can then be further optimied using conventional logic snthesis methods and tools. Generating two SNs X and Y with a desired level of SCC is a problem that has not been studied before. We propose to use the circuit structure shown in Fig. 7, which generates X b generator generator 2 generator 3 SCC magn r r 2 r 3 SCC sign Figure 7. Generating SNs with a specified SCC. Step : Determine a suitable approimating function according to Eq. (4) in which the input vector I is defined b Eq. (3) and Table II. Step 2: Determine the error function given b Eq. (5). Step 3: Minimie b adjusting the t k and SCC parameters in. Step 4: Insert these parameters into the structure of Fig. 6, and use standard logic snthesis methods to optimie the resulting circuit. Figure 8. Algorithm CCC to snthesie a stochastic function with correlated inputs. means of a standard SN generator and, depending on the sign SCC sign and magnitude SCC magn of SCC, mies uncorrelated and correlated versions of Y together. For eample, if SCC = +, then Y is generated from the same random number source used b X, so X and Y become highl correlated. Note that Fig. 7 is a programmable structure, and not all the components shown are needed in ever design. For eample, if a circuit requires X and Y to be generated with SCC = +, then the select inputs of the multipleer are set to 0, impling that random number generators 2 and 3, and their associated circuits can be removed. Fig. 8 summaries our proposed correlated combinational circuit (CCC) snthesis algorithm. As an eample, consider the problem of snthesiing a circuit for the target function In Step of CCC, is prepared in the form of Eq. (4), and in Step 2 the error function is prepared in the form of Eq. (5). Then, the t k s and SCC are adjusted until is minimied. For the running eample,, i.e., the eact target function, can be achieved b assigning t 0 = 0, t = 0, t 2 = 0, t 3 =, and SCC = +. On plugging these values into the circuit of Fig. 6, we obtain that of Fig. 9a, i.e., an AND gate. Observe that the AND implements onl if its inputs have SCC = +. In order to generate X and Y, we use the circuit of Fig. 7 and plug in SCC = + (SCC sign = 0, SCC magn = ), ielding the circuit of Fig. 9b. This circuit is smaller than one emploing two of the independent SN generators in Fig. 2c, so generating correlated SNs is cheaper than generating independent ones in this case. Table III shows eamples of circuits snthesied b the CCC algorithm. Most of the target functions are useful nonlinear functions that have no efficient stochastic implementation when the inputs are uncorrelated. The last snthesied function in the table is the multipleer-based scaled adder in which correlation of the input data does not matter. Another tpe of SC adder, a saturating adder, is also shown. This generator Figure 9. Implementing the function : snthesied stochastic circuit, and corresponding SN generator. r

6 TABLE III. EXAMPLES OF SYNTHESIZED STOCHASTIC CIRCUITS EXPLOITING VARIOUS CORRELATION LEVELS Target function [ ] SCC Snthesied circuit [ ] + AND gate with positivel correlated inputs [ ] + OR gate with positivel correlated inputs [ ] + XOR gate with positivel correlated inputs; implements absolute-valued subtraction [ ] OR gate with negativel correlated inputs; implements saturating add [ ] An Multipleer with arbitrar correlation among its data inputs; implements scaled add circuit adds its inputs without scaling until the saturating value is reached. Finall, observe that in some cases, the circuits snthesied b CCC are the same as the standard designs. For eample, the smallest SC multiplier is the AND gate of Fig., which requires uncorrelated inputs. This shows that the CCC is capable of replicating circuits snthesied b eisting methods, because SCC = 0 is also allowed in CCC. Table IV compares the circuits snthesied b CCC and those designed b the spectral snthesis method of [], which makes the usual independent-inputs assumption, i.e., SCC = 0. In addition to the circuits of Table III, a few other functions were implemented. Since it is normall impossible to implement real-valued functions eactl, some are approimated before snthesis. Area is estimated b mapping the circuits to a generic librar of cells using 0.35 m CMOS technolog [9]. For a fair comparison, we also report the measured mean error between the snthesied and target functions F and F. The results indicate that in most cases, the circuits snthesied b CCC are smaller and more accurate than those designed b the method of []. When dealing with more than two signals, considering their pairwise SCC values ma be insufficient, as higher-order correlations can eist among groups of three or more of the signals. To handle such cases, we suggest using PTMs that are large enough to embed all the signal correlations of interest. Circuits with man inputs can also be designed b decom- TABLE IV. COMPARISON BETWEEN CIRCUITS SYNTHESIZED IN THIS PAPER AND THOSE SYNTHESIZED BY THE SPECTRAL METHOD OF [] Target function Saturating adder Saturating subtracter Multiplier Scaled adder Snthesis method and correlation assumption Area* ( m 2 ) Mean Error (%) [] with SCC = 0,628 0 CCC with SCC =,09 0 [] with SCC = 0,663 0 CCC with SCC = +,88 0 [] with SCC = 0,646 0 CCC with SCC = 0,646 0 [] with SCC = 0,857 0 CCC with SCC = +,320 0 [] with SCC = 0,980 2 CCC with SCC = +,443 7 [] with SCC = 0 2,306 5 CCC with SCC =,760 9 A multi-variate [] with SCC = 0 2,086 9 polnomial CCC with SCC = /2 2,473 4 * Circuits with uncorrelated (independent) inputs were snthesied according to [] with polnomials of degree. All the reported area numbers include stochastic number generators. v w Figure 0. Stochastic circuit for image edge-detection [3]. posing the target function into subfunctions of two variables. The CCC method can then be used to snthesie the pieces and put them back together. For eample, consider the fourvariable function which performs the ver useful image-processing task of edge detection [3]. It decomposes into two absolute-valued subtraction functions and, which are then combined b a scaled add to produce : 0 r,, All three of these functions can be snthesied b CCC, as indicated in Table III. Fig. 0 shows the result; the XOR gates perform absolute-valued subtraction, while the multipleer performs scaled addition. Note that the select input of the multipleer is fed b an auiliar input r with p r = /2. V. SEQUENTIAL CIRCUITS As with conventional binar logic, stochastic sequential circuits (stochastic FSMs) can lead to more efficient designs than combinational ones. For instance, Fig. shows a sequential update node of the tpe commonl used in stochastic LDPC decoders [7]. It implements the function for which no more efficient combinational equivalent is known. Sequential stochastic circuits have also been proposed to implement arithmetic functions such as division and hperbolic tangent [4] [6] [4]. However, the proposed designs are mostl ad hoc, or require state transition diagrams of a ver restricted structure [4] [4]. Analing stochastic sequential circuits with arbitrar state transition diagrams is difficult since the state variables tend to be correlated. For eample, in the three-state circuit of Fig. 2, the state never occurs, so the SNs corresponding to state bits w 0 and w are correlated. A general method to anale and design arbitrar stochastic sequential circuits does not presentl eist.

7 (2/8) 0 0 (6/8) clock J K Q (4/8) Figure. Stochastic update node used in LDPC decoders. A sequential logic circuit implements two combinational functions: the net-state function δ that, given the current state and the current inputs, produces the net state, and the output function λ that produces the corresponding output signals. In the contet of SC, net-state transitions are treated as probabilistic. Hence, we are mainl interested in the stochastic behavior of the state-defining function δ, since λ is a simple combinational function and can be analed b eisting methods. Like our approach to combinational circuits with correlation, we use PTMs to anale sequential circuits. The state probabilit distribution of a sequential circuit is represented b a vector S = [s s 2 s l ] in which s i denotes the probabilit of being in state i. For eample, the vector [/3 /3 /3] corresponding to the circuit of Fig. 2 indicates that the probabilit of being in each state is /3. The state transition behavior of the sequential circuit can be epressed as a transition matri T, in which element t ij denotes the probabilit of a transition from state i to j. The transition matri corresponding to Fig. 2 is [ ] Given the state probabilit distribution S(t) in a particular clock ccle t and the transition matri T, we can write the state distribution of the net clock ccle t + as S(t + ) = S(t) T. For instance, with S(t) = [/3 /3 /3] and =, we get S(t + ) = [0 /3 2/3]. After enough clock ccles, the state distribution of the sequential circuit tpicall converges to a stationar distribution, which can represent the circuit s stochastic behavior. Fortunatel, finding is a well-known mathematical problem. The transition matri T of a sequential circuit can be interpreted as a Markov chain [8], which is an FSM with probabilistic state transitions. The stationar state of a Markov chain is an eigenvector of T with the defining propert = T. For eample, the stationar distribution of the circuit in Fig. 2 is [ ] [ ] [ ] And since the output onl becomes in the state w w 0 = 0, i.e., the second state, we conclude that. D D Q w 0 w Q ' ' w w ' 0 Figure 2. A sequential stochastic circuit C and its state transition diagram. In order to find the stationar distribution of an l-state FSM with an arbitrar transition matri T, we need to solve the equation = T and compute the elements of = [s s 2 s l ]. The equation corresponding to each state is of the form, in which the G i s collectivel represent the stochastic behavior of the net-state function δ of the FSM. As noted earlier, the function δ is combinational, and according to [], its stochastic behavior can be represented as a polnomial. This implies that all the G i s, and hence all the elements of T, are polnomial functions with respect to the input SN values. This fact allows us to obtain an analtical solution to = T and leads to the following result. Theorem: Given a sequential circuit with m inputs,, m, and a transition matri T whose elements are polnomial functions of, its stationar state distribution has elements of the form F / F 2, where F and F 2 are polnomial functions. The theorem follows from the fact that in solving = T, the coefficients of the variables s i become parameteried polnomial functions. The standard row-transformation steps in solution methods like Gaussian elimination onl appl the operations addition, multiplication, and division to the matri elements. Since these are polnomial functions, adding and multipling them produces polnomials, and dividing them gives rational polnomials. The degrees of the polnomials depend on the number of states l and the degrees of the polnomial elements of T. If these polnomials are linear, the final solution is of degree 2 l 2. For eample, the transfer matri corresponding to the twostate LDPC update circuit of Fig. is [ ] To find the stationar distribution of T, we form the equations () Since the circuit is either in state s 0 or s, we have the additional constraint s 0 + s =. Hence, s = s 0, ielding

8 Thus, we obtain the epected function [7]: Another eample is a four-state sequential circuit from [4], which has the transition matri [ ] Now = T ields the following set of equations: with the additional constraint Solving these equations analticall we obtain, etc., which is consistent with the analsis of [4]. VI. CONCLUSIONS Stochastic computing (SC) has recentl re-emerged as an attractive alternative technique for some important computing tasks with etreme demands for small sie and low power. A ke unsolved problem in designing stochastic circuits has been to overcome the computational inaccuracies that result from undefined and undesired correlations among signals. The usual solution has been to avoid correlation entirel at the cost of introducing man independent stochastic number sources or re-randomiers. This paper has investigated in depth the impact of correlation on stochastic computing. We have shown that not all correlation is harmful. In fact, contrar to what one would intuitivel epect, correlation can serve as a resource in designing stochastic circuits. We have given the first general and rigorous definition of correlation for SC, which has enabled us to anale both combinational and sequential stochastic circuits in the presence of correlation. We demonstrated how probabilistic transfer matrices aid this analsis, and lead to a general approach to designing stochastic circuits with correlated inputs. We further demonstrated how to implement various useful functions such as saturating addition and subtraction that had no previous efficient SC implementations. We reported a comparative stud indicating that the circuits with correlated inputs are generall smaller and more accurate than those with independent inputs. Finall, we presented the first sstematic analsis of sequential stochastic circuits, a problem which has generall resisted attack since the 960s. ACKNOWLEDGEMENTS This work was supported b Grant CCF-0742 from the U.S. National Science Foundation. The authors are grateful to Igor L. Markov for helpful discussions. REFERENCES [] Alaghi, A. and Haes, J.P., A spectral transform approach to stochastic circuits, Proc. ICCD, pp , 202. [2] Alaghi, A. and Haes, J.P., Surve of stochastic computing, ACM Trans. Embedded Computing Sstems, 2, no. 2s, pp. 92:-92:9, Ma 203. [3] Alaghi, A., Li, C. and Haes, J.P., Stochastic circuits for realtime image-processing applications, Proc. DAC, pp. 36:-6, June 203. [4] Brown, B.D. and Card, H.C., Stochastic neural computation I: computational elements IEEE Trans. Computers, 50, pp , 200. [5] Choi, S.S., Cha, S.H. and Tappert, C., A surve of binar similarit and distance measures, Journ. Sstemics, Cbernetics and Informatics, 8, pp , 200. [6] Gaines, B.R., Stochastic computing sstems, Advances in Information Sstems Science, 2, pp , 969. [7] Gaudet, V.C. and Raple, A.C., Iterative decoding using stochastic computation, Electronics Letters, 39, , [8] Grinstead, C.M., and Snell, L.J., Grinstead and Snell s Introduction to Probabilit, version of 4 Jul 2006, American Math. Soc., [9] Gross, W.J., Gaudet, V.C. and Milner, A., Stochastic implementation of LDPC decoders, Proc. Asilomar Conf., pp , [0] Golomb, S.W. and Gong, G., Signal Design for Good Correlation, New York: Cambridge Univ. Press, [] Jeavons, P., Cohen, D.A. and Shawe-Talor, J., Generating binar sequences for stochastic computing, IEEE Trans. Info. Theor, 40, pp , 994. [2] Krishnaswam, S., Viamontes, G.F., Markov, I.L. and Haes, J.P., Probabilistic transfer matrices in smbolic reliabilit analsis of logic circuits, ACM Trans. Design Automation of Electronic Sstems, 3, no., pp.8:-8:35, Jan [3] Li, P. and Lilja, D.J., Using stochastic computing to implement digital image processing algorithms, Proc. ICCD, pp. 54-6, 20. [4] Li, P., Qian, W., Riedel, M.D., Baargan, K. and Lilja, D.J., The snthesis of linear finite state machine-based stochastic computational elements, Proc. ASP-DAC., pp , 202. [5] Ma, C., Zhong, S. and Dang, H., Understanding variance propagation in stochastic computing sstems, Proc. ICCD, pp , 202. [6] Naderi, A., Mannor, S., Sawan, M. and Gross, W.J., Delaed stochastic decoding of LDPC codes, IEEE Tran. Signal Proc., 59, pp , 20. [7] Poppelbaum, W.J., Afuso, C. and Esch, J.W., Stochastic computing elements and sstems, Proc. AFIPS Fall Joint Computer Conf., pp , 967. [8] Qian, W., LI, X., Riedel, M.D., Baargan, K. and Lilja, D.J., An architecture for fault-tolerant computation with stochastic logic, IEEE Trans. Computers, 60, pp , 20. [9] Sentovich, E.M. et al., SIS: A Sstem for sequential circuit snthesis, Univ. of California, Berkele, Tech. Report UCB/ERL M92/4, Electronics Research Lab, 992.