A Likelihood Ratio Formula for Two- Dimensional Random Fields

Size: px

Start display at page:

Download "A Likelihood Ratio Formula for Two- Dimensional Random Fields"

Ami Wells
5 years ago
Views:

1 418 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-20, NO. 4, JULY 1974 A Likelihood Ratio Formula for Two- Dimenional Rom Field EUGENE WONG, FELLOW, IEEE Abtract-Thi paper i concerned with the detection of a rom ignal in white Gauian noie when both the ignal the noie are two-dimenional rom field. The principal reult i the derivation of a recurive formula for the likelihood ratio relating it to certain conditional moment of the ignal. It i alo hown that, except for ome relatively unintereting cae, a imple exponential formula for the likelihood ratio, uch a one ha in one dimenion, i not poible. A I. INTRODUCTION DDITIVE white Gauian noie ha proved to be a ueful model for many ignal proceing problem. On the one h, it i often an adequate approximation of the underlying phyical ituation. On the other h, the aumption of additive white Gauian noie often allow the analyi to be carried to fruition. Thi i illutrated by the familiar binary detection problem outlined here. Let ct, 0 I t 5 T, be the oberved proce. Let S,, 0 5 t < T, be a poibly rom ignal. We wih to tet between the two hypothee Ho: 5, i a zero-mean Gauian proce with covariance function E,,~,~, = (t - ) H, : qt = 5, - S, i a zero-mean Gauian proce with covariance function E,q,q, = (t - ). (1.1) Under rather general condition on S,, the likelihood ratio i given by the well-known formula (ee, e.g., [l]) T L = exp $,c dt , dt (1.2) where 3, = El@, I C,, 0 5 z I t) the integral J;f $5, dt i to be interpreted a an It6 integral. Equation (1.2) how that the likelihood-ratio detector can be realized by an etimator matched-filter combination provided that the matched filter operation (viz: JE $,c, dt) i carefully interpreted a an It6 integral. The likelihood ratio can alo be expreed in a recurive form. Let L, be defined by L, = E,(L I 5,, z < t). (1.3) Then an application of the Ita differentiation rule (ee, e.g., [7]) how that L, atifie the equation I L, = 1 + L,$t, dz (1.4) Manucript received October 24, 1973; revied January 21, Thi work wa upported in part by the U.S. Army Reearch Office, Durham, N.C., under Contract DAHC C-0046 in part by the National Science Foundation under Grant GK-10656X3. The author i with the Department of Electrical Engineering Computer Science the Electronic Reearch Laboratory, Univerity of California, Berkeley, Calif I Fig. 1. Parameter value for conditional moment. which how that the incremental change in L, i expreed by dl, = Lt+dt - L, = L$, (, 5, dr) where the right ide depend on the current value of L, 9, on the new obervation J : 5, dz. In thi paper we hall derive a generalization of (1.4) for two-dimenional rom field. Let &ti,t2), 0 I t, I T,, 0 I t, 5 T2, be the oberved field let S,, t E [0,7,] x [O,T,], be the ignal field. Again we wih to tet between the pair of hypothee given by (1.1). If we now let L, be defined by L, = Eo(L I 5,, 7 E [W,] x [W,]) (1.5) then our reult will imply that the incremental change dl, = L (tt+dtl,t2tdt2) - L(t,+dt,,tl) - L(tt,t2+dt2) + L(tt,t2) i expreible in term of L, (1.6) 3, = &(S, I t,, z E [W,] x [W,]) (1.7) E [] x [W,]. (1.8) Fig. 1 give an another interpretation of the previou reult. The increment dl, can be expreed in term of L, the conditional mean of S, (z being on the boundary of the rectangle), given the value of < in the rectangle, of the conditional covariance of pair of value of S, one on each leg of the boundary, given the value of 5 in the rectangle. If the ignal S the noie q are jointly Gauian (under hypothei H,) then the conditional covariance will be a determinitid function. In uch a cae, the conditional mean of S on the boundary given the obervation in the rectangle L, will uffice to determine dl,.

2 WONG: LIKELIHOOD RATIO FORMULA 419 II. THE ENER PROCESS III. STOCHASTIC INTEGRALS A in one dimenion, we define a two-parameter white Since both (1.2) (1.3) involve tochatic integral, it i noie (u],, t E R2} a a zero-mean rom field with not urpriing that we need a generalization of the Ita Ev],v],, = 8(t - t ). (2.1) integral to the two-dimenional parameter cae. Thi can be done a follow [5], [6]. Jut a in the one-dimenional cae, the calculu of white Let {W,, t E R, } be a Wiener proce {c$~, t E Rt2) noie i made precie by the introduction of a rom field be a rom field uch that a follow: fl f2 (a) for any t E Rt2, (c$,, W,, E A,} i independent of w, = Yh,d d, d,, t E R+2 = [O,OO)~ (2.2) {J+ (A), A = A,) (b),+* which i a zero-mean rom field with E&2 tit < co. (3.1) EW,W,, = min (tl,ti ) min (t2,t2 ). (2.3) If rl i Gauian, we hall call W a Wiener proce. Actually, it i convenient to introduce a rom et function W(A) = J P A vl d, A c R2 (2.4) We interpret [a) to mean that future increment of W are independent of the pat of W, 4. Firt, take a rectangle A of finite area ubdivide it by a equence of partition II,, = {A,,,} uch that max, area (A,, ) ZO. We define hw(dt) = lim in 9.m. C 4,,,,W(A,,J (3.2) which ha a covariance property A n-+m Y where lim in q.m. i the limit in the quadratic mean t,,, EW(A)W(B) = d. (2.5) denote the lower left corner of A,,n; i.e., "AnB t If rl i a Gauian white noie, then W(A) i an additive y,n = (inf t,, inf t2), t E 4w proce (or a proce with independent area). That i, if It i clear that (3.2) make the tochatic integral a forward A,,-42,. * *,A,,, are dijoint et, then W(A,), W(A,),..., increment integral. To complete our definition et W(A,) are independent rom variable. For point in R2, we define a partial ordering > by c$,w(dt) = lim in q.m. d+w(dt). (3.3) R+Z m+m t > t e t, 2 t1, t, 2 t,. The tochatic integral o defined ha a number of With repect to >, the Wiener proce i a martingale; i.e., important propertie which are direct generalization of onedimenional counterpart. E(W, I W,, -< t ) = w,,, t < t. (2.6) Linearity: To prove (2.6), let A, denote the rectangle [O,t,] x [O,t,] r (a4 + WJWdt) A, denote it complement. Then, for t > t we have JR+~ W, = W(A,) = W(A,) + W(A, A A,,). =a Since A,, A, n A,, are dijoint, W(A,,) W(A, n A,,) are independent, o that E(W, I W,, < t ) = W(A,,) + EW(A, n A,.) = W,,. Multiparameter Wiener procee are not new have been tudied by a number of author [2]-[4]. They provide a natural framework for dealing with white Gauian noie in a precie way. For example, the two hypothee in (1.1) can now be retated for two-parameter rom field a follow. Let T = [O,T,] x [O,T,] let X,, St, t E T, be a pair of rom field. We wih to tet between the pair of hypothee Martingale: 4tW(dt) + b $,W(dt). (3.4) R+Z R+2 IfX, = $,W(d), then {X,, t E R+2} i a martingale. It i ample continuou if a eparable verion i choen. (3.5) If X, = ( 4,WW, J.4 then Y, = X, - 42 d i a martingale. (3.6) J We note that (3.5) implie that EX, = 0 (3.6) implie that Ho : X, i a.wiener proce EX, = qbs2 d H,: N, = X, - S, dz i a Wiener proce. (2.7) which together with linearity imply that Throughout thi paper A, will alway denote the rectangle CI x P,t,l.

3 420 IEEE TRANSACTIONS ON INFORMATION THEORY, JULY 1974 So far there are no urprie. For a one-dimenional Wiener proce W,, t E [O,co), W, - t i a martingale which can be expreed (by Ito differentiation rule) a Wt2 - t = 2 J, W, dw,. For t E R+2, although Wt2 - t,t2 i a martingale, it cannot be expreed a a tochatic integral JA, 4, W (d) for any 4. In thi ene the integral defined by (3.2) (3.3) are incomplete. We need tochatic integral of a econd type which we hall denote by [6] [,, 2 R+2 1 It wa hown in [6] that any quare-integrable functional of a two-parameter Wiener proce can be repreented in term of tochatic integral of the firt econd type. Thi wa proved by uing an important differentiation rule which will be tated a follow. Let {X,, t E R+2} be a martingale defined by x, = c $,W(d) (3.13) J $(t,t )W(dt)W(dt ) define V, = 1 c#$ d. (3.14) where $ atifie the two following condition imilar to J (3.1). Suppoe that f(x,t) i a function atifying a) For any t E R+2, {$(, ), W();, E A,} i independent of {W(A), A c A,}. +f (x,t)vv, + Vf(x,t) = 0 (3.15) b) E$2(t,t ) R+2 R+2 dt dt < co. Briefly, the definition i given a follow. Again, firt conider a finite rectangle ubdivide it by a equence of rectangular partition {A,,,}. Denote the lower left corner of A,,, by t,,,. We et where V denote gradient with repect to t f = (13 f/ax ). Then,f(X,,t) i a martingale can be expreed a = f V,,M,w(d) +; VI f VS.S~~ v )$,&W(d)W(d ) $(t,t )W(dt)W(dt ) x AxA (3.16) = lim in 9.m. c c ~(t,,,,t,,)w(a,,~)w(a,,,) (3.8) n+m VP where the double um i taken over only thoe pair (v,~) for which t,,, t,,, are unordered. Extending the integral from a finite rectangle A to R+2 can be done in the uual manner. Stochatic integral of the econd kind have a number of intereting propertie, ome of which are given in the following. More detail can be found in [6]. Linearity: Martingale : wheref = (aflax)j = (a2flax2) v = (max (l, l ), max (2, ~~ 1). Function atifying (3.15) may be called harmonic function. One important example of a harmonic function i x2 - V,, for which (3.16) yield x 2 _ I/ = 2 f X4,JVd) + [JJI x h#,,w(d)w(d ). Another important example i exp (x - +I,>. If we et (3.17) AxA [Mu ) + ~~(t,t )lw(dt)w(w A, = exp (X, - +V,) (3.18) then =a $(t,t )W(dt)W(dt ) = 1+ M Wd) +b 4(t,t )W(dt)W(dt ). (3.9) c#&asvs,w(d)w(d ). (3.19) $(,.f)w(d)w(d ) i a martingale. (3.10) It i clear now that (3.19) i the bai for a generalization to (1.4). IV. A LIKELIHOOD RATIO FORMULA The integr t,b(t,t ) can be et equal to zero at all Let T be a rectangle, ay [O,a,] x [O,a,]. Let {X,, S,, t E T} be a pair of procee with Xrepreenting the oberved ordered pair (t,t ) without changing the integral. (3.11) proce S the ignal. Conider the two hypothee The integral i ymmetric, o that t,b(t,t ) can be replaced by -&[rl/(t,t ) + +(t,t)] without changing the integral. (3.12) H,,: X, i a Wiener proce H, : W, = X, - S, dz i a Wiener proce. (4.1)

4 WONG : LIKELIHOOD RATIO FORMULA 421 Let 8, 8, denote the probability meaure under thee proce. That i, p = BOX. It alo mean that two hypothee, POX,.?Plx denote their retriction to the c-field of event generated by X. We are intereted in exp - S,W(dz) - ;, S,2 dt)] -I condition which enure that B, i abolutely continuou [ (f T with repect to POX in finding the likelihood ratio = exp S,X(dz) - ; S2 dz (1 T T 1 L, = E, (+=I X,, rdt). (4.2) i finite with 3 probability 1 J? mut be equivalent to 9,. We hall aume condition which, though not unreaon- Therefore, L, can be calculated by able, are much tronger than neceary in order to implify the derivation of the main reult. L, = E($lX,,tA,) Theorem 1: Suppoe that under hypothei H, S W are independent procee JT St2 dt < cc with probability 1. Then 91X i mutually abolutely continuou with repect to POX L, atifie the equation L, = 1 + W,(S, I X,, E AM(d4 L,.,~E,(S,S,~ I X,, E A,,,,)X (WX (W where z v z = (max (zl,zl ), max (z2,z2 )). Proof: integral under the condition (4.3) Firt, we need to generalize the tochatic r J d,w(dt) R+2 c$~ dt < co with probability 1 R+Z intead of the condition where!el dg Now, define - exp S,X(dz) - f 1 SC2 dr). (4.4) (S T T A, = exp S,X(dz) - ; ST2 dz, t E T. (4.5) (1 f 1 Then we know from (3.19) that with g probability 1 = 1 + A,W + 1 z Cd4 Ed, dt < co. =t+ E (A,S, I X,, E AJX(d4 R+2 Thi i eaily done by defining JR + 24, W (dt) a the limit in probability of a equence jr+2 4,,, W(dt) where {4,} i a equence of truncation of 4 uch that for each n, A,V,S,S,.X (dz)x (dz ). (4.6) Becaue T = [O,a,] x [O,a,], we have (dg,/d$) = A,, EA, = 1 implie that A, i a martingale, o that L, = &A, ) X,, E A,). (4.7) Equation (4.3) i obtained from (4.6) a follow. Firt L, = E(A, 1 X,, E A,) +;,?(A,,,S,S,. 1 X,, E.4,)X (dt)x (dz ). x Ar (4.8) &t2 dt < n. Becaue under 9 X i a Wiener proce independent of S R+2 E(A,S, I X,, E A,) = E(A,S, I X,, E 4, z E A, Next, conider a tranformation of the probability meaure 8, by the formula S,W(dT) - ; Becaue W S are independent under 8,, given {S,, z E T} the tochatic integral ST S,W (dz) i a Gauian variable with zero mean variance jt ST2 dz. Hence E (AT r* S,S,, I X,, E A,) = E (A,,,&S, I X,, E 4,,,), 5, z E A,. Finally, the well-known rule of tranforming a conditional expectation under a change in probability [7, pp yield t,exp(-jts,w(dl)) =exp(i,s:dr) &A$, I X,, E A,) = L,E,(S, I X,, E A,) E (A,,,, S,S,. I X,, E A,,,,) = L,.,.E,(S,S,, I X,, E A,.,,). $ mut be a probability meaure. It i eay to how that (S,X) are ditributed exactly under 3 a (S, W) under 8, Inerting thee reult in (4.8) complete the derivation of (cf. [7, pp ). Therefore, under 8 X i a Wiener (4.3) the proof of Theorem 1.

5 422 IEEE TRANSACTIONS ON INFORMATION THEORY, JULY 1974 t2tdt2 12 Fig. 2. Parameter value for likelihood ratio formula. Remark: It i clear that the ame proof uffice a long a it i poible to find a $ probability meaure on the ample pace of (X,S) uch that X i a Wiener proce under 3, {X(A), A c A,} i independent of {X,, S,, r E A,} for each t, Let db,- - exp S,X(dz) - ; d3 dlt = L(tl+dt@2+dtz) - L(ti+dtl,fz) - L(f&+dtz) + L(t,,tz)* Then, roughly peaking, we have from (4.3) the relationhip dl, = L,E,(S, 1 X,, E A,)X(dt) * X(dT,,dt,)X(dt,,dz, ). (4.9) Fig. 2 illutrate the variou parameter in (4.9). Equation (4.9) jutifie the aertion that we made in the introduction, relate L, to conditional moment up to the econd order of S on the boundary of A, given X within A,. Suppoe that under H, the ignal S i Gauian. Then, ince S W are independent, S X are jointly Gauian. Let,,, be defined by $,t = J%(& I x,, 8 E 4 (4.10) R(T, 7 ; t) = E,[(S, - $J(S,, -,,,,) I X,, E A,]. (4.11) Then R(z,z ;t) mut be a determinitic function. We can now rewrite (4.3) a L, = 1+ L&,X tdz) L,, T,R(z, 6; z v z )X (dz)x (dz ) (4.12) (4.9) can be rewritten a dl, = L,{$,,,X(dt) + I R[tr,,l,),(t,,z,~,; tlxtdz,,dt,)x(dt,,dz, )}. 0 0 (4.13) Since R i determinitic, (4.13) how that dl, i completely pecified by L, S,,, for z on the boundary of A,. For the Gauian cae an expreion for (d~, /d~, ) can be obtained in term of multiple Wiener integral certain kernel which can be obtained from the covariance function of S [4]. Thu far we have not been able to demontrate it equivalence to (4.12). Finally, we note that (4.3) dahe any hope that L, can be expreed in the form L, = exp (S S,X(dz) - i/ $,2 dr) becaue thi would require that i?$, 3, = E,tS, I X,, E 4 = E,(S,.S, 1 X,, E A,,,,). The only example that we have been able to find which atify thee condition are thoe in which 1) S i determinitic, which i a trivial cae; 2) S i a functional of X, which i excluded by the aumption of Theorem 1. REFERENCES [l] T. Kailath, A further note on a general likelihood formula for rom ignal in Gauian noie, IEEE Tran. Inform. Theory, vol. IT-16, pp , July [2] K. Ito, Multiple Wiener integral, J. Math. Sot. Japan, vol. 3, pp , [3] J. Yeh, Wtener meaure in a pace of function of two variable, Amer. Math. Sot. Tran., vol. 95, pp , [4] W. J. Park, A multiparameter Gauian proce, Ann. Math. Statit., vol. 41, pp , [5] R. Catroli, Sur une equation differentielle tochatique, Compte pg; z Acad. SC. Pari, vol. 274, er. A, pp , June 12, [6] E. Wang M. Zakai, Martingale tochatic integral for procee with a multidimenional parameter, to be publihed in Zeit. Wahrcheinlichkeittheorie. [7] E. Wong, Stochatic Procee in Information Dynamical Sytem. New York: McGraw-Hill, 1971.

Factor Analysis with Poisson Output

Factor Analysis with Poisson Output Factor Analyi with Poion Output Gopal Santhanam Byron Yu Krihna V. Shenoy, Department of Electrical Engineering, Neurocience Program Stanford Univerity Stanford, CA 94305, USA {gopal,byronyu,henoy}@tanford.edu