Standard Bayesian Approach to Quantized Measurements and Imprecise Likelihoods

Size: px

Start display at page:

Download "Standard Bayesian Approach to Quantized Measurements and Imprecise Likelihoods"

Bathsheba Cameron
6 years ago
Views:

1 Stadard Bayesia Approach to Quatized Measuremets ad Imprecise Likelihoods Lawrece D. Stoe Metro Ic. Resto, VA USA Abstract I this paper we show that the stadard defiitio of likelihood fuctio used i Bayesia iferece simply ad correctly hadles imprecise likelihood fuctios ad quatized measuremets. Some recet papers have stated or implied that methods ivolvig radom sets, fuzzy membership fuctios, geeralized likelihood fuctios, or Dempster-Shafer cocepts are required to hadle imprecise likelihood fuctios ad quatized measuremets. While it is true that oe ca use these methods, employig them adds uecessary complicatio ad possibly cofusio to the solutio of a simple problem. I the spirit of Occam s razor, we feel the simplest correct solutio is the best. Keywords Likelihood; Bayes; Radom Sets; Fuzzy Logic; Dempster-Shafer; Imprecise; Quatized Measuremets I. INTRODUC TION Refereces [] ad [] cosider the problem of costructig a likelihood fuctio for quatized measuremets ad propose that these types of measuremets require a geeralizatio of the stadard otio of likelihood fuctio that ivolves the use of radom sets, cocepts from fuzzy logic ad Dempster-Shafer theory, as well as geeralized or imprecise likelihood fuctios. The purpose of this paper is to show that the stadard cocept of likelihood fuctio as defied i [3] or [4] is sufficiet to solve the geeral problem i a easy ad straightforward maer. This is a importat poit because i the spirit of Occam s razor, the simplest correct solutio to a problem is the best oe. Simplicity allows readers to clearly uderstad the ature of the problem ad its solutio. It facilitates the use of the cocept i applicatios ad makes it easier to exted the cocept to more challegig problems. It eables progress. Of course, i performig research it is importat to explore ew ad differet methods ad cocepts. Oe of the virtues of the Dempster-Shafer (DS) model of ucertaity is that it expads the ucertaity calculus to situatios i which it is ot possible or reasoable to assig probabilities to the poits of a fiite state space S. Istead, oe assigs probability (belief mass) to subsets. Assigig a belief mass to a subset meas that the (probability) mass is allowed to float amog the members of the subset. As a example, the otio of havig o iformatio about a parameter other tha it is cotaied i S ca be modeled by assigig (probability) mass to the set S itself ad to all other subsets of S. A drawback to DS is that oe must assig a (probability) belief mass to every subset of S. The size of the (probability) belief space is equal to S where S is the umber of poits i S. This becomes astroomically large as the umber of poits i S icreases. 3 For example,.3. For eve modest state spaces, the computatioal problems associated with DS become quite dautig. A special case of the DS model occurs whe all the (probability) mass is assiged to sigleto sets. This is called a Bayesia belief fuctio [8] ad is equivalet to the stadard Bayesia probability fuctio. Whe this special case hos, oe is better off followig the stadard Bayesia calculus with its more reasoable computatioal load ad larger set of tools (such as expectatio) tha are available i DS. It is ot a service to the research ad applicatios commuities to leave the impressio that oe of these more complex ad sometimes difficult methods are required to solve a problem that ca be hadled by simpler, stadard methods. What both commuities desire is a uderstadig of whe these alterate methods are required or provide a extra beefit. This leads to the followig ope questio. What situatios ivolvig quatized measuremets or imprecise likelihood fuctios require the use of alterate, o-bayesia models of ucertaity? II. BAYESIAN INFERENCE FORMULATION Before begiig the discussio, we give the formulatio of the basic Bayesia iferece problem that is preseted i [3] ad is cosistet with that i [4]. There is a ukow parameter that we wish to estimate. There is a prior distributio p o such that p () θ Pr{ Θ θ } ()

2 where Pr idicates either probability or probability desity as appropriate. We obtai a measuremets Z from a sesor. The measuremet is viewed as a radom variable whose distributio depeds o. We defie the likelihood fuctio lz ( θ) Pr { Z z θ} Θ. () If we receive a measuremet Z z, we compute the posterior distributio p ( θ Z z) l( z θ) l( z θ ) p ( θ ) dθ where itegratio is replaced by summatio if the distributio o is discrete. III. QUANTIZED MEASUREMENTS Refereces [] ad [] have illustrated the ecessity of their approaches through the example of performig iferece usig quatized measuremets. We use this same example to show that stadard likelihood fuctios ad the Bayesia iferece process as give i () (3) provide a straight-forward ad correct way of icorporatig quatized measuremets ito Bayesia iferece. No geeralizatio is required, ad o extesios of the stadard Bayesia probability cocepts are eeded. We start with the digital voltmeter example give i []. Measuremets are take by a digital voltmeter that provides voltage readigs to two decimal places. From the digital voltmeter measuremet we wish to estimate the actual voltage. Let p be the prior o. We cosider three cases, measuremets without oise, measuremets with oise, ad measuremets where the quatizatio is ukow. We also cosider a fourth case where the bis defiig the quatizatio of the measuremets are ot itervals. This case is motivated by acoustic detectios i deep ocea areas. A. Quatized Measuremets without oise kow quatizatio If there is o oise added to the actual voltage, the ay voltage i the iterval (99.975, ] will produce a measuremet of The measuremet space is a discrete set of poits o the real lie of the form where is a iteger such that. Ay voltage i the set will produce a measuremet Z.. From the defiitio of likelihood fuctio i (), we have (3) (4) l(. θ) Pr{ Z. θ} if θ S (5) otherwise. For otatioal coveiece, we shall use Z for the measuremet ad l( θ ) for the likelihood fuctio i (5). The posterior o the actual voltage is computed by p ( θ Z ) l( θ) l( θ ) p ( θ ) dθ if θ S ( θ ) dθ S otherwise. B. Quatized Measuremets with oise kow quatizatio I this example we suppose the received voltage r at the digital voltmeter is the true voltage plus oise ε. Specifically, the true voltage is θ+ ε where ε has the desity fuctio f( y) Pr { ε y} for < ε<. The digital voltmeter produces measuremets to two decimal places as above. I this case the likelihood fuctio becomes l( θ) Pr { Z θ} Pr{. < θ+ ε.+ } Pr{. θ < ε. θ+ }. θ+ f ( y) dy.. θ ad the posterior distributio o Θ give the measuremet Z is p ( θ Z ) l( θ) p ( θ ) l( θ ) dθ. θ+ f ( y) dy. θ+. θ + ( θ ) f ( y) dy dθ. θ Example : As a example, let us cosider the situatio where ( y σ ) f( y) η,, (6) (7)

3 where η(,, σ ) is the probability desity fuctio for the ormal distributio with mea ad variace σ. The otatio η(,, σ ) is used to idicate the fuctio of oe variable obtaied by fixig the values of the d ad 3 rd variables at ad σ. We use a similar otatio for a fuctio of two variables whe we wish to fix the value of oe of the variables. Let z Φ ( z, σ ) η( y,, σ ) dy for z < <. (8) The the likelihood fuctio l( θ ) i (7) becomes ( θ) Φ (. θ+, σ ) Φ ( θ σ ) l.,. Figure shows plots of the likelihood fuctio i (9) for Z ad σ.,.4,ad.6. (9) where θ plays the role of the variable z i []. C. Quatized Measuremets whe Quatizatio is Ukow I the case where the quatizatio is ukow, we expad the state space o which we perform iferece to simultaeously estimate the voltage ad the quatizatio. To illustrate how this is doe withi the covetioal Bayesia iferece formalism, we cosider the case where there is o oise added to the voltage ad the quatized bis have a kow ad equal size. However, we do ot kow the achor poit for the bis. Followig Example, we take the bi size to be.. However we do t kow the achor poit of the bis. Specifically there is a ukow parameter such that ad S ( Δ ) (. +Δ,.+ +Δ ]. The iferece problem is to estimate both ad. I classic Bayesia fashio, we impose a prior distributio o which represets our prior kowledge (or ucertaity) about. As a example we suppose that the priors o the two parameters are idepedet ad the oit desity o is give by g (,) θδ p () θq () δ for < θ< ; < δ. The likelihood fuctio for the observatio Z. is l( ( θδ, )) Pr{ Z. ( Θ, Δ ) ( θδ, )} if θ S ( δ) otherwise. Let g be the posterior oit desity o Z. The give Fig.. Likelihood Fuctios for (. volt readig o voltmeter) whe σ.,.4, ad.6 The figure above looks very much like Figure i []. I fact oe ca obtai that figure if he cosiders the case where the quatizatio has bi size 5 ad the measuremet Z idicates the iterval (5,5( + )]. I this case the sets S i (4) become S (5,5( + )], ad the likelihood fuctio i (9) becomes g ((,) θδ Z ) S ( δ ) l( ( θδ, )) p q ( δ) ( ) l ( θ, δ ) p ( θ ) q ( δ ) dθ dδ q( δ) ( θ ) q( δ ) dθ dδ otherwise. for θ S ( δ) ( ) ( ) l( θ) Φ ( 5+ ) θ, σ Φ 5 θ, σ

4 D. Quatized Measuremets with Noise Bis that are ot Itervals The bis, e.g., the sets S i (4), do ot have to be itervals. They ca be the uios of disoit itervals or eve more geeral sets. The approach give above will still work. Example : Let us cosider a case from uderwater acoustics. Whe a passive acoustic sesor is located i a deep water area of the ocea, the soud propagatio coditios ofte produce detectio areas that are disoit. For example, there may be good detectio coditios from the sesor s locatio out to rage 5 NM. This is typically called the direct path regio. I additio there are ofte covergece zoe regios at rages of roughly 3 NM, 6 NM, ad eve farther out. A covergece zoe is a regio where the acoustic rays coverge ad produce low propagatio loss ad icreased detectio probability for the sesor. Suppose the covergece zoes are 5 NM wide. It is ofte the case that the ucertaity the about source level (loudess of soud emitted) of a potetial target, meas that although oe caot calculate the detectio probability as a fuctio of rage, oe does kow that if a target is detected it is oe of these regios. I this case, a detectio meas that the target is i oe of the above rage itervals, i.e., its rage is i the set mi Φ( 5 r, σ ), Φ( r 5, σ ) for r mi { Φ(7.5 r, σ ), Φ( r 3.5, σ )} ( r) for < r 45 mi { Φ(57.5 r, σ ), Φ( r 6.5, σ )} for 45 < r <. () Figure shows the likelihood fuctio ( ) whe σ.5. S {[, 5] [7.5,3.5] [57.7, 6.5] } () Geerally oe does ot kow the edges of the itervals i S exactly. Depedig o the source level of the target ad the level of ambiet oise i the ocea, these areas ca be a bit larger or smaller tha the omial umbers i (). We will model this ucertaity with a likelihood fuctio that is similar to the oe give i Example with the exceptio that there is oly oe bi Z correspodig to a detectio. Specifically we let r deote the rage of the target ad ε be ormally distributed with mea ad variace σ. The the likelihood fuctio l d for a detectio becomes ( r) Pr { Z target at rage r} Pr{ r 5 + ε} for r Pr{ 7.5 ε r ε} for < r 45 Pr{ 57.5 ε r ε} for 45 < r< () I terms of Φ defied i (8), the likelihood fuctio i () becomes Fig.. Likelihood fuctio ( ) for accoustic detectio i a covergece zoe eviromet IV. FUSION I this sectio we show how oe ca fuse the iformatio from quatized sesors. Example 3. We cosider two sesors of the type i Example. For simplicity we will cosider a oe dimesioal state space. I most applicatios the positio space wou be two or three dimesioal, but this oe dimesioal example will illustrate the methods ivolved. We take the target state space to be X [ NM, 7 NM] ad the prior to be uiform over X. Sesor is located at x ad sesor is located at x 33 The likelihood fuctio l d ( ) for a detectio from sesor is

5 mi Φ( 5 x, σ ), Φ( x 5, σ ) for x mi Φ(7.5 x, σ ), Φ( x 3.5, σ ) ( x) for < x 45 mi Φ(57.5 x, σ ), Φ( x 6.5, σ ) for 45 < r < 7. regio ear x has bee zeroed out. The first covergece zoe regio cetered a little to right of x 3 is much arrower tha the similar regios i Figures ad 3, ad the secod covergece zoe regio cetered ear x 6 is arrower ad has somewhat lower likelihood tha i either Figure or 3. The posterior o x is proportioal to the likelihood fuctio show i Figure 4. A plot of this likelihood fuctio wou look the same as Figure with Rage replaced by x o the horizotal axis. The likelihood fuctio for a detectio from sesor is mi Φ(8 r, σ ), Φ( r 38, σ ) for r 45 ( X) mi { Φ(6.5 r, σ ), Φ( r 65.5, σ ) } for 45 < r 7. Figure 3 shows the likelihood fuctio l d ( ). Notice this likelihood is the same as the oe i Figure shifted to the right by 33 NM with the followig differeces. The secod covergece zoe is outside the state space X [ NM,7 NM] ad the full direct path regio is ow i the state space ad is cetered aroud the sesor locatio at x 33. Fig. 4. Joit likelihood for a detectio from sesors ad. V. TRACKING MOVING TARGETS WITH QUANTIZED MEASUREMENTS The target cosidered i sectio IV is statioary. Oe ca also track movig targets usig sesors with quatized measuremets. Sice the measuremets do ot satisfy the liear Gaussia assumptios required for a Kalma filter, it will usually be best to perform the trackig usig a particle filter as described i [5] or chapter 3 of [6]. The geeral procedure is straight-forward. The particles are motio updated to the time of a measuremet. The likelihood fuctio for the quatized measuremet is applied to the weight of each particle to produce the posterior distributio o target state. The particles are resampled as ecessary ad the motio updated to the time of the ext measuremet. I particular, suppose that the target state distributio at time t is represeted by the set of particles {( ( ), ( )} x t w t for,, N Fig. 3. Likelihood fuctio l ( ) for detectio from sesor located at x 33 d The oit likelihood fuctio for detectios from both sesor ad sesor is the product of the two likelihood fuctios ad is show Figure 4. Notice that direct path where x is the state of the th particle ad w is its probability. This set of particles represets a discrete probability approximatio to the distributio o target state at the time t. Suppose we obtai a quatized measuremet Z z such as the oes i Example 3. Let lz ( x ) be the

6 likelihood fuctio for this measuremet. The posterior distributio o target state at time t is give by where {( x ( t), w ( t) } for,, N (3) l( z x( t) ) w( t) w () t. N l( z x ( t) ) w ( t) If the ext measuremet is received at time t > t, the posterior particle filter represetatio i (3) ca be motioupdated to the time t to act as a proposal distributio for the icorporatio of the measuremet at time t. Ofte the posterior i (3) is resampled before the motio update is performed. I additio other proposal distributios ca used to improve the particle filter performace as discussed i [5]. Oe cou employ a particle filter to track a submarie usig the sesors described i Example 3. No extesio beyod stadard Bayesia likelihood fuctios ad iferece is required. Referece [7] ivestigates the problem of trackig a target with quatized measuremets. The authors assume a Gaussia motio model for the target ad develop a approximate Miimum Mea Squared Error (MMSE) solutio. They provide a umerical algorithm for obtaiig this solutio. This is very impressive work, ad oe must admire the authors for the cleveress of their solutio. However, the solutio is complex ad does require may special assumptios. By cotrast the particle filter approach is very geeral ad simple. Oe is ot costraied to Gaussia motio models or measuremet errors. It is straight-forward to icorporate a wide variety of types of measuremets. A possible drawback to usig a particle filter is that it that it may be too computer itesive to be practical i some applicatios. However, the icreasig capability of computers makes this less ad less likely to be a problem. VI. CONCLUSIONS I the examples give above we have show how to costruct likelihood fuctios for quatized measuremets usig the stadard Bayesia approach with stadard likelihood fuctios. We have show how to fuse iformatio from oe or multiple quatized measuremets to compute a posterior distributio. We have also oted that quatized measuremets ca be applied to movig target problems usig particle filters ad likelihood fuctios for the quatized measuremets i a straight-forward, Bayesia fashio. The power of a likelihood fuctio is that it coverts measuremets from (almost) ay measuremet space ito a fuctio o the target state space. This allows us to icorporate the iformatio i these measuremets ito the posterior distributio o the target state space. The examples give above illustrate this process with quatized measuremets, but the method is applicable to wide rage of types of measuremets ad sesors. I particular, it is applicable to ay measuremet for which oe ca compute a likelihood fuctio usig the defiitio i (). This is why likelihood fuctios are the commo currecy of iformatio i Bayesia iferece. The examples give above demostrate this fact. We have discussed above the virtues of usig the simplest solutio to a problem. I additio, there ca be drawbacks to uecessary complexity. For example, if we employ Dempster Shafer methods to hadle quatized measuremets, the we will be limited i applicatios to fiite discrete state spaces sice there has bee o satisfactory extesio of Dempster- Shafer theory to cotiuous state spaces. Eve if the state space is fiite, the computatios ivolved with Dempster- Shafer methods grow expoetially with the size of the state space which limits its applicability to real problems. REFERENCES [] B Ristic, Bayesia estimatio with imprecise likelihoods: Radom set approach, IEEE Sigal Processig Letters, vol. 8, pp , July. [] R Mahler, Geeral Bayes filterig of quatized measuremets pp i Proceedigs of 4 th Iteratioal Coferece o Iformatio Fusio, Chicago, USA, July 5-8,. [3] L. Stoe, A. Barlow, ad T. Corwi, Bayesia Multiple Target Trackig, Bosto, MA: Artech House, 999 [4] J. Berger, Statistical Decsio Theory ad Bayesia Aalysis, d ed, New York, NY: Spriger-Verlag, 985. [5] B. Ristic, S. Arulampalam, ad N Gordo, Beyod the Kalma Filter, Bosto, MA. Artech House, 4. [6] L. Stoe, et al, Bayesia Multiple Target Trackig d Editio, Bosto, MA. Artech House, to be published [7] Z. Dau, V. Jilkov, ad X, Lie, State Estimatio with Quatized Measuremets: Approximate MMSE Approach pp i Proceedigs of 3 th Iteratioal Coferece o Iformatio Fusio, Ediburgh Scotlad, July 6-9,. [8] G. Shafer, A Mathematical Theory of Evidece, Priceto Uiversity Press 976.

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a