A Fast and Fault-Tolerant Convex Combination Fusion Algorithm under Unknown Cross-Correlation

Size: px
Start display at page:

Download "A Fast and Fault-Tolerant Convex Combination Fusion Algorithm under Unknown Cross-Correlation"

Transcription

1 1th Internatonal Conference on Informaton Fuson Seattle, WA, USA, July 6-9, 9 A Fast and Fault-Tolerant Convex Combnaton Fuson Algorthm under Unknown Cross-Correlaton Ymn Wang School of Electroncs and Informaton Engneerng X an Jaotong Unversty X an, Shaanx 719, P.R.Chna erc.wang.xjtu@gmal.com Abstract Knowledge of the cross-correlaton of errors of local estmates s needed for many dstrbuted fuson algorthms. However, n a fully dstrbuted system or decentralzed network, the calculaton of crosscorrelaton between local estmates s qute nvolved and may be mpractcal. The Covarance Intersecton CI algorthm has been proposed under unknown correlaton. But the CI algorthm has hgh computatonal complexty because t requres optmzaton of a nonlnear cost functon. Ths paper presents a fast CI algorthm, and an alternatve optmzaton crteron wth a closed form soluton. Based on ths crteron, a fast and faulttolerant convex combnaton fuson algorthm s presented by ntroducng an adaptve parameter, whch can obtan robust estmate when estmates to be fused are nconsstent wth each other, and the degree of robustness of fuson result vares wth that of nconsstency between estmates to be fused. Keywords: Dstrbuted fuson, decentralzed network, convex combnaton, covarance ntersecton, chernoff nformaton, fault-tolerant. 1 Introducton Estmaton fuson, or data fuson for estmaton, s the problem of how to best utlze useful nformaton contaned n multple sets of data for the purpose of estmatng a quantty a parameter or process at a tme 1]. There are two basc fuson archtectures, centralzed and dstrbuted, dependng on whether raw data are used drectly for fuson or not. In centralzed fuson, all sensors send ther raw data to a fuson center, whch can produce globally optmal estmates. In dstrbuted fuson, each local sensor processes ts own raw data and sends the local estmate to other sensors or a fuson center to obtan global estmates. In dstrbuted fuson, two basc fuson archtectures are well known: herarchcal and fully dstrbuted 6], and sometmes, Research supported n part by Project 863 through grant 6AA1Z16 and ARO through grant W911NF X. Rong L Department of Electrcal Engneerng Unversty of New Orleans New Orleans, LA 718, U.S.A. xl@uno.edu the fully dstrbuted archtecture s referred to as decentralzed network. Generally, dstrbuted fuson has a less communcatonal burden and hgher survvablty, and t s more flexble and relable. However, dstrbuted fuson also has challengng problems not present n the centralzed fuson. One of the major ssues s dealng wth the cross-correlaton of errors of local estmates due to the common process nose, pror, correlaton among measurement noses across sensors, etc. Most of the exstng dstrbuted fuson algorthms see, e.g., 1]-8] accountng for ths problem try to explot the correlaton n the estmates. Consder a decentralzed network of N nodes whose connecton topology s completely arbtrary.e., t mght nclude loops and cycles and can change dynamcally. Each node has nformaton only about ts local connecton topology e.g., the number of nodes wth whch t drectly communcates and the type of data sent across each communcaton lnk 11]. In ths case, the calculaton of the cross-correlaton of local estmaton errors s qute nvolved and not practcal. 9]-11] proposed the Covarance Intersecton CI algorthm under unknown correlaton, whch bypasses ths problem, and used a convex combnaton of the local estmates to mnmze the trace or determnant of the mean-square error MSE matrx. 1] showed that the CI s equvalent to the log-lnear combnaton of two Gaussan functons, and dscussed the crteron that mnmzes the determnant of the fused MSE matrx and an alternatve crteron of mnmzatng Chernoff nformaton. However, mnmzaton of the determnant or Chernoff nformaton s computatonally complex. 1] presented a fast CI algorthm w.r.t. determnant mnmzaton. In some specal cases, the estmates to be fused are nconsstent wth each other e.g., f one poston estmate s far from another by more than a klometer, but ther respectve MSE matrces showthat eachestmate s accurate to wthn a meter, then there must be somethng wrong. In 1], covarance unon CU was ISIF 71

2 proposed to handle ths problem; the Mahalanobs dstance was suggested for detectng statstcal devatons between estmates; and a user-defned threshold was proposed to detect the nconsstency. A fault-tolerant mechansm combnaton of the CI and CU CI/CU was also presented. However, how to determne the thresholdsanopenproblem. In ths paper, a fast CI algorthm based on a proposed optmzaton crteron s presented, whch has a closed form soluton. Because tdoesnotrequreoptmzaton of a nonlnear cost functon, a large amount of computatonal tme can be saved. Snce the proposed crteron s to fnd a halfway pont between the dstrbutons of estmates, t wll be more robust when the estmates to be fused are nconsstent. Combnng ths crteron wth a proposed adaptve parameter, a fast and fault-tolerant convex combnaton fuson algorthm s presented. It can obtan robust fuson results when estmates to be fused are nconsstent, and the degree of robustness can vary adaptvely wth that of nconsstency between estmates. It s easy to be mplemented wthout any pre-defned threshold. Most mportantly, t does not need to deal wth any optmzaton problem, and t offers decentralzed fault tolerance meanwhle. The structure of ths paper s as follows. Secton presents a fast CI algorthm based on a proposed mnmzaton crteron. In secton 3 a fast and robust convex combnaton fuson algorthm under unknown correlaton s proposed. Illustratve smulaton examples are gven n Secton. Secton provdes conclusons. A New Fast CI Algorthm Suppose two estmates are to be fused, ˆx,=1,, wth MSE matrx P. The CI algorthm s gven by ˆx CI = P CI ωp 1 ˆx 1 +1 ωp ] ˆx 1 P CI = ωp1 +1 ωp ] If the parameter ω 1 s chosen to mnmze detp CI, t s called the general CI algorthm n ths paper. Then, under the determnant mnmzaton crteron, the fused estmate s ˆx,P. For the Gaussan case, the probablty densty functon pdf of the th local estmate s gven by: 1 p x= e 1/ πp x ˆxT P x ˆx 3 Then, the CI algorthm can be generalzed to a technque that selects a fused pdf that s a log-lnear combnaton of two orgnal pdfs 1]: p ω x= pω 1 x p1 ω x p ω 1 x p 1 ω x dx and mnmzaton of det P CI s equvalent to mnmzatng of the Shannon nformaton or contnuous entropy of the fused Gaussan functon: H p ω = p ω xlogp ω xdx = 1 log πn P ω ]+ k where k s dmenson, and P ω = P CI. An alternatve crteron s Chernoff nformaton mnmzaton: C p 1,p = mn ω 1 It s equvalent to log p ω 1 xp1 ω x dx 6 D D p ω p 1 =D p ω p 7 where D p A p B s the Kullback-Lebler K-L dstance from p A top B, and ω mnmzes Equaton 6. Then, t s natural to consder p ω x as the halfway pont snce the K-L dstance from p ω xto p 1 x s equal to that from p ω x top x. However, computaton of ths ω s complcated. A smple way to defne a halfway pont s to let ω =1/, whch s known as the Bhattacharyya dstance: B p 1,p = log p 1/ 1 xp 1/ x dx 8 Inspred by Equaton 7, an alternatve to defne the halfway pont p ω s: D D p 1 p ω =D p p ω 9 In contrast to the Chernoff nformaton crteron, ths problem has a closed form soluton: ω = D p 1 p D p 1 p +D p p 1 1 Ths determnes the parameter ω n the CI algorthm. Thus the new fast CI algorthm FCI s gven by: ˆx FCI = P FCI ω P1 ˆx 1 +1 ω P ] ˆx 11 P FCI = ω P1 +1 ω P ] 1 For the Gaussan dstrbutons, for,j =1,, D p p j = 1 ln P j P k +ˆx ˆx j T P j ˆx ˆx j +tr P P ] j 13 Note that the Chernoff nformaton D, whch s the halfway dstance between p 1 x andp x evaluated usng p ω x, s the best achevable exponent for the Bayesan probablty of error 13]. However, our proposed D, whch s the halfway dstance evaluated usng 7

3 8 6 6 FCI Fgure 1: Equal means FCI Fgure : Unequal means p 1 x andp x respectvely, does not have ths property n general. Ths appears make D more appealng, but our D s computatonally superor. For Gaussan dstrbutons, f pror dstrbutons have the same MSE matrces, that s, P 1 = P, the proposed equal dstance D and Chernoff nformaton D are dentcal, and ω = ω =1/. Fgures 1 and compare the fused estmates of each par of estmates calculated by and FCI, respectvely. Dotted ellpses are for the estmates to be fused. In Fgure 1, the means are equal and the fuson results are ˆx 1 =ˆx =ˆx =ˆx FCI = ] ] P 1 = P = P = ] P FCI = ] and for the estmate ˆx 1,P 1 concde, and the fuson results are: ] 1 ˆx 1 =ˆx = P 1 = P = ] 1 ˆx = P 1. = ] ˆx FCI = P.67 FCI = 33.3 The proposed crteron has several advantages: It has a closed form soluton, whch does not requre optmzaton of a nonlnear cost functon. It can save a large amount of computaton. As shown n Fgure, for the determnant mnmzaton crteron.e.,, f P <P j, j, the fused estmate wll be ˆx,P, although the two estmates are far from each other. Snce the determnant mnmzaton crteron, whch focuses on the uncertanty of the estmate, does not utlze the values of ˆx,theoptmalω s ndependent of the values of ˆx. As P ncreases from below P j to above P j, the fused estmate wll jump from ˆx,P to ˆx j,p j, whch s hghly undesrable. The proposed crterons to fnd a halfway pont between the dstrbutons of estmates, and both the uncertanty of the estmates and the dstance between estmates are taken nto account. Thus, when the estmates are nconsstent, the proposed crteron wll be more robust. Consderng the scalar case, n general, for fuson of unbased estmates, wll be a better choce snce t pcks the estmates wth a smaller varance, but for fuson of based estmates e.g., one s based, and the other s unbased but wth a larger varance, our proposed crteron would be better. For hgher dmensons, t s complcated and depends on specfc system desgn requrements and the bas of the estmates. CI can be generalzed to an arbtrary number of N updates usng the followng equatons 11]: ˆx CI = P CI N P CI = N =1 =1 ω P ˆx 1 ω P 1 N ω = 1 16 =1 By our proposed crteron, a generalzed constrant s gven by In Fgure, the means are not equal, P 1 <P. It can be shown that the ellpses for the fused estmate 73 D p j p ω Dp p j ω j =,,j =1,,...,N 17

4 Followng the generalzaton method of 1], the largest lnearly ndependent subset can be represented by D p +1 p ω Dp p +1 ω +1 =, =1,,...,N 1 18 Wth α = D p +1 p andβ = D p p +1, combnng the constrants 16 and 18 yelds the lnear system: α 1 β 1 ω 1 α β ω α N β N ω =. N ω N 1 19 Then, for the FCI algorthm, ω can be obtaned by where P 1 and P are the actual MSE matrces of ˆx 1 and ˆx, respectvely. For,j =1,, P j = E x ] x T j. The key to ths method s that δ s consdered as an overall belef of all estmates to be fused. If ˆx 1 =ˆx and P 1 = P, the two estmates are dentcal, obvously, δ =1. When the estmates dffer, especally f the dstance between estmates s large, the estmates are regarded as nconsstent wth each other n general. Gven that we have no dea whch one s more credble, a natural proposal s to scale-down the overall belef : δ<1. In nformaton theory, entropy s a measure of the uncertanty of a random varable. The relatve entropy or K-L dstance s a measure of the dstance between two dstrbutons. Then, we construct the adaptve parameter δ 1as ω = j=1 α N j k= β k N =1 j=1 α j N k= β k 3 A Fast and Fault-Tolerant Convex Combnaton Fuson Algorthm In some cases, the estmates to be fused are nconsstent wth each other. In 1], covarance unon CU was proposed to handle ths problem, and the Mahalanobs dstance was suggested for detectng statstcal devatons between estmates. Frst, a user-defned threshold was proposed to detect the nconsstency. If the threshold s exceeded, the estmates are regarded as nconsstent wth each other, mplyng that at least one of the estmates s not a consstent estmate defned n 1]. Then, the CU algorthm can be taken to resolve such nconsstency, whch s referred to as deconflcton. Also, a fault-tolerant mechansm as a combnaton of CI and CU CI/CU was presented. However, how to determne the threshold s an open problem. An adaptve parameter δ s ntroduced to solve ths problem. Suppose there are two estmates to be fused, ˆx, =1,, wth MSE matrx P. Consder the followng general convex combnaton fuson problem: ˆx CC = P CC ω1 P1 ˆx 1 + ω P ˆx 1 P CC = ω 1 P1 + ω P ω 1 + ω = δ 3 The estmaton error correspondng to estmate ˆx CC s x CC = P CC ω1 P1 x 1 + ω P x where x 1 and x are the estmaton error of ˆx 1 and ˆx, respectvely. Then the actual MSE matrx P CC s P CC = P CC ω 1 P 1 +ω 1 ω P P 1 P1 + ω 1 ω P1 P 1 P1 + ω P P P P 1 P ] PCC δ = H p 1 +H p H p 1 +Hp +J p 1,p 6 where H p stheentropyofp x, H p 1 +Hp s the overall uncertanty of p 1 x andp x, and J p 1,p s symmetrzed K-L dstance between two dstrbutons known as J-dvergence 16]: J p 1,p =D p 1 p +D p p 1 7 The δ of 6 s ntutvely appealng: gven the overall entropy H p 1 +H p, the larger the J-dvergence s, the less the overall belef δ s. Usually, δ s nversely proportonal to J p 1,p, but ths s not true, f J p 1,p s large but the overall entropy s much larger, δ wll not decrease. Snce δ 1, the consstency of ˆx CC,P CC remans guaranteed, that means P CC P CC. Ths convex combnaton algorthm s equvalent to the CI algorthm wth all MSE matrces nflated by a factor of 1/δ. Snce the proposed mnmzaton crteron s more robust, as dscussed n Secton, combnng t wth the adaptve parameter δ, a fast and fault-tolerant convex combnaton FFCC fuson algorthm can be obtaned: ˆx FFCC = P FFCC ω1 P1 ˆx 1 + ω P ] ˆx 8 P FFCC = ω 1 P1 + ω P ] 9 ω 1 + ω = δ 3 ω 1 = δd p 1 p D p 1 p +D p p 1 31 Fgures 3 and compare the fused estmates of each par of estmates calculated by, FCI and CU, respectvely. In Fgure 3, the overall belef δ =.6, and 7

5 FFCC CU Fgure 3: Overall belef δ =.6 FFCC CU 1 3 Fgure : Overall belef δ =.1 the fuson results are: ˆx 1 = ˆx =.3 ˆx =.1.7 ˆx FFCC = 3.9 ˆx CU = ] 1 P 1 = ] 8 1 P = 1 ] P = ] P FFCC = ] P CU = In Fgure, the overall belef δ =.1 and the fuson results are: ˆx 1 = ˆx = ˆx = ˆx FFCC = ˆx CU = ] 1 P 1 = 1 ] 1 P = 1 ] 1 P = 1 ] 6.6 P FFCC = 6.6 ] P CU = Fgures 3 and llustrate that the MSE matrx of the FFCC estmate can adapt to the overall belef δ. When δ = 1, FFCC s just the FCI algorthm. Wth a decrease n δ,themsematrxsscaledupby1/δ tmes to resolve the nconsstency among estmates. Consderng a smple decentralzed network wth only two nodes and synchronous nformaton exchange, n contrast to our adaptve fault-tolerant mechansm, gven a pre-defned threshold the CI/CU mechansm needs to make a hard decson to determne whether the estmates are nconsstent wth each other. If the userdefned threshold s not exceeded, then CI wll be appled to obtan the fused estmate, whch wll be mantaned for the next update. When an nconsstency s detected, gven that nether node can determne whch of the estmates s spurous, both apply CU to produce a consstent estmate, and the CU estmate wll be mantaned tll the next communcaton. Obvously, however, the FFCC algorthm s easy to be mplemented wthout any pre-defned threshold. Most mportantly, the computatonal complexty of the FFCC algorthm s far less than CI/CU. FFCC can also be generalzed to an arbtrary number of N updates. Let δ j = Hp +Hp j, j 3 H p +Hp j +J p,p j Then the overall belef of the N estmates can be obtaned by N δ N = δ j /m 33,j N where m = = N!/N!. For N, the FFCC algorthm s gven by ˆx FFCC = P FFCC N P FFCC = N ω P ω P ˆx 3 3 N ω = δ N 36 7

6 where ω = j=1 α N j k= β k N =1 j=1 α j Smulaton Examples N k= β k δ N 37 To evaluate the performance of the FCI and FFCC algorthms, we consder a constant-velocty movng target n one dmenson. A target dynamc model s assumed: where F = x k+1 = Fx k + Gw k 38 1 T 1 ] T, G= / T wth samplng tme T = 1 and zero-mean whte Gaussan process nose w k wth varance q. A smple decentralzed fuson system consstng of two nodes s used for trackng ths target, but there s no fuson center n the system. The measurements of the two nodes are modeled as where z k = H x k + v k, =1, 39 H 1 = 1 ], H = 1 ] Each node measures dfferent quanttes about the target: node 1 observes poston wth varance rk 1,node measures velocty wth varance rk.thetwomeasure- ment noses are assumed to be ndependent. Assume a full communcaton rate s employed and the nformaton exchange between nodes s synchronous. Each node apples a Kalman flter to estmate the state. At tme k n the th node, when each node obtans ts own measurement, the local estmates ˇx k k, ˇP k k can be obtaned by usng a standard Kalman flter gven the prevous stored estmate ˆx k k,p k k. Then each node propagates ts own estmates to another node and fuses ts own estmate wth the recevedone to obtan a fnal estmates ˆx k k,p k k accordng to a specfc fuson mechansm. If both nodes apply the same fuson method, then the fnal estmates at tme k are dentcal..1 FCI and It s assumed that both nodes apply the same fuson mechansm: FCI or. Wth the model descrbed above, the total tme span s 1 seconds, gven q =, E v k] = r 1 k =, r k = and the flter s ntalzed at the true state, whch s generated Gaussan dstrbuted wth mean and covarance: ] ] 1 1 x =, P = ] RMS Poston Error RMS Velocty Error 1 1 Poston Error FCI Tme k Velocty Error Tme k Fgure : FCI and FCI Fgure compares the RMS poston errors and RMS velocty errors of FCI and, respectvely, computed from 1 Monte Carlo runs. As can be seen, the two algorthms have smlar performance, and FCI s somewhat better than n estmatng poston, but s slghtly better than FCI n estmatng velocty. The average computatonal tmes of and FCI for Matlab 7..1 over 1 seconds are: t =.87 s t FCI =.13 s respectvely. The fuson estmate was computed by usng the routne presented n 11]. The tmes lsted above were obtaned on a computer wth Core Duo CPU at.33 GHz. Obvously, the computaton of FCI s far less than that of the.. FFCC and CI/CU To compare the performance of the FFCC and CI/CU algorthms, t s assumed that both nodes apply the same fuson mechansm: FFCC or CI/CU. Consderng the same scenaro presented above, t s assumed that node s corrupted by whte Gaussan nose θ k wth mean E θ k ] = 1 and varance varθ k ] = 9 durng the tme from 1s to 6s. The measurement equaton for node s gven by: { z k = H x k + v k + θ k, 1 k 6 z k = H x k + v k, others The model msmatch of the Kalman flter n node makes the estmate nconsstent wth the one obtaned 76

7 by node 1. For CI/CU, to address ths sensor falure problem, a threshold should be defned frst. In order to detect nconsstency as early as possble, we calculated the Mahalanobs dstance between ˇx 1 1, ˇP 1 1 : ˇx 1 1 1, ˇP T Md 1 = ˇx ˇP ˇx ˇP 1 1 ˇx ˇx 1 1 and 1 and the average Mahalanobs dstance over 1 Monte Carlo runs at tme 1s s AMd 1 =8.. Thus the threshold was determned to be 8. Ths s an deal selecton of threshold for CI/CU, not sensble for real systems. Fgure 6 compares the RMS poston errors and RMS velocty errors of FFCC and CI/CU, respectvely, computed from 1 Monte Carlo runs. As can be seen, CI/CU and FFCC have smlar performance n estmatng velocty. However, n estmatng poston, FFCC s better than CI/CU. It also can be shown that the proposed adaptve parameter δ can really reflect the degree of nconsstency between local estmates. The average computatonal tmes of CI/CU and FFCC for Matlab 7..1 over 1 seconds are: t CI/CU =.7 s t FFCC =.1 s respectvely. The CU fuson estmate was computed by usng SeDuM 17] and YALMIP 18]. The tmes lsted above were obtaned on a computer wth Core Duo CPU at.33 GHz. Obvously, FFCC has far less computaton than CI/CU. It s worth notng that numercal problem wll arse n the FFCC fuson f δ. A user-defned lower bound l can be used to sovle ths problem. For example, l =.1, f δ<l,thenletδ = l. Conclusons Ths paper has consdered the estmaton fuson problem underunknowncross-correlatonbetween the errors of estmates to be fused. We have presented a fast CI FCI algorthm based on a proposed mnmzaton crteron, whch leads to a close form soluton. Thus, the computaton of the FCI algorthm s far less than the CI algorthm under the determnant mnmzaton crteron. We have ntroduced an adaptve parameter δ,and a fast and fault-tolerant convex combnaton FFCC fuson algorthm has been worked out wthout a need for any pre-defned threshold. Both the FCI and FFCC algorthms can be generalzed to the case of more than two estmates. The performance of our proposed FCI and FFCC algorthms has been compared wth that of and CI/CU. In llustratve smulaton examples, t RMS Poston Error RMS Velocty Error AMd δ 3 1 Poston Error Tme k Velocty Error Tme k Average Mahalanobs dstance Tme k Average overall "belef" Tme k Fgure 6: FFCC and CI/CU CI/CU FFCC CI/CU FFCC AMd Threshold has been shownthat FCI and have smlar performance when fusng consstent estmates. Wth a slght performance degradaton n estmatng velocty, FFCC s better than another fault-tolerant mechansm CI/CU n estmatng poston. Moreover, FFCC algorthm can save a large amount of computaton. References 1] X. R. L, Y. M. Zhu, J. Wang, and C. Z. Han, Optmal lnear estmaton fuson - part I: Unfed fuson rules, IEEE Transactons on Informaton Theory, vol. 9, no. 9, pp. 19 8, September 3. ] C. Y. Chong, Herarchcal estmaton, n Proceedngs of the MIT/ONR Workshop on C3,Monterey, CA,

8 3] C. Y. Chong, S. Mor, and K. C. Chang, Adaptve dstrbuted estmaton, n Proc. 6th IEEE Conf. Decson and Contr., Los Angeles, CA, Dec. pp , ] Y. Bar-Shalom and L. Campo, The effect of the common process nose on the two-sensor fused-track covarance, IEEE Transactons on Aerospace and Electronc Systems, vol., no. 6, pp. 83 8, November ] K. C. Chang, R. K. Saha, and Y. Bar-Shalom, On optmal trak-to-track fuson, IEEE Transactons on Aerospace and Electronc Systems, vol. 33, no., pp , October ]M.Lggns,C.Y.Chong,I.Kadar,M.G.Alford, V. Vanncola, and S. Thomopoulos, Dstrbuted fuson archtectures and algorthms for target trackng, n Proc. IEEE, vol. 8, pp. 9 17, Jan ] W. Nehsen, Informaton fuson based on fast covarance ntersecton flterng, n Proceedngs of the th Internatonal Conference of Informaton Fuson, Annapols, MD, USA, pp. 91-9,. 1] J. K. Uhlmann. Covarance consstency methods for fault-tolerant dstrbuted data fuson, Informaton. Fuson, pp. 1 1, 3. 16] H. Jeffreys, An nvarant form for the pror probablty n estmaton problems, n Proc. Roy. Soc. Lon., Ser. A, vol. 186, pp. 3-61, ] J. F. Sturm, Usng SeDuM 1., a Matlab toolbox for optmzaton over symmetrc cones, Optmzaton Methods and Software, no. 11-1, pp. 6 63, ] J. Lofberg, YALMIP: A toolbox for modelng and optmzaton n MATLAB, n IEEE Internatonal Symposum on Computer Aded Control Systems Desgn, Tape, pp. 8-89,. 7]S.Mor,W.H.Barker,C.Y.Chong,andK.C. Chang, Track assocaton and track fuson wth non-determnstc target dynamcs, IEEE Transactons on Aerospace and Electronc Systems, vol. 38, no., pp , Aprl. 8] O.E. Drummond, On track and tracklet fuson flterng, n Proceedngs of SPIE, Sgnal and Data Processng of Small Targets, vol.78, pp ,. 9] J. K. Uhlmann. General data fuson for estmates wth unknown cross covarances, n Proceedngs of the SPIE Aerosense Conference, vol.7, pp. 36-7, ] S.J. Juler and J.K. Uhlmann, Non-dvergent estmaton algorthm n the presence of unknown correlatons, n Proc. Amer. Control Conf., vol., pp , ] S.J. Juler and J.K. Uhlmann, General decentralzed data fuson wth Covarance Intersecton CI, n Handbook of Multsensor Data Fuson, D.Hall and J. Llans, Eds. Boca Raton, FL: CRC Press, ch. 1, pp , 1. 1] M. Hurley, An nformaton theoretc justfcaton for covarance ntersecton and ts generalzaton, n Proceedngs of the th Internatonal Conference of Informaton Fuson, Annapols, MD, USA, pp. -11,. 13] T. M. Cover and J. A. Thomas, Elements of Informaton Theory, Wley-Interscence, New York,

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton

More information

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:

More information

Composite Hypotheses testing

Composite Hypotheses testing Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter

More information

Synchronized Multi-sensor Tracks Association and Fusion

Synchronized Multi-sensor Tracks Association and Fusion Synchronzed Mult-sensor Tracks Assocaton and Fuson Dongguang Zuo Chongzhao an School of Electronc and nformaton Engneerng X an Jaotong Unversty Xan 749, P.R. Chna Zlz_3@sna.com.cn czhan@jtu.edu.cn Abstract

More information

ECE559VV Project Report

ECE559VV Project Report ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate

More information

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4) I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes

More information

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute

More information

Computing MLE Bias Empirically

Computing MLE Bias Empirically Computng MLE Bas Emprcally Kar Wa Lm Australan atonal Unversty January 3, 27 Abstract Ths note studes the bas arses from the MLE estmate of the rate parameter and the mean parameter of an exponental dstrbuton.

More information

Tracking with Kalman Filter

Tracking with Kalman Filter Trackng wth Kalman Flter Scott T. Acton Vrgna Image and Vdeo Analyss (VIVA), Charles L. Brown Department of Electrcal and Computer Engneerng Department of Bomedcal Engneerng Unversty of Vrgna, Charlottesvlle,

More information

4DVAR, according to the name, is a four-dimensional variational method.

4DVAR, according to the name, is a four-dimensional variational method. 4D-Varatonal Data Assmlaton (4D-Var) 4DVAR, accordng to the name, s a four-dmensonal varatonal method. 4D-Var s actually a drect generalzaton of 3D-Var to handle observatons that are dstrbuted n tme. The

More information

Lecture Notes on Linear Regression

Lecture Notes on Linear Regression Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume

More information

Lecture 12: Classification

Lecture 12: Classification Lecture : Classfcaton g Dscrmnant functons g The optmal Bayes classfer g Quadratc classfers g Eucldean and Mahalanobs metrcs g K Nearest Neghbor Classfers Intellgent Sensor Systems Rcardo Guterrez-Osuna

More information

LOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin

LOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence

More information

Appendix B: Resampling Algorithms

Appendix B: Resampling Algorithms 407 Appendx B: Resamplng Algorthms A common problem of all partcle flters s the degeneracy of weghts, whch conssts of the unbounded ncrease of the varance of the mportance weghts ω [ ] of the partcles

More information

MACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression

MACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression 11 MACHINE APPLIED MACHINE LEARNING LEARNING MACHINE LEARNING Gaussan Mture Regresson 22 MACHINE APPLIED MACHINE LEARNING LEARNING Bref summary of last week s lecture 33 MACHINE APPLIED MACHINE LEARNING

More information

Parameter Estimation for Dynamic System using Unscented Kalman filter

Parameter Estimation for Dynamic System using Unscented Kalman filter Parameter Estmaton for Dynamc System usng Unscented Kalman flter Jhoon Seung 1,a, Amr Atya F. 2,b, Alexander G.Parlos 3,c, and Klto Chong 1,4,d* 1 Dvson of Electroncs Engneerng, Chonbuk Natonal Unversty,

More information

Gaussian Mixture Models

Gaussian Mixture Models Lab Gaussan Mxture Models Lab Objectve: Understand the formulaton of Gaussan Mxture Models (GMMs) and how to estmate GMM parameters. You ve already seen GMMs as the observaton dstrbuton n certan contnuous

More information

P R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /

P R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering / Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons

More information

CSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography

CSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve

More information

Parametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010

Parametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010 Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton

More information

Linear Feature Engineering 11

Linear Feature Engineering 11 Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19

More information

Generalized Linear Methods

Generalized Linear Methods Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set

More information

U.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017

U.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017 U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that

More information

A Particle Filter Algorithm based on Mixing of Prior probability density and UKF as Generate Importance Function

A Particle Filter Algorithm based on Mixing of Prior probability density and UKF as Generate Importance Function Advanced Scence and Technology Letters, pp.83-87 http://dx.do.org/10.14257/astl.2014.53.20 A Partcle Flter Algorthm based on Mxng of Pror probablty densty and UKF as Generate Importance Functon Lu Lu 1,1,

More information

Lecture 3: Shannon s Theorem

Lecture 3: Shannon s Theorem CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts

More information

Classification as a Regression Problem

Classification as a Regression Problem Target varable y C C, C,, ; Classfcaton as a Regresson Problem { }, 3 L C K To treat classfcaton as a regresson problem we should transform the target y nto numercal values; The choce of numercal class

More information

Hidden Markov Models & The Multivariate Gaussian (10/26/04)

Hidden Markov Models & The Multivariate Gaussian (10/26/04) CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models

More information

Power law and dimension of the maximum value for belief distribution with the max Deng entropy

Power law and dimension of the maximum value for belief distribution with the max Deng entropy Power law and dmenson of the maxmum value for belef dstrbuton wth the max Deng entropy Bngy Kang a, a College of Informaton Engneerng, Northwest A&F Unversty, Yanglng, Shaanx, 712100, Chna. Abstract Deng

More information

An Application of Fuzzy Hypotheses Testing in Radar Detection

An Application of Fuzzy Hypotheses Testing in Radar Detection Proceedngs of the th WSES Internatonal Conference on FUZZY SYSEMS n pplcaton of Fuy Hypotheses estng n Radar Detecton.K.ELSHERIF, F.M.BBDY, G.M.BDELHMID Department of Mathematcs Mltary echncal Collage

More information

PHYS 705: Classical Mechanics. Calculus of Variations II

PHYS 705: Classical Mechanics. Calculus of Variations II 1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary

More information

Outline. Communication. Bellman Ford Algorithm. Bellman Ford Example. Bellman Ford Shortest Path [1]

Outline. Communication. Bellman Ford Algorithm. Bellman Ford Example. Bellman Ford Shortest Path [1] DYNAMIC SHORTEST PATH SEARCH AND SYNCHRONIZED TASK SWITCHING Jay Wagenpfel, Adran Trachte 2 Outlne Shortest Communcaton Path Searchng Bellmann Ford algorthm Algorthm for dynamc case Modfcatons to our algorthm

More information

Comparison of the Population Variance Estimators. of 2-Parameter Exponential Distribution Based on. Multiple Criteria Decision Making Method

Comparison of the Population Variance Estimators. of 2-Parameter Exponential Distribution Based on. Multiple Criteria Decision Making Method Appled Mathematcal Scences, Vol. 7, 0, no. 47, 07-0 HIARI Ltd, www.m-hkar.com Comparson of the Populaton Varance Estmators of -Parameter Exponental Dstrbuton Based on Multple Crtera Decson Makng Method

More information

Time-Varying Systems and Computations Lecture 6

Time-Varying Systems and Computations Lecture 6 Tme-Varyng Systems and Computatons Lecture 6 Klaus Depold 14. Januar 2014 The Kalman Flter The Kalman estmaton flter attempts to estmate the actual state of an unknown dscrete dynamcal system, gven nosy

More information

Uncertainty as the Overlap of Alternate Conditional Distributions

Uncertainty as the Overlap of Alternate Conditional Distributions Uncertanty as the Overlap of Alternate Condtonal Dstrbutons Olena Babak and Clayton V. Deutsch Centre for Computatonal Geostatstcs Department of Cvl & Envronmental Engneerng Unversty of Alberta An mportant

More information

This column is a continuation of our previous column

This column is a continuation of our previous column Comparson of Goodness of Ft Statstcs for Lnear Regresson, Part II The authors contnue ther dscusson of the correlaton coeffcent n developng a calbraton for quanttatve analyss. Jerome Workman Jr. and Howard

More information

Chapter 2 Robust Covariance Intersection Fusion Steady-State Kalman Filter with Uncertain Parameters

Chapter 2 Robust Covariance Intersection Fusion Steady-State Kalman Filter with Uncertain Parameters Chapter 2 Robust Covarance Intersecton Fuson Steady-State Kalman Flter wth Uncertan Parameters Wenjuan Q, Xueme Wang, Wenqang Lu and Zl Deng Abstract For the lnear dscrete tme-nvarant system wth uncertan

More information

Computation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models

Computation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models Computaton of Hgher Order Moments from Two Multnomal Overdsperson Lkelhood Models BY J. T. NEWCOMER, N. K. NEERCHAL Department of Mathematcs and Statstcs, Unversty of Maryland, Baltmore County, Baltmore,

More information

Multigradient for Neural Networks for Equalizers 1

Multigradient for Neural Networks for Equalizers 1 Multgradent for Neural Netorks for Equalzers 1 Chulhee ee, Jnook Go and Heeyoung Km Department of Electrcal and Electronc Engneerng Yonse Unversty 134 Shnchon-Dong, Seodaemun-Ku, Seoul 1-749, Korea ABSTRACT

More information

Using T.O.M to Estimate Parameter of distributions that have not Single Exponential Family

Using T.O.M to Estimate Parameter of distributions that have not Single Exponential Family IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran

More information

Estimating the Fundamental Matrix by Transforming Image Points in Projective Space 1

Estimating the Fundamental Matrix by Transforming Image Points in Projective Space 1 Estmatng the Fundamental Matrx by Transformng Image Ponts n Projectve Space 1 Zhengyou Zhang and Charles Loop Mcrosoft Research, One Mcrosoft Way, Redmond, WA 98052, USA E-mal: fzhang,cloopg@mcrosoft.com

More information

Basic Statistical Analysis and Yield Calculations

Basic Statistical Analysis and Yield Calculations October 17, 007 Basc Statstcal Analyss and Yeld Calculatons Dr. José Ernesto Rayas Sánchez 1 Outlne Sources of desgn-performance uncertanty Desgn and development processes Desgn for manufacturablty A general

More information

Statistical analysis using matlab. HY 439 Presented by: George Fortetsanakis

Statistical analysis using matlab. HY 439 Presented by: George Fortetsanakis Statstcal analyss usng matlab HY 439 Presented by: George Fortetsanaks Roadmap Probablty dstrbutons Statstcal estmaton Fttng data to probablty dstrbutons Contnuous dstrbutons Contnuous random varable X

More information

A Robust Method for Calculating the Correlation Coefficient

A Robust Method for Calculating the Correlation Coefficient A Robust Method for Calculatng the Correlaton Coeffcent E.B. Nven and C. V. Deutsch Relatonshps between prmary and secondary data are frequently quantfed usng the correlaton coeffcent; however, the tradtonal

More information

The Prncpal Component Transform The Prncpal Component Transform s also called Karhunen-Loeve Transform (KLT, Hotellng Transform, oregenvector Transfor

The Prncpal Component Transform The Prncpal Component Transform s also called Karhunen-Loeve Transform (KLT, Hotellng Transform, oregenvector Transfor Prncpal Component Transform Multvarate Random Sgnals A real tme sgnal x(t can be consdered as a random process and ts samples x m (m =0; ;N, 1 a random vector: The mean vector of X s X =[x0; ;x N,1] T

More information

CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE

CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng

More information

Simulated Power of the Discrete Cramér-von Mises Goodness-of-Fit Tests

Simulated Power of the Discrete Cramér-von Mises Goodness-of-Fit Tests Smulated of the Cramér-von Mses Goodness-of-Ft Tests Steele, M., Chaselng, J. and 3 Hurst, C. School of Mathematcal and Physcal Scences, James Cook Unversty, Australan School of Envronmental Studes, Grffth

More information

However, since P is a symmetric idempotent matrix, of P are either 0 or 1 [Eigen-values

However, since P is a symmetric idempotent matrix, of P are either 0 or 1 [Eigen-values Fall 007 Soluton to Mdterm Examnaton STAT 7 Dr. Goel. [0 ponts] For the general lnear model = X + ε, wth uncorrelated errors havng mean zero and varance σ, suppose that the desgn matrx X s not necessarly

More information

Homework Assignment 3 Due in class, Thursday October 15

Homework Assignment 3 Due in class, Thursday October 15 Homework Assgnment 3 Due n class, Thursday October 15 SDS 383C Statstcal Modelng I 1 Rdge regresson and Lasso 1. Get the Prostrate cancer data from http://statweb.stanford.edu/~tbs/elemstatlearn/ datasets/prostate.data.

More information

Chat eld, C. and A.J.Collins, Introduction to multivariate analysis. Chapman & Hall, 1980

Chat eld, C. and A.J.Collins, Introduction to multivariate analysis. Chapman & Hall, 1980 MT07: Multvarate Statstcal Methods Mke Tso: emal mke.tso@manchester.ac.uk Webpage for notes: http://www.maths.manchester.ac.uk/~mkt/new_teachng.htm. Introducton to multvarate data. Books Chat eld, C. and

More information

The Geometry of Logit and Probit

The Geometry of Logit and Probit The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.

More information

Numerical Heat and Mass Transfer

Numerical Heat and Mass Transfer Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and

More information

Markov Chain Monte Carlo Lecture 6

Markov Chain Monte Carlo Lecture 6 where (x 1,..., x N ) X N, N s called the populaton sze, f(x) f (x) for at least one {1, 2,..., N}, and those dfferent from f(x) are called the tral dstrbutons n terms of mportance samplng. Dfferent ways

More information

Estimation: Part 2. Chapter GREG estimation

Estimation: Part 2. Chapter GREG estimation Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the

More information

Linear Approximation with Regularization and Moving Least Squares

Linear Approximation with Regularization and Moving Least Squares Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...

More information

VQ widely used in coding speech, image, and video

VQ widely used in coding speech, image, and video at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng

More information

On Mixture Reduction for Multiple Target Tracking

On Mixture Reduction for Multiple Target Tracking On Mxture Reducton for Multple Target Tracng Tohd Ardeshr, Umut Orguner, Chrstan Lundqust, Thomas B. Schön Dvson of Automatc Control, Department of Electrcal Engneerng, Lnöpng Unversty, 8 83 Lnöpng, Sweden

More information

Copyright 2017 by Taylor Enterprises, Inc., All Rights Reserved. Adjusted Control Limits for U Charts. Dr. Wayne A. Taylor

Copyright 2017 by Taylor Enterprises, Inc., All Rights Reserved. Adjusted Control Limits for U Charts. Dr. Wayne A. Taylor Taylor Enterprses, Inc. Adjusted Control Lmts for U Charts Copyrght 207 by Taylor Enterprses, Inc., All Rghts Reserved. Adjusted Control Lmts for U Charts Dr. Wayne A. Taylor Abstract: U charts are used

More information

System in Weibull Distribution

System in Weibull Distribution Internatonal Matheatcal Foru 4 9 no. 9 94-95 Relablty Equvalence Factors of a Seres-Parallel Syste n Webull Dstrbuton M. A. El-Dacese Matheatcs Departent Faculty of Scence Tanta Unversty Tanta Egypt eldacese@yahoo.co

More information

Durban Watson for Testing the Lack-of-Fit of Polynomial Regression Models without Replications

Durban Watson for Testing the Lack-of-Fit of Polynomial Regression Models without Replications Durban Watson for Testng the Lack-of-Ft of Polynomal Regresson Models wthout Replcatons Ruba A. Alyaf, Maha A. Omar, Abdullah A. Al-Shha ralyaf@ksu.edu.sa, maomar@ksu.edu.sa, aalshha@ksu.edu.sa Department

More information

8/25/17. Data Modeling. Data Modeling. Data Modeling. Patrice Koehl Department of Biological Sciences National University of Singapore

8/25/17. Data Modeling. Data Modeling. Data Modeling. Patrice Koehl Department of Biological Sciences National University of Singapore 8/5/17 Data Modelng Patrce Koehl Department of Bologcal Scences atonal Unversty of Sngapore http://www.cs.ucdavs.edu/~koehl/teachng/bl59 koehl@cs.ucdavs.edu Data Modelng Ø Data Modelng: least squares Ø

More information

Chapter 11: Simple Linear Regression and Correlation

Chapter 11: Simple Linear Regression and Correlation Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests

More information

j) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1

j) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1 Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons

More information

Chapter 7 Generalized and Weighted Least Squares Estimation. In this method, the deviation between the observed and expected values of

Chapter 7 Generalized and Weighted Least Squares Estimation. In this method, the deviation between the observed and expected values of Chapter 7 Generalzed and Weghted Least Squares Estmaton The usual lnear regresson model assumes that all the random error components are dentcally and ndependently dstrbuted wth constant varance. When

More information

Error Probability for M Signals

Error Probability for M Signals Chapter 3 rror Probablty for M Sgnals In ths chapter we dscuss the error probablty n decdng whch of M sgnals was transmtted over an arbtrary channel. We assume the sgnals are represented by a set of orthonormal

More information

Mathematical Preparations

Mathematical Preparations 1 Introducton Mathematcal Preparatons The theory of relatvty was developed to explan experments whch studed the propagaton of electromagnetc radaton n movng coordnate systems. Wthn expermental error the

More information

Chapter 8 Indicator Variables

Chapter 8 Indicator Variables Chapter 8 Indcator Varables In general, e explanatory varables n any regresson analyss are assumed to be quanttatve n nature. For example, e varables lke temperature, dstance, age etc. are quanttatve n

More information

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 31 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 6. Rdge regresson The OLSE s the best lnear unbased

More information

SIO 224. m(r) =(ρ(r),k s (r),µ(r))

SIO 224. m(r) =(ρ(r),k s (r),µ(r)) SIO 224 1. A bref look at resoluton analyss Here s some background for the Masters and Gubbns resoluton paper. Global Earth models are usually found teratvely by assumng a startng model and fndng small

More information

Kernel Methods and SVMs Extension

Kernel Methods and SVMs Extension Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general

More information

Formulas for the Determinant

Formulas for the Determinant page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use

More information

Why Bayesian? 3. Bayes and Normal Models. State of nature: class. Decision rule. Rev. Thomas Bayes ( ) Bayes Theorem (yes, the famous one)

Why Bayesian? 3. Bayes and Normal Models. State of nature: class. Decision rule. Rev. Thomas Bayes ( ) Bayes Theorem (yes, the famous one) Why Bayesan? 3. Bayes and Normal Models Alex M. Martnez alex@ece.osu.edu Handouts Handoutsfor forece ECE874 874Sp Sp007 If all our research (n PR was to dsappear and you could only save one theory, whch

More information

Explaining the Stein Paradox

Explaining the Stein Paradox Explanng the Sten Paradox Kwong Hu Yung 1999/06/10 Abstract Ths report offers several ratonale for the Sten paradox. Sectons 1 and defnes the multvarate normal mean estmaton problem and ntroduces Sten

More information

A PROBABILITY-DRIVEN SEARCH ALGORITHM FOR SOLVING MULTI-OBJECTIVE OPTIMIZATION PROBLEMS

A PROBABILITY-DRIVEN SEARCH ALGORITHM FOR SOLVING MULTI-OBJECTIVE OPTIMIZATION PROBLEMS HCMC Unversty of Pedagogy Thong Nguyen Huu et al. A PROBABILITY-DRIVEN SEARCH ALGORITHM FOR SOLVING MULTI-OBJECTIVE OPTIMIZATION PROBLEMS Thong Nguyen Huu and Hao Tran Van Department of mathematcs-nformaton,

More information

Clock Synchronization in WSN: from Traditional Estimation Theory to Distributed Signal Processing

Clock Synchronization in WSN: from Traditional Estimation Theory to Distributed Signal Processing Clock Synchronzaton n WS: from Tradtonal Estmaton Theory to Dstrbuted Sgnal Processng Yk-Chung WU The Unversty of Hong Kong Emal: ycwu@eee.hku.hk, Webpage: www.eee.hku.hk/~ycwu Applcatons requre clock

More information

Structure and Drive Paul A. Jensen Copyright July 20, 2003

Structure and Drive Paul A. Jensen Copyright July 20, 2003 Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.

More information

Economics 130. Lecture 4 Simple Linear Regression Continued

Economics 130. Lecture 4 Simple Linear Regression Continued Economcs 130 Lecture 4 Contnued Readngs for Week 4 Text, Chapter and 3. We contnue wth addressng our second ssue + add n how we evaluate these relatonshps: Where do we get data to do ths analyss? How do

More information

Design and Optimization of Fuzzy Controller for Inverse Pendulum System Using Genetic Algorithm

Design and Optimization of Fuzzy Controller for Inverse Pendulum System Using Genetic Algorithm Desgn and Optmzaton of Fuzzy Controller for Inverse Pendulum System Usng Genetc Algorthm H. Mehraban A. Ashoor Unversty of Tehran Unversty of Tehran h.mehraban@ece.ut.ac.r a.ashoor@ece.ut.ac.r Abstract:

More information

What would be a reasonable choice of the quantization step Δ?

What would be a reasonable choice of the quantization step Δ? CE 108 HOMEWORK 4 EXERCISE 1. Suppose you are samplng the output of a sensor at 10 KHz and quantze t wth a unform quantzer at 10 ts per sample. Assume that the margnal pdf of the sgnal s Gaussan wth mean

More information

Global Sensitivity. Tuesday 20 th February, 2018

Global Sensitivity. Tuesday 20 th February, 2018 Global Senstvty Tuesday 2 th February, 28 ) Local Senstvty Most senstvty analyses [] are based on local estmates of senstvty, typcally by expandng the response n a Taylor seres about some specfc values

More information

A Hybrid Variational Iteration Method for Blasius Equation

A Hybrid Variational Iteration Method for Blasius Equation Avalable at http://pvamu.edu/aam Appl. Appl. Math. ISSN: 1932-9466 Vol. 10, Issue 1 (June 2015), pp. 223-229 Applcatons and Appled Mathematcs: An Internatonal Journal (AAM) A Hybrd Varatonal Iteraton Method

More information

2016 Wiley. Study Session 2: Ethical and Professional Standards Application

2016 Wiley. Study Session 2: Ethical and Professional Standards Application 6 Wley Study Sesson : Ethcal and Professonal Standards Applcaton LESSON : CORRECTION ANALYSIS Readng 9: Correlaton and Regresson LOS 9a: Calculate and nterpret a sample covarance and a sample correlaton

More information

PHYS 450 Spring semester Lecture 02: Dealing with Experimental Uncertainties. Ron Reifenberger Birck Nanotechnology Center Purdue University

PHYS 450 Spring semester Lecture 02: Dealing with Experimental Uncertainties. Ron Reifenberger Birck Nanotechnology Center Purdue University PHYS 45 Sprng semester 7 Lecture : Dealng wth Expermental Uncertantes Ron Refenberger Brck anotechnology Center Purdue Unversty Lecture Introductory Comments Expermental errors (really expermental uncertantes)

More information

Errors for Linear Systems

Errors for Linear Systems Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch

More information

Supporting Information

Supporting Information Supportng Informaton The neural network f n Eq. 1 s gven by: f x l = ReLU W atom x l + b atom, 2 where ReLU s the element-wse rectfed lnear unt, 21.e., ReLUx = max0, x, W atom R d d s the weght matrx to

More information

ANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)

ANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U) Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of

More information

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons

More information

Stat260: Bayesian Modeling and Inference Lecture Date: February 22, Reference Priors

Stat260: Bayesian Modeling and Inference Lecture Date: February 22, Reference Priors Stat60: Bayesan Modelng and Inference Lecture Date: February, 00 Reference Prors Lecturer: Mchael I. Jordan Scrbe: Steven Troxler and Wayne Lee In ths lecture, we assume that θ R; n hgher-dmensons, reference

More information

Information Weighted Consensus

Information Weighted Consensus Informaton Weghted Consensus A. T. Kamal, J. A. Farrell and A. K. Roy-Chowdhury Unversty of Calforna, Rversde, CA-92521 Abstract Consensus-based dstrbuted estmaton schemes are becomng ncreasngly popular

More information

Problem Set 9 Solutions

Problem Set 9 Solutions Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem

More information

Joint Statistical Meetings - Biopharmaceutical Section

Joint Statistical Meetings - Biopharmaceutical Section Iteratve Ch-Square Test for Equvalence of Multple Treatment Groups Te-Hua Ng*, U.S. Food and Drug Admnstraton 1401 Rockvlle Pke, #200S, HFM-217, Rockvlle, MD 20852-1448 Key Words: Equvalence Testng; Actve

More information

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009 College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:

More information

= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.

= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system. Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and

More information

Pop-Click Noise Detection Using Inter-Frame Correlation for Improved Portable Auditory Sensing

Pop-Click Noise Detection Using Inter-Frame Correlation for Improved Portable Auditory Sensing Advanced Scence and Technology Letters, pp.164-168 http://dx.do.org/10.14257/astl.2013 Pop-Clc Nose Detecton Usng Inter-Frame Correlaton for Improved Portable Audtory Sensng Dong Yun Lee, Kwang Myung Jeon,

More information

Radar Trackers. Study Guide. All chapters, problems, examples and page numbers refer to Applied Optimal Estimation, A. Gelb, Ed.

Radar Trackers. Study Guide. All chapters, problems, examples and page numbers refer to Applied Optimal Estimation, A. Gelb, Ed. Radar rackers Study Gude All chapters, problems, examples and page numbers refer to Appled Optmal Estmaton, A. Gelb, Ed. Chapter Example.0- Problem Statement wo sensors Each has a sngle nose measurement

More information

JAB Chain. Long-tail claims development. ASTIN - September 2005 B.Verdier A. Klinger

JAB Chain. Long-tail claims development. ASTIN - September 2005 B.Verdier A. Klinger JAB Chan Long-tal clams development ASTIN - September 2005 B.Verder A. Klnger Outlne Chan Ladder : comments A frst soluton: Munch Chan Ladder JAB Chan Chan Ladder: Comments Black lne: average pad to ncurred

More information

Statistical Inference. 2.3 Summary Statistics Measures of Center and Spread. parameters ( population characteristics )

Statistical Inference. 2.3 Summary Statistics Measures of Center and Spread. parameters ( population characteristics ) Ismor Fscher, 8//008 Stat 54 / -8.3 Summary Statstcs Measures of Center and Spread Dstrbuton of dscrete contnuous POPULATION Random Varable, numercal True center =??? True spread =???? parameters ( populaton

More information

10-701/ Machine Learning, Fall 2005 Homework 3

10-701/ Machine Learning, Fall 2005 Homework 3 10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40

More information

Lecture 3 Stat102, Spring 2007

Lecture 3 Stat102, Spring 2007 Lecture 3 Stat0, Sprng 007 Chapter 3. 3.: Introducton to regresson analyss Lnear regresson as a descrptve technque The least-squares equatons Chapter 3.3 Samplng dstrbuton of b 0, b. Contnued n net lecture

More information

Learning Theory: Lecture Notes

Learning Theory: Lecture Notes Learnng Theory: Lecture Notes Lecturer: Kamalka Chaudhur Scrbe: Qush Wang October 27, 2012 1 The Agnostc PAC Model Recall that one of the constrants of the PAC model s that the data dstrbuton has to be

More information

ENG 8801/ Special Topics in Computer Engineering: Pattern Recognition. Memorial University of Newfoundland Pattern Recognition

ENG 8801/ Special Topics in Computer Engineering: Pattern Recognition. Memorial University of Newfoundland Pattern Recognition EG 880/988 - Specal opcs n Computer Engneerng: Pattern Recognton Memoral Unversty of ewfoundland Pattern Recognton Lecture 7 May 3, 006 http://wwwengrmunca/~charlesr Offce Hours: uesdays hursdays 8:30-9:30

More information