Parameter Estimation for Relational Kalman Filtering

Transcription

1 Sascal Relaonal AI: Papers from he AAAI-4 Workshop Parameer Esmaon for Relaonal Kalman Flerng Jaesk Cho School of Elecrcal and Compuer Engneerng Ulsan Naonal Insue of Scence and Technology Ulsan, Korea Eyal Amr Deparmen of Compuer Scence Unversy of Illnos a Urbana-Champagn Urbana, IL, USA Tanfang Xu and Alber J Valocch Deparmen of Cvl and Envronmenal Engneerng Unversy of Illnos a Urbana-Champagn Urbana, IL, USA Absrac The Kalman Fler (KF) s pervasvely used o conrol a vas array of consumer, healh and defense producs By groupng ses of symmerc sae varables, he Relaonal Kalman Fler (RKF) enables o scale he exac KF for large-scale dynamc sysems In hs paper, we provde a parameer learnng algorhm for RKF, and a regroupng algorhm ha prevens he degeneraon of he relaonal srucure for effcen flerng The proposed algorhms sgnfcanly expand he applcably of he RKFs by solvng he followng quesons: () how o learn parameers for RKF n paral observaons; and () how o regroup he degeneraed sae varables by nosy real-world observaons We show ha our new algorhms mprove he effcency of flerng he largescale dynamc sysem Inroducon any real-world sysems can be modeled by connuous varables and relaonshps (or dependences) among hem The Kalman Fler (KF) (Kalman 960) accuraely esmaes he sae of varables n a lnear dynamc sysem wh Gaussan nose gven a sequence of conrol-npus and observaons The KF has been appled n a broad range of domans such as robocs, fnance (Bahman-Oskooee and Brown 004) and envronmenal scence (FP and Berkens 00; Clark e al 008) Gven a sequence of observaons and lnear dependences wh Gaussan nose beween varables, he KF calculaes he condonal probably densy of he sae varables a each me sep Unforunaely, he KF compuaons are cubc n he number of sae varables, whch lms he use of exsng exac mehods o domans wh a large number of sae varables Ths has led o he combnaon of approxmaon and samplng n he Ensemble Kalman Fler (Evensen 994), and recenly o he Relaonal Kalman Flers (RKFs) over grouped sae varables (Cho, Guzman-Rvera, and Amr 0; Ahmad, Kersng, and Sanner 0) The RKFs Copyrgh c 04, Assocaon for he Advancemen of Arfcal Inellgence (wwwaaaorg) All rghs reserved leverage he ably of relaonal languages o specfy models wh sze of represenaon ndependen of he sze of populaons nvolved (Fredman e al 999; Poole 003; Rchardson and Domngos 006) Lfed nference algorhms for relaonal connuous models (Wang and Domngos 008; Cho, Hll, and Amr 00; Ahmad, Kersng, and Sanner 0) degenerae (or spl) relaonal srucures upon ndvdual observaons Lfed RKF (Cho, Guzman-Rvera, and Amr 0) manans relaonal srucure when he same number of observaons are made Oherwse, also degeneraes (possbly rapdly) he relaonal srucure, hus lfed RKF may no be useful wh sparse observaons The man conrbuons of hs paper are () o regroup he degeneraed sae varables from nosy real-world observaons wh a gh error bounds; and () o learn parameers for RKFs We propose a new learnng algorhm for RKFs We show ha he relaonal learnng expedes flerng, and acheves accurae flerng n heory and pracce The key nuon s ha he axmum Lkelhood Esmae (LE) of RKF parameers s he emprcal mean and varance over sae varables ncluded n each group For paral observaons, he parameers can be calculaed smlarly We show ha under reasonable condons varances of degeneraed sae varables on paral observaons converge exponenally Thus, our approxmae regroupng algorhm has bounded errors compared o he exac KF We could show ha he RKF wh regroupng s more robus agans degeneracy han he Lfed RKF n pracce wh paral observaons Relaonal Lnear Dynamc Sysems In hs secon, we defne relaonal lnear dynamc sysems Dependences among varables are represened usng relaonal aoms, or jus aoms The relaonal aoms are useful when he jon probably of varables nvolves common ypes of funcons When represenng he jon probably For comprehensve defnons, see (Poole 003; de Salvo Braz, Amr, and Roh 005)

2 dsrbuon, here are producs of he parameerzed funcons (or poenals) 03 W (405N,03W) r (40N,03W) 0 W 0 W r 00 W 99 W 98 W 4 N 40 N A parwse Gaussan facor, or jus a facor, f = ((x, x ), φ) s a par where φ s a poenal funcon on (x, x ) from R o R + where (x, x ) s a par of ground random varables derved by ground subsuons from (X(Lθ), X (L θ )) A facor f defnes a weghng funcon on a valuaon (x, x ) = (v, v ): w f (x, x ) = φ(v, v ) The weghng funcon for a parfacor g s he produc of he weghng funcons over all of s ground subsuons (facors), w g (v) = f g w f (v) Hence, a se of parfacors G defnes a probably densy, w G (v) = w f (v), Z g G f g Colorado Nebraska Kansas les Sream nework 39 N where Z s he normalzng consan 3 In hs way, we can represen he jon probably of all random varables (eg all wells n regon and regon ) Relaonal Transon odels (RTs) characerze he dependence of relaonal aoms beween me seps X (a) and X j + (a ) are relaonal aoms a me sep and + respecvely when a and a are ground subsuons, eg, θ =(40N,98W) U (a) s he conrol-npu nformaon A RT akes he followng form, Fgure : Republcan Rver Basn coverng porons of easern Colorado, norhwes Kansas and souhwes Nebraska Ths fgure shows wo clusered waer wells; regon (r ) and regon (r ) Waer wells n each regon have he same (lnear Gaussan) relaonshps wh wells n oher regons Relaonal aoms represen he se of sae varables correspondng o all ground subsuons of s parameer varables For example, le X r (Laude, Longude) be an aom for he (waer level of) wells n regon, θ=(40n,03w ) When we subsue Laude and Longude wh θ, he aom becomes a sae varable X r (40N, 03W ) whch represens he level (or predcon) of well head a (Laude=40N, Longude=03W ) Formally, applyng a subsuon θ o an aom X(L) yelds a new aom X(Lθ) where Lθ s obaned by renamng he parameer varables n L accordng o θ If θ s a ground subsuon, X(Lθ) s a ground sae varable lke X r (40N, 03W ) X(L) or jus X denoes he he number of dsnc sae varables generaed from X by all subsuons A parwse Gaussan parfacor ((X, X ), φ) s composed of a par of wo aoms (X, X ) and a lnear Gaussan poenal φ beween wo aoms n he followng form [ (X X φ(x, X µ) ] ) exp For example, a parwse Gaussan parfacor φ r,r (X r, X r ) represens he lnear Gaussan relaonshp beween wo ground varables chosen from regon and regon respecvely Here, we assume ha he ground sae varables are unvarae, eg, doman of x s R However, s sraghforward o exend hs model o he one wh mulvarae ground varables X j + (a ) = B j X X (a) + B j U U (a) + G j RT, () where G j RT N (0, j RT ) and N (m, ) s he normal dsrbuon wh mean m and varance B j X and Bj U are he lnear ranson coeffcens In he lnear Gaussan represenaon, he ranson models ake he followng form, φ RT (X j + (a ) X(a), U (a)) [ exp (Xj + (a ) B j X X (a) B j U U ] (a)) () j RT The mos common ranson s he one from he sae X (a) o he sae self X +(a) a he nex me sep, X +(a) = B XX (a) + B U U (a) + G RT (3) Relaonal Observaon odels (s) represen he relaonshps beween he hdden (sae) varables, X (a), and he observaons made drecly on he relaed varable, O (a) (drec observaons), O (a) = H X (a) + G, G N (0, ) (4) H s he lnear coeffcen from X (a) o O (a) s also represen he relaonshps beween he hdden varables X (a) and he observaons made ndrecly on a varable n he aom O (a ) where a a (relaonal observaons), O (a ) = H X (a)+g, G N (0, ) (5) 3 The codon of beng a probably densy s ha a leas a random varable has a pror dsrbuon, see (Cho, Hll, and Amr 00)

3 n mos cases, s reasonable o se he varance of drec observaon var(g ) s smaller han he varance of relaonal one var(g ) s For he well example, X r (40N, 03W ), an observaon made a he exac locaon O r (40N, 03W ) wll have he smaller varance (more ceran) han an observaon made a a nearby locaon O r (405N, 03W ) In he lnear Gaussan represenaon, s ake he followng form, [ ] φ (O(a) X (a)) exp (O (a) HX (a)) Relaonal Parwse odels (RPs) represen lnear dependences beween pars of relaonal aoms, R j X(a) = R j X j (a ) + G j RP, Gj RP N (0, j RP ) (6) s he coeffcen Noe ha RTs and s represen he naure of dynamc sysems (eg he sae a he nex me sep depends on he curren me sep) The produc of RPs s an effcen way o represen he relaonal srucure over aoms, groups of sae varables Relaonal Kalman Fler (RKF) s a flerng procedure wh a relaonal lnear dynamc sysem whch s composed of RTs, s and RPs Tha s, he jon probably of sae varables are represened by he produc of parwse Gaussan parfacors Lfed RKF compues he poseror of he sae varables gven a pror (curren) belef and full or paral observaons The npu o he problem s: () relaonal parfacors (RTs, RPs and s); () curren belef over aoms (X0); (3) a sequence of conrolnpus (U,, UT ); and (4) a sequence of observaons (O,, OT ) The oupu s he mulvarae Gaussan dsrbuon over he aoms (XT ) a each me sep T The flerng problem s solved by algorhms represened (Cho, Guzman-Rvera, and Amr 0; Ahmad, Kersng, and Sanner 0) Here, we urn our focus o he parameer learnng problem 3 Learnng Relaonal Kalman Fler The wo mporan parameers of he RKF are he ranson models and observaon models In hs secon, we presen a learnng algorhm ha derves he maxmum lkelhood esmaes of RTs and s For smplcy, we wll presen a soluon wh fully observed model, frs A soluon for paral observaons can be derved wh a slgh modfcaon 3 Algorhm LearnngRKF Algorhm LearnngRKF esmaes he parameer of RKF gven a sequence of observaons such as measuremens of waer wells for several years The overall procedure s smlar o he one wh he parameer learnng for he ground KF Here, he man dfference s ha he coeffcens and covarances of RTs and s are he block marces A subroune, BlockAverage, averages ou he dagonal and nondagonal enres of an npu marx, and hen oupus a block marx where each block ncludes he emprcal means, varances and covarances n each block In he followng secons, we wll show ha he block marx compued by BlockAverage s he LE esmae Algorhm LearnngRKF npu: a sequence of obs (O,, O T ) (B, Σ T, C, Σ O ) (I, I, I, I) currenll repea prevll currenll (B, Σ T, C, Σ O) LearnGroundT(O,B,Σ T,C,Σ O) (B, Σ T, C, Σ O ) BlockAverage(B, Σ T, C, Σ O ) currenll log P (O X, B, Σ T, C, Σ O ) unl prevll - currenll < ɛ oupu: esmaed parameers (B, Σ T, C, Σ O ) 3 Learnng Transon odels Here, we derve he parameer of he RTs: lnear coeffcen B and Gaussan nose G RT Learnng Transon Nose s o calculae he mean and he covarance marx n he followng block forms, Σ, Σ, Σ,n µ T = µ µ µ n, Σ Σ, Σ, Σ,n T = (7) Σ n, Σ n, Σ n,n Where µ s a vecor of sze X (=n ); Σ,j s a marx of sze n by n j Gven a pror, a lnear coeffcen B and a sequence of full observaons, we derve he esmae X a me sep assumng he Gaussan nose n he ranson model The LE esmaon of µ and Σ for he RT can be derved: (µ T max, Σ T max ) = arg max log f N ( X ; µ T, Σ T ) µ T,Σ T =,,T where X =X BX Proposon Gven a RKF wh a sngle aom, he maxmum lkelhood esmaes of he Gaussan ranson nose are he emprcal mean, varance and covarance as follows, m m µ LE = such ha, m= n T m T = a A = n(n ) T, Σ LE = T = X (a), = n T T ( X (a) m) = a A ( ) ( X (a) m X (a ) m) a,a A a a where n = X (A) and T =T 4

4 Proof The LEs of he parameers (µ T, Σ T ) are derved by he paral dervaves of he log lkelhood, µ T Σ T T log f N ( X ; µ T, Σ T )=0, = T log f N ( X ; µ T, Σ T )=0 = All ground varables generaed from he aom X have he same mean, varance and covarances as shown n Equaon (8) Now, we can specfy he followng lnear consrans; m = T X (a ) = = T X (a n ) Tha s, m= n T a X (a) The covarance marx of he RT s also calculaed from he emprcal covarance marx The dagonal enres are derved for he varances; = T ( X (a ) m) = = T ( X (a n ) m) Thus, = ( X n T a (a) m) Non-dagonal enres (covarances) are derved smlarly wh n(n ) emprcal covarances 4 Ths resul s conssen wh he resul n non-relaonal KF because he LE esmaes of he ground KF (µ T and Σ T ) are known o be he emprcal mean and he emprcal covarance marx (Rowes and Ghahraman 999) In he general case, when we have mulple aoms, he mean vecor and he covarance marx are block forms as shown n Equaon 7 Tha s, he mean and covarance values are same n each subblock In case of, wo aoms X and X j, he means and covarances are as follows: µ T = [ ] µ, Σ µ T = [ ] Σ, Σ, Σ, Σ, The LE parameers of he RT are derved smlarly wh emprcal means and covarances of subblocks Proposon Gven a RKF wh mulple aoms, he maxmum lkelhood esmaes of he Gaussan ranson nose are he emprcal means, varances and covarances, µ = [m,, m ] T s m = n T T X (a), = Σ, = Σ,j = 4 One ges he same resul when dfferenang m, and drecly from he log-lkelhood a = n T = = T ( ) ( ) X (a) m X (a) m = a n (n ) T n n j T T T = a,a A (a a ) = a A,b B where n = X and T =T ( X (a) m ) ( X (a ) m ), ( X (a) m ) ( X j (b) m j ) Proof The prncples used n he proof of Proposon are appled because he Σ, and Σ,j are block marces Learnng Lnear Coeffcen s o esmae he lnear coeffcen B beween X and X In hs case, gven oher parameers, Gaussan nose of RTs and s, he LE of B s derved as follows (Rowes and Ghahraman 999), B = X X T X X T =,,T =,,T Here, we call he B lnear coeffcen of he ground T, and B s reshaped as a block marx by averagng he coeffcen n each subblock When B,j denoes he subblock for he lnear ranson from X j o X The LE of he block coeffcen s represened as follows, b b b b b b b b B, = B,j = b b b b b such ha, b= n, n k= B k,k, b = n (n ) b = (n,n j),j n n j (k,l)=(,) B k,l (n,n ) (k,l)=(,) B, k,l, k l BlockAverage n Algorhm LearnngRKF denoes hs averagng-ou procedure The block coeffcen marx B s also he LE of RT 33 Learnng Observaon odels Gven RTs and a sequence of full observaons, we derve he esmae X a me sep assumng ha here s no observaon nose Learnng Observaon Nose s o esmae he mean vecor and covarance marx for he The LEs problem s formulaed as follows, (µ LE, Σ LE ) = arg max µ O,Σ O T log N ( O ; µ O, Σ O ) = where O = O C X The dervaon s smlar o RTs One can subsue O for X n Proposon 5

5 Learnng Lnear Coeffcen C s o calculae he lnear coeffcen beween X and O Ĉ = O X T X X T =,,T =,,T Here C s also calculaed from Ĉ by averagng ou each subblock as n learnng B 4 LRKF wh Regroupngs Wh he esmaed parameers, he RKF predcs he sae varables n he relaonal lnear dynamc models Ths secon presens a new lfed Kalman flerng algorhm, whch approxmaely regroups degeneraed relaonal srucures Exsng lfed Kalman flerng algorhms (Cho, Guzman- Rvera, and Amr 0; Ahmad, Kersng, and Sanner 0) suffer degeneraons of relaonal srucures when sparse observaons are made 5 Algorhm LRKF-Regroup also degeneraes he domans of relaonal aoms by callng DegenAom when sae varables are observed n dfferen me, hus dfferen Obs (eg, Obs (,a) Obs (,a )) Here, Obs (,a) sores he mos recenly observed me for a ground subson a n he -h aom To overcome such degeneracy, LRKF-Regroup nroduces a new subroune, called ergeaom, whch merges covarance srucures when random varables are no drecly observed for a ceran me seps, say k Algorhm LRKF-Regroup (Predcon w/ esng daa) Inpu: params (B, Σ T, C, Σ O ), obs (O,, O T ) repea µ 0 0, Σ 0 0 (Obs (,),,Obs (n,n)) (0,, 0) for o T do (µ, Σ ) Predc-RT(µ, Σ, B, Σ T ) for all (, a) s O (a) s observed do Obs (,a) (B,Σ T,C,Σ O ) DegenAom(Obs, B,Σ T, C, Σ O ) (µ, Σ ) Updae-(µ, Σ, C, Σ O ) (B,Σ T,C,Σ O ) ergeaom(obs,, B,Σ T,C,Σ O ) unl s T Oupu: sae esmaons ((µ 0, Σ 0 ),, (µ T, Σ T )) The ergeaom operaon eraes aoms and fnd all sae varables whch are no observed for a ceran me seps k The seleced varables are sored n mls Then, he Blockerge respecvely averages dagonal enres and nondagonal enres n mls, and ses he averaged values o sae varables n mls In hs way, rebulds he compac relaonal srucure agan 5 Noe ha, he lfed algorhm n (Cho, Guzman-Rvera, and Amr 0) only vald when he same number of observaons are made a he same me seps Algorhm 3 ergeaom npu: recen obs me Obs, me, params (B,Σ T,C,Σ O ) mls for = o n do for each a s Obs (,a) +k do mls mls {a} (B,Σ T, C, Σ O ) = BlockAverage(mls, B, Σ T, C, Σ O ) oupu: merged relaonal srucures (B, Σ T, C, Σ O ) Lemma 3 When () a leas one relaonal observaon s made on O a each me sep; he varance of any sae varable X (a) n LRKF-Regroup s bounded by and converges o RT RT Proof Le he varance of he -h aom be RT and he varances of drec and relaonal observaons respecvely be and where < as Equaon (4) and (5) Le (a) be he varance of a sae varable X (a) a me Ths varable wll be updaed by a leas one relaonal observaon by he LRKF-Regroup Then, he new varance s = + + (a) (a) Tha s, +(a) mn((a), ) Use he followng equaon for ranson and updae n each flerng sep, + (a) (a) + RT + For he convergence, le +(a)= (a)= (a) The varance s (a) = RT RT Theorem 4 When () a leas one relaonal observaon s made on O a each me sep; and () no drec observaon s made on wo sae varables X (a) and X (a ) a leas for k me seps, he dfference of he varances of wo sae varables (a) and (a ) s bounded by c k RT (+ (a) / ) where c s (a) /( (a) + ) and c Proof We follow he resul of he Lemma 3 and use (a) The varance of each me sep follows he recursve form, + = (a) (a) + RT + (8) An exac (non-recursve) formula for +(a) s non rval Thus, we nroduce anoher smpler, convergng sequence, +(a) = c( (a) + RT ) Snce (a) (a) s posve and convex when c <, 0 (a) (a) (a), (a) (a) (a) (a) 6

6 Varance w/o relaonal obs w/ relaonal obs In he synhec daase, we assume ha an aom wh 300 ground subsuons, X =30 Then we make a sparse observaons wh a rae of 90% Tha s, 90% of sae varables wll be observed n each me sep Then, we repor he numbers of shaered (or degeneraed) groups and average flerng me n he Lfed RKF and n our LRKF-Regroup The resul s shown n Fgure Tme seps Fgure : Varances of sae varables wh one observaon per 0 me seps whou a relaonal obs (square shaped marks) and wh relaonal obs (crcle shaped marks) Ths fgure shows he varances (y axs) n me seps (x axs) Seng ha RT =0, =0, =05, and le observaons made a dfferen me seps Ths smulaon shows he nuon of Lemma 3 The convergence of he smpler form s slower han he orgnal one n Equaon (8) However, provdes an exac formulaon and converge exponenally, (a) = c k k(a) + RT c k c WLOG, we se X (a ) has no drec observaon longer han X (a) The varance of X (a ) has he same formulaon wh a subsuon of k+α for k Thus, varance of X (a ) s (a ) =c k+α k α(a ) +RT c k+α c Noe ha, +α(a) +α(a ) + (a ) +α(a ) (a) + (a ) (a) = c k ( RT /( c) +k (a) ) = c k ( RT ( + (a) / ) +k (a) ) c k RT ( + (a) / ) Fgure 3: Comparons of Lfed RKF and LRKF-Regoup n a smulaon daase To verfy he effcency of Lfed-Regroup, we use a regonal groundwaer flow ODFLOW model, he Republcan Rver Compac Assocaon (RRCA) odel (ckusck 003) The exraced daase s a small subse of he whole ground waer model, monhly measured head (waer level) a over 3,000 wells We choose a closely relaed 00 wells and randomly selece 00 measuremen perods (00 monhs) Then, we repor he degeneraces n Lfed-RKF and LRKF- Regroup and average flerng me affeced by he degeneraons n Fgure 4 5 Expermenal Resuls To compare he mprove he effcency, we mplemen he Lfed RKF (Cho, Guzman-Rvera, and Amr 0) and our LRKF-Regroup The algorhm s compared n wo daases wh sparse observaons: one synhec daase and one realworld ground waer daase Noe ha, he lfed RKF wll no degenerae model on full, dense, observaons In boh expermen, we se k o be 4 Tha s, wo sae varables wll be merged f hey have he same observaon numbers and ypes when a leas a relaonal observaon s made Wh relaonal observaons, he varances of degeneraed sae varables are reasonably small even afer 4 me seps Fgure 4: Comparons of Lfed RKF and LRKF-Regoup n he groundwaer model 7

7 6 Concluson Ths paper provdes new answers and nsghs on () how o learn parameers for RKF; and () how o regroup he sae varables from nosy real-world daa We propose a new algorhm ha regroups he sae varables when ndvdual observaons are made o RKF n dfferen me seps We use he RKF n a smulaed and a real-world daase, and demonsrae ha he RKF mproves he effcency of flerng he large-scale dynamc sysem 7 Acknowledgmens Ths work s suppored by he Naonal Scence Foundaon Hydrologc Scence Program under Gran No Ths wor was suppored by he year of 04 Research Fund of UNIST (Ulsan Naonal Insue of Scence and Technology) ckusck, V 003 Fnal repor for he specal maser wh cerfcae of adopon of rrca groundwaer model Sae of Kansas v Sae of Nebraska and Sae of Colorado, n he Supreme Cour of he Uned Saes 3605(45):4440 Poole, D 003 Frs-order probablsc nference In Proceedngs of he Inernaonal Jon Conference on Arfcal Inellgence, Rchardson,, and Domngos, P 006 arkov logc neworks achne Learnng 6(-):07 36 Rowes, S, and Ghahraman, Z 999 A unfyng revew of lnear gaussan models Neural Compu (): Wang, J, and Domngos, P 008 Hybrd markov logc neworks In Proceedngs of he AAAI Conference on Arfcal Inellgence, 06 References Ahmad, B; Kersng, K; and Sanner, S 0 ulevdence lfed message passng, wh applcaon o pagerank and he kalman fler In Proceedngs of he Inernaonal Jon Conference on Arfcal Inellgence Bahman-Oskooee,, and Brown, F 004 Kalman fler approach o esmae he demand for nernaonal reserves Appled Economcs 36(5): Cho, J; Guzman-Rvera, A; and Amr, E 0 Lfed relaonal kalman flerng In Proceedngs of he Inernaonal Jon Conference on Arfcal Inellgence Cho, J; Hll, D J; and Amr, E 00 Lfed nference for relaonal connuous models In Proceedngs of he Conference on Uncerany n Arfcal Inellgence, 6 34 Clark, P; Rupp, D E; Woods, R A; Zheng, X; Ibb, R P; Slaer, A G; Schmd, J; and Uddsrom, J 008 Hydrologcal daa assmlaon wh he ensemble kalman fler: Use of sreamflow observaons o updae saes n a dsrbued hydrologcal model Advances n Waer Resources 3(0): de Salvo Braz, R; Amr, E; and Roh, D 005 Lfed frsorder probablsc nference In Proceedngs of he Inernaonal Jon Conference on Arfcal Inellgence, Evensen, G 994 Sequenal daa assmlaon wh a nonlnear quas-geosrophc model usng mone carlo mehods o forecas error sascs Journal of Geophyscal Research 99: FP,, and Berkens 00 Spao-emporal modellng of he sol waer balance usng a sochasc model and sol profle descrpons Geoderma 03( ):7 50 Fredman, N; Geoor, L; Koller, D; and Pfeffer, A 999 Learnng probablsc relaonal models In Proceedngs of he Inernaonal Jon Conference on Arfcal Inellgence, Kalman, R E 960 A new approach o lnear flerng and predcon problems Transacons of he ASE Journal of Basc Engneerng 8(Seres D):