Hidden Markov Models

Size: px
Start display at page:

Download "Hidden Markov Models"

Transcription

1 Hdde Marov Models Magus Karlsso Bacgroud Hdde Marov chas was orgally troduced ad studed the late 960s ad early 970s. Durg the 980s the models became creasgly popular. he reaso for ths s twofolded. Frstly, the hdde Marov models are very rch mathematcal structure ad hece ca form the theoretcal bass for a wde rage of applcatos. Secodly, the models have, whe appled properly, tured out to be hghly successful. Some of the otable applcatos are speech recogto ad boformatcs partcular prote modellg. I ths wor, bascs for the hdde Marov models are descrbed. Problems, whch eed to be solved are outled, ad setches of the solutos are gve. A possble exteso of the models s dscussed ad some mplemetato ssues are cosdered. Fally, three examples of dfferet applcatos are dscussed. he vast maorty of the theoretcal results ths wor s a summary of the results Raber (989). he example speech recogto s due to Raber (989), the example of prote modellg s due to Krogh et al. (994) ad fally a applcato fatgue aalyss s due to Johaesso (999). What s Hdde Marov Models? Hdde Marov models (HMM) ca be see as a exteso of Marov models to the case where the observato s a probablstc fucto of the state,.e. the resultg model s a doubly embedded stochastc process, whch s ot ecessarly observable, but ca be observed through aother set of stochastc processes that produce the sequece of observatos. o get a better uderstadg for ths the followg example mght be useful: Example Cosder a room wth urs. Wth each ur there are a large umber of coloured balls. We assume that there s M dfferet colours total. Furthermore, assume that a ur s tally chose accordg to some probablty dstrbuto. From ths ur, a ball s chose at radom, ad ts colour s recorded as the observato. he ball s the replaced the ur from whch t was selected. A ew ur s selected accordg to a radom selecto process assocated wth the curret ur. Fgure. A -state ur ad ball model, whch llustrates the geeral case of a dscrete symbol HMM. From Raber (989).

2 he ball selecto process s repeated for the ew ur, after whch the ext ur s selected accordg to a selecto process assocated wth the secod ur, ad so forth. he etre process geerates a fte observato sequece of colours, whch we would le to model as the observable output of a HMM. We ca ow see that we have a uderlyg Marov cha, where each state correspods to the selecto of a partcular ur. hs cha s however ot observable, but ca be observed through the sequece of colours whch obvously s a probablstc fucto of the embedded Marov cha, sce a colour s chose radomly depedg o the state whch we are curretly,.e. the ur, whch we are curretly choosg the ball from. Descrpto of HMM Raber (989) suggest that a HMM ca be descrbed by the followg:., the umber of states the model. Although the states are hdde, for may practcal applcatos there s ofte some physcal sgfcace attached to the states or to sets of states of the model. I the example wth the balls ad urs above, correspods to the umber of urs. We deote the dvdual states as S { s, s2,..., s }, ad the state at tme as Z. 2. M, the umber of dstct observato symbols per state. he observato symbols correspod to the physcal output of the system beg modelled. I our example above M correspods to the umber of colours of the balls. We deote the dvdual symbols as V { v, v2,..., v M }, ad the symbol at tme as X. P, where 3. he state trasto probablty matrx { } ( Z s Z s ) P P P +,, () 4. he observato symbol probablty dstrbuto state ( ) P( X v Z s ) { } s, B b ( ), where b, M (2) 5. he tal state dstrbuto π { π } where π P ( Z s ), (3) 0 It ca be see from the above that a complete specfcato of a HMM requres specfcato of the two model parameters ( ad M ), specfcato of the observato symbols ad the specfcato of the three probablty measures P, B ad π. For coveece, we use the compact otato ( P, B ) λ,π (4) It should be oted here that the above dscusso has cosdered oly the case whe the observatos s charactersed as dscrete symbols. I prcple, ths s however ot ecessary. he symbols or outputs ca be ether dscrete or cotuous, ad ether scalar or vectorvalued. However, all cases we eed to assume that the stochastc process { } + cha havg the property that X { X,..., } ad { Z Z } Z s a Marov 0 X Z + +, + 2,... are codtoally depedet gve Z { Z 0,..., Z }. We wll, however, from ow o assume that we have the case wth dscrete scalar symbols.

3 hree basc problems for HMMs I order for the hdde Marov models to be useful real-world applcatos Raber (989) presets three basc problems: Problem : Gve the observato sequece X ( x0, x,..., x ) λ ( P, B,π ) P X, ad a model, how do we effcetly compute ( λ), the probablty of the observato sequece, gve the model? X x x,...,, ad the model Problem 2: Gve the observato sequece ( 0, x ) λ ( P, B,π ) Z ( z0, z,..., z ) Problem 3: How do we adust the model parameters λ ( P, B,π ), how do we choose a correspodg state sequece, whch s optmal some meagful sese? to maxmse ( X λ) P? Problem ca be see as oe of scorg how well a gve model matches a gve observato sequece,.e. the soluto to ths problem would gve us a tool to choose betwee competg models. Problem 2 ca be see as the problem of ucoverg the hdde part of the model,.e. to fd the correct state sequece. Problem 3 s the oe whch we try to optmse the model parameters so as to best descrbe how a gve observato sequece comes about. he observato sequece used to adust the model parameters s called a trag sequece. he trag problem s the most crucal oe for most applcatos of HMMs. We wll ow move o to some dscusso o the mathematcal solutos of each of the three problems above. Soluto to problem : he problem s to calculate the probablty of the observato sequece gve the model λ. It s possble to do ths a straghtforward way, but ths s ufortuately computatoally ufeasble, eve for small values of ad. However, there exsts a more effcet procedure called the forward-bacward procedure. Cosder the forward varable α () defed as α () P( X, X 2,..., X, Z s λ) (5).e., the probablty of the partal observato sequece, ( X X,..., ), 2 X utl tme ad state s at tme, gve the model λ. We ca here use ducto for the problem. Frst for 0, we have Iducto leads to Sce t follows that () b ( ) α,. (6) 0 π X 0 α + ( ) α () P b ( X + ),, (7) α () P( X, X 2,..., X, Z s λ) ( X ) α () (8) P λ (9) Usg the forward varable α () we have ow solved the frst problem above. (ote that ths does ot clude ay bacward varable. he bacward varable s actually ot ecessary for the soluto ad s therefore excluded here, but t wll appear the soluto for problem 3.)

4 Soluto to problem 2: Ule, problem where a exact soluto ca be gve, there are several possble ways of solvg problem 2,.e. fdg the optmal state sequece assocated wth the gve observato sequece. he dffculty comes from the fact that there are several dfferet optmalty crtera. Oe possble optmalty crtero s to choose the states Z, whch are dvdually most lely. hs optmalty crtero maxmses the expected umber of correct dvdual states, but t does ot tae to cosderato whether the sequece of states s possble. For stace although the trasto betwee two states s mpossble.e. P 0 for some ad, they may stll be the most lely at the very stats. hs s due to the fact that the soluto of ths problem smply determes the most lely state at every stat, wthout cosderg the probablty of occurrece of sequeces of states. he most wdely used crtero s stead to fd the sgle best state sequece,.e. to maxmse P ( Z X,λ), whch s equvalet to maxmsg P ( Z, X λ). A algorthm for solvg ths problem has bee foud ad s called the Vterb algorthm. hs algorthm ca smply be see as the maxmum lelhood estmate. he algorthm ca be summarsed as follows: o fd the best state sequece, Z { Z 0, Z,..., Z }, for the gve observato X { X X,..., }, we eed to defe the quatty.e. ( ) 0, X δ ( s ) max P( Z, Z,..., Z s, X, X,..., X λ) arg 0 0 Z Z,..., Z 0, δ s s the best score (hghest probablty) alog a sgle path, at tme, whch accouts for the frst + observatos ad eds state s. By ducto we have [ P ] b ( X ) ( s ) δ ( s ) + max + (0) δ () o actually retreve the state sequece, we eed to eep trac of the argumet whch maxmsed the above the above equato, for each ad. We do ths wth the array ψ ( s ). he procedure for fdg the best state sequece ow follows as: ) Italsato: 2) Recurso: 3) ermato: ( s ) π b ( ) ( ) 0 δ, 0 X 0 0 s ( s ) maxδ ( s ) P b ( ) arg max[ δ ( s ) P ] ψ (2) [ ] ( X ) δ,, ψ,, (3) s P Z max [ δ ( s )] [ δ ( s )] arg max (4) 4) State sequece bactracg: Z ψ Z,, 2,...,,0. (5) ( ) + +

5 he best sequece accordg to the Vterb algorthm s thus foud as Z ( Z 0, Z,..., Z ). It should be oted that apart from the bactracg step the Vterb algorthm s rather smlar to the forward calculato used problem. Soluto to problem 3: he by far most dffcult of the three problems s to determe a method to adust the model parameters λ ( P, B,π ) to maxmse the probablty of the observato sequece gve the model. hs problem s fact ot possble to solve usg a fte observato sequece as trag data, but we ca choose λ ( P, B,π ) such that P ( X λ) s locally maxmsed usg a teratve procedure such as the Baum-Welch method. (Equvalet results wll be foud usg the EM method.) We start of wth troducg a bacward varable β () defed as () P( X, X 2,..., X Z s ) β + +,λ (6).e. the probablty of the partal observato sequece from + to the ed, gve the state β ductvely, as follows: s at tme ad the model λ. Aga we ca solve for ( ) ad ducto leads to β (), () P b ( X ) ( ) + β + (7) β, 2,...,, 0, (8) I order to descrbe the procedure for reestmato of HMM parameters, we also defe ξ (, ), the probablty of beg state s at tme ad state s at tme +, gve the model ad the observato sequece,.e. (, ) P( Z s, Z s X ) ξ +,λ (9) From the defto of the forward ad bacward varables t follows that we ca wrte ξ the form ( ), ξ (, ) α ( ) P b ( X ) β ( ) P + ( X λ) + α ( ) P b ( X ) β ( ) α + () P b ( X ) β ( ) where the umerator s smply P( Z s, Z + s X λ) ad the dvso by ( X λ), P gves the desred probablty measure. We also eed to defe γ ( ) as the probablty of beg state s at tme, gve the observato sequece ad the model. It follows that () ξ (, ) (20) γ (2) If we sum γ () over the tme dex up to tme we get a quatty, whch ca be terpreted as the expected umber of trastos made from state s. Smlarly, summato of ξ (, ) up to tme ca be terpreted as the expected umber of trastos from state s to state s. We ca also sum γ () over the tme dex up to tme, whch ca be

6 terpreted as the expected umber of tmes state s. Usg ths, we ca get a method for reestmato of the parameters a HMM. he reestmato formulas ca be foud as π expected frequecy (umber of tmes) state s at tme ( t ) γ (22) () P expected umber of trastos from state s to state s expected umber of trastos from state s ξ γ (, ) () (23) expected umber of tmes state s ad observg symbol v b ( ) expected umber of tmes state s γ ( ) I ( X v ) γ ( ). (24) he reestmato procedure ow rus as follows. We defe the curret model as λ ( P, B,π ), ad use that to compute the rght-had sde of the above equatos, whch s put equal to the left-had sde. he left-had sdes are the parameters the model ad ths ca be used to further mprove the model by repeatg the procedure utl a lmtg pot s reached. A Exteso of the stadard HMMs here are aturally may extesos to the smple scalar, dscrete case, whch has bee troduced here. Oe of these terestg extesos of the stadard HMMs preseted here would be to model state durato,.e. that the sequece stays a state for a o-zero amout of tme. For the stadard HMMs, t ca be show that the heret durato probablty desty p (d) assocated wth state s,.e. the probablty of d cosecutve observatos state s s of the form: p ( d) P d ( P ), (25) where P s the self-trasto coeffcet for state s. For most applcatos, ths expoetal state durato desty s approprate. Istead, t s preferable to explctly model durato desty some aalytcal form. hs meas that the HMM would ru as follows. Frst a tal state s s chose accordg to some dstrbuto π, ad the a durato d 0 s chose accordg to the state durato desty p ( d 0 ). Observatos for the observg tmes t 0,..., d 0 are chose accordg to the ot desty b Z ( X, ) 0 0 X,..., X d. Fally, the ext 0 state s chose accordg to the state trasto probabltes P, where P 0 sce we have determe the state durato to be exactly d 0. he procedure s the repeated for the secod d state ad so forth. It should be oted that for the specal case where p ( d) P ( P ), the stuato s equvalet to the stadard HMM. he formulato wth state durato desty caot be drectly appled to the soluto of the three problems descrbed above, but assumg that etre durato tervals are cluded the observato sequece t s possble to fd smlar solutos to the problems.

7 It should, however, be oted that there a umber of drawbacs wth the corporato of durato destes. Oe s the crease of computatoal load. Aother oteworthy problem s that, geeral, a larger trag data set s requred, sce fewer state trastos are made wth ths model compared to the stadard HMM. Implemetato ssues for HMMs here are a umber of detals to pay atteto to whe mplemetg the HMMs. Examples of these are scalg ssues, tal parameter estmates, ad suffcet trag data. he ssues are setched ad some deas about solutos are gve here. Scalg I order to see why scalg s of mportace whe mplemetg the reestmato procedure, cosder the defto of the forward varable α ( ). It ca be see the defto that α cossts of the sum of a large umber of terms, each of the form () s 0 P Z Z s s+ s 0 b Z s ( X ) Z s (26) where Z s. Sce each of the factors the product geerally s sgfcatly less tha t ca be see that as starts to get bg each term α ( ) starts to head expoetally to zero. hs meas that after a suffcetly log tme ay computer wll ru to problems wth precso rage. For ths reaso a scalg procedure s ecessary. he basc procedure, whch s used, s to multply α ( ) wth a scalg coeffcet depedet of, wth the goal of eepg the scaled α ( ) wth the dyamc rage for each value of.e. 0.he suggested scalg Raber (989) s to multply α ( ) wth a factor c α () (27) he scaled coeffcets are thus foud as ( ) c α ( ) ˆ α (28) A smlar scalg s doe for the bacward varables β () usg the same scalg factor,.e. ( ) c β ( ) ˆ β (29) It ca the be show that whe calculatg P due to cacellatos we get the same results whe usg ˆ α ( ) ad ˆ β ( ) stead of α ( ) ad β () respectvely. he oly really mportat chage the solutos of the problems lsted above comes the calculato of P ( X λ), sce oe caot smply sum up the ˆ α ( ) terms sce they are scaled already. However, t turs out that t s stll possble to calculate logarthm of P ( X λ). I the Vterb algorthm t turs out that o scalg s ecessary f oe uses logarthms the four steps of the algorthm. hs meas that oe wll arrve at log( P ) rather tha P, but wth less computg ad o umercal errors.

8 Ital parameter estmatos I prcpal there are o straghtforward aswer o how to choose the tal estmates of the HMM parameters. It appears as the dstrbuto of the tal dstrbuto π ad the trasto matrx P s rather sestve. (For stace, uform tal estmates ca be used.) However for the parameters B, the tal estmates are crucal, especally the cotuous case,.e. whe the observato symbols come from a cotuous dstrbuto. here are a umber of suggestos o how to obta good tal estmates, e.g. maual segmetato of the observato sequece to states wth averagg of observatos wth states, ad maxmum lelhood segmetato of observatos wth averagg. Isuffcet trag data A obvous problem wth the trag of HMM parameters, s that the observato sequece s fte. hs meas that there s ofte suffcet umbers of occurreces of the dfferet model evets to gve good parameter estmates. A atural way of solvg ths problem s to gather more data, but ths ofte mpossble practcal stuatos ad therefore t s ecessary to fd a techque, whch deals wth the data at had. Oe possble soluto s smply to reduce the sze of the model, e.g. the umber of states, umber of symbols per state, etc. However, may practcal stuatos the ature of the model s gve by a physcal stuato ad thus reducto of the model s ot possble. A thrd possblty s to terpolate oe set of parameter estmates wth aother set of parameter estmates from a model for whch a adequate amout of trag data exsts. he dea s to use the trag data to desg two models, oe correspodg to the desred oe, ad oe whch s smaller, but for whch the trag data s suffcet. he smaller model s created by teg oe or more sets of parameters of the tal model together. he fal result s obtaed by terpolato betwee the two models. A ey ssue s to uderstad how much weght should be put o the tal model ad how much o the reduced model. here are however some results o ths topc, whch ca provde a optmal weght. Applcatos ad Examples hree examples of very dfferet applcatos wll be gve here. he frst s the perhaps most classc the feld.e. speech recogto. he secod comes from the bologcal area, ad refers to prote modellg. Fally, a more theoretcal result useful fatgue aalyss wll be gve. Speech recogto Arguably, oe of the most oteworthy applcatos of HMMs s speech recogto. he example gve here s due to Raber (989) ad deals wth solated word recogto. Assume there are total V words, whch are to be recogsed ad that there are K occurreces of each spoe word. Each occurrece of the word costtutes a observato. he observatos of words are typcally represeted terms of spectra ad/or tme sgals. I order to do the solated word recogto, there are two tass that are ecessary to perform:. Frst t s ecessary to buld HMMs for each word the vocabulary,.e. for each word v, we eed to estmate the model parameters λ v ( Pv, Bv, π v ), whch optmse the lelhood of the trag set observato vectors for the word. 2. For each uow word the observato sequece s aalysed ad calculatos of model lelhoods for all possble models,.e. all possble words, are performed. Fally, the model gves the recogsed word as the oe wth the hghest model lelhood.

9 Oe of the possble ways to perform the aalyss ad obta the observato vector X s to coduct a spectral aalyss. A commo techque s the to use lear predctve codg (LPC) to extract observato vectors. Prote modellg he modellg of protes s ot as urelated to the case wth speech recogto as t frst appears. A more geeral speech recogto whe a sequece of words or phoemes s cosdered ca be see as a patter recogto tas. hs s also true for the prote modellg case, where the tas s to model a sequece of amo acds, whch buld up protes. I fact the words correspod to the 20 amo acds from whch prote molecules are costructed. he example of a hdde Marov model for protes cosdered here s due to Krogh et al. (994). he structural tuto of a prote ca be see the followg way: a) A sequece of postos, each wth ts o dstrbuto over the amo acds; b) the possblty of ether sppg a posto or sertg extra amo acds betwee cosecutve postos; ad c) allowg for the possblty that cotug a serto or deleto s more lely tha startg oe. Krogh et al. (994) costruct ther hdde Marov model to catch the propertes lsted above. he ma le of the HMM cotas a sequece of M states, whch we wll call match states, correspodg to the postos a prote or colums a multple algmet. Each of the M states ca geerate a letter x from the 20-letter amo acd alphabet accordg to the dstrbuto P ( X x Z m ),,..., M,.e. each geerated letter correspod to a specfc amo acd. he otato P ( X x Z m ) meas that each of the match states m, M, have dstct dstrbutos. I order to model the possblty of sppg the posto there s a deleto state d for each state m, whch s smply a dummy state. Fally, order to model the possblty of sertg extra amo acds there are a total of M + sert states to ether sde of the match states, whch geerate letters from the amo acd alphabet exactly the same way as the match state, but use the probablty dstrbutos P ( X x Z ), 0,,..., M. For smplcty purposes a dummy state has bee added the begg ad the ed, deoted m 0 ad m M +, whch do ot produce ay amo acds. he stuato ca be see below for the case whe M 4. Fgure. he prote model for M 4. From Krogh et al. (994). otce that the model allows for several extra amo acds sce there s a postve selftrasto probablty for the sert states. From each state, there are three possble trastos. rastos to match states or deleto states always move forward the model whereas trastos to sert states do ot. he trasto probablty from a state q to a state r P ( Z r Z q) s here deoted ( r q), whch correspods to the more famlar otato P rq.

10 A sequece from the model s geerated the followg way: Startg the dummy state m 0, choose a trasto to m, d, or 0 radomly accordg to the trasto probabltes ( m m0 ), ( d m0 ) ad ( 0 m0 ). Wheever we are a serto or matchg state a letter x correspodg to a amo acd s geerated. For stace f we are state m a amo acd s geerated accordg to the probablty dstrbuto P ( X x Z m ). If o the other had we are a deleto state o amo acd s geerated. he ext state s chose accordg to the possble trastos the curret state. he procedure cotues utl the sequece reaches the state m, whch s the dummy ed state, where o amo acd s geerated. he geerated sequece x, x2,..., xl s ow a sequece of letters correspodg to the dfferet amo acds, where the sequece has bee foud followg a path of states q 0, q,..., q, q +, where q 0 m0 ad q + mm +. Sce the deleto states does ot create ay amo acds we ca coclude that (the umber of states a path) s larger or equal to L (the legth of the sequece). If q s a match or serto state, we defe l () to be the dex the sequece x, x2,..., xl of the amo acd produced state q. he probablty of the evet that the path q 0, q,..., q, q + s tae ad the sequece x, x2,..., xl s geerated s P ( x xl, q0,..., q + model) ( m + q ) ( q q ) P( xl() q ),..., (30) ( ) where P x l () q f q s a deleto state. he probablty of ay sequece x, x2,..., xl of amo acds ca be foud as the sum over all possble paths that could produce that sequece ( x,..., xl model) P( x,..., xl, q0,..., q + model) P (3) paths q0,...,q + A way of estmatg the parameters s the model s the followg: For a gve set of trag sequeces s ( ),..., s( ), oe ca see how well a model fts them by calculatg the probablty t geerates them. hs s smply a product of terms of the form gve by the sum above, where we for each,...,, let x, x2,..., xl s( ). he result s the lelhood fucto ad maxmsg wth respect to the parameters the model leads to the best model accordg to the maxmum lelhood method. Fatgue aalyss Oe of the maor reasos for structural falure the automotve dustry s fatgue. Over the years varous methods of extractg fatgue relevat data from radom load-tme hstores have bee developed. Oe way of dealg wth ths problem s to form equvalet load cycles ad the use damage accumulato methods, such as the Palmgre-Mer rule. he method that has show best results s the raflow cycle coutg method. It has become the most commoly used coutg method egeerg. he way of costructg the cycles s based o coutg hysteress cycles for the load the stress-stra plae. A defto sutable for mathematcal aalyss s the followg, frst preseted by Rychl (987): Defto: From the :th local maxmum (value M ) oe loos at the lowest values forward ad bacward drectos betwee M ad the earest pot at whch the load exceeds M. he larger (less egatve) of those two values, deoted by m, s the raflow mmum pared wth M,.e. m s the least drop

11 before reachg the value M aga o ether sde. hus the :th raflow par m, M ad the raflow rage s H M m. s ( ) hs defto s probably best uderstood from a fgure: H M m m + m t t + t Fgure. he defto of the raflow cycle. he raflow cycle s deoted H. A vehcle s usually drve very dfferet evromet, for stace t s possble to dstct betwee drvg curves, slopes, flat straghts or performg mauoevers. hese cases wll create very dfferet sort of loads, ot the least because of the dffereces speed. Oe possble way of modellg these loads s by hdde Marov chas, where the states of the uderlyg Marov cha correspods to the partcular drvg mode. he observed load at tme s deoted X. For a more complete treatmet of ths see Johaesso (999). We ca regard the observed load sgal { } 0 { } X as a radom process wth the state space v,...,v M, such that a successve value s gve by a Marov trasto accordg to oe of r possble trasto matrces, correspodg to the dfferet drvg modes. Whch trasto Z wth possble values,..., r. matrx to choose s determed by the regme process { } 0 he regme process s assumed to be a Marov cha wth the trasto matrx P ( P ) + havg the property that χ { X 0,... X }, + { Z +, Z + 2,... } depedet gve Z { Z,..., } depedetly of the prevous ( Z Z ) P Z are codtoally. I partcular the regme trastos tae place 0 Z X values.e. P ( Z Z, χ ) P. he evoluto of the process { } 0 ( z) probabltes P( X v X v Z z,, Z ), X s descrbed by the trasto q, χ 2 ( X v X v, Z z) ( z ) ( z ) Q ( q ), z,..., r gvg the trasto matrces, P,. We ow have a specal case of the stadard HMMs where we ow that the observato process { X } s a Marov cha codtog o the hdde Marov cha { Z } { X } a swtchg Marov cha (wth Marov regme), ad call the process { }. I ths case t s commo to call the process Z the regme X tself does ot satsfy the Marov property, however t ca be show that the process. { } ot process {( )}, Z P X s a Marov cha, that s + 0 ( X, Z + ) ( v, s+ )( X, Z ) ( v, s ),...,( X 0, Z ) ( v0, s )) ( X Z ) ( v, s )( X, Z ) ( v, s ) he ot process has state space {( )}, v r z ( ) P (32), + +,, z cotag r states ad trasto matrx

12 ( ) Q Q, where Q ( z, w), ( ) r z, w ( z ) Q ad ( ) he r r matrx Q descrbes a trasto from to for { } { Z } may swtch state. For fxed we ca defe the colum vector q q ( q m ), ( ) m q m qm2... qmr q mz q, ( X > v X vm, Z z) cotag the probabltes that ( X, Z ) ( v z) Q z, w q p (33) zw X where the regme process z P q (34) ml l + m, are followed by a value X > v. Let π be the statoary dstrbuto of the ot Marov cha, wth trasto matrx Q (defed as above), the π ( π ), π ( π π 2... π r ). Fally let ~ π ( π π 2... π ). ow defe the followg submatrces of Q, A ( Q ml ), m, l ad C ( Q ml ), m, l for 2,..., ad,...,. he matrx C cotas the probabltes that the process umps from the terval [, ] to [, ] ad A that the process stays wth the terval [, ]. (For we defe A 0 ad C 0.) Furthermore,defe the colum vectors d ( q q q ) 2..., e ( q ) q.+... q, where d descrbes a drect trasto from,..., to a value above ad e a trasto from,..., to above. Wth ths otato Johaesso (999) showed the followg theorem: heorem: For fxed values ad ( 2,...,,,..., ), the raflow coutg testy for the sequece { X } s gve by ( e) ( ) ~ µ, π d CA e ~ + π d + C( I A), (35) 0 where the row vector s ~ π ( π π π ) 2... above as well as the sub-matrces A ad C., the colum vectors d ad e are as defed he raflow coutg testy ca for stace be used to calculate the expected cumulatve fatgue damage caused by a load sequece. Coclusos A short summary of the some of the theory behd the hdde Marov models have bee gve. For uderstadg purposes, the teto has bee to show relatvely smple parts of the theory. However, the examples of results ad very dfferet ds of applcatos showed here stll gve a ht of the usefuless of the hdde Marov models.

13 Refereces P. Johaesso. Raflow Aalyss of Swtchg Marov Loads. PhD thess, Lud Isttute of echology, Lud, 999. A. Krogh, M. Brow, I. S. Ma, K. Sölader, ad D. Haussler. Hdde Marov models computatoal bology: Applcatos to prote modelg. Joural of Molecular Bology, 235:50-53, 994. L. R. Raber. A tutoral o Hdde Marov Models ad selected applcatos speech recogto. Proceedgs of the IEEE, 77: , 989. I. Rychl. A ew defto of the raflow cycle coutg method, Iteratoal Joural of Fatgue, 9:9-2, 987.

Summary of the lecture in Biostatistics

Summary of the lecture in Biostatistics Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the

More information

Chapter 9 Jordan Block Matrices

Chapter 9 Jordan Block Matrices Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.

More information

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur

More information

Econometric Methods. Review of Estimation

Econometric Methods. Review of Estimation Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators

More information

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution: Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed

More information

1 Onto functions and bijections Applications to Counting

1 Onto functions and bijections Applications to Counting 1 Oto fuctos ad bectos Applcatos to Coutg Now we move o to a ew topc. Defto 1.1 (Surecto. A fucto f : A B s sad to be surectve or oto f for each b B there s some a A so that f(a B. What are examples of

More information

CHAPTER 4 RADICAL EXPRESSIONS

CHAPTER 4 RADICAL EXPRESSIONS 6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube

More information

PTAS for Bin-Packing

PTAS for Bin-Packing CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,

More information

Functions of Random Variables

Functions of Random Variables Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,

More information

1 Mixed Quantum State. 2 Density Matrix. CS Density Matrices, von Neumann Entropy 3/7/07 Spring 2007 Lecture 13. ψ = α x x. ρ = p i ψ i ψ i.

1 Mixed Quantum State. 2 Density Matrix. CS Density Matrices, von Neumann Entropy 3/7/07 Spring 2007 Lecture 13. ψ = α x x. ρ = p i ψ i ψ i. CS 94- Desty Matrces, vo Neuma Etropy 3/7/07 Sprg 007 Lecture 3 I ths lecture, we wll dscuss the bascs of quatum formato theory I partcular, we wll dscuss mxed quatum states, desty matrces, vo Neuma etropy

More information

2.28 The Wall Street Journal is probably referring to the average number of cubes used per glass measured for some population that they have chosen.

2.28 The Wall Street Journal is probably referring to the average number of cubes used per glass measured for some population that they have chosen. .5 x 54.5 a. x 7. 786 7 b. The raked observatos are: 7.4, 7.5, 7.7, 7.8, 7.9, 8.0, 8.. Sce the sample sze 7 s odd, the meda s the (+)/ 4 th raked observato, or meda 7.8 c. The cosumer would more lkely

More information

Assignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)

Assignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class) Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form

More information

Introduction to local (nonparametric) density estimation. methods

Introduction to local (nonparametric) density estimation. methods Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest

More information

Lecture 3. Sampling, sampling distributions, and parameter estimation

Lecture 3. Sampling, sampling distributions, and parameter estimation Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called

More information

CHAPTER VI Statistical Analysis of Experimental Data

CHAPTER VI Statistical Analysis of Experimental Data Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca

More information

Point Estimation: definition of estimators

Point Estimation: definition of estimators Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.

More information

The number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter

The number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s

More information

A Markov Chain Competition Model

A Markov Chain Competition Model Academc Forum 3 5-6 A Marov Cha Competto Model Mchael Lloyd, Ph.D. Mathematcs ad Computer Scece Abstract A brth ad death cha for two or more speces s examed aalytcally ad umercally. Descrpto of the Model

More information

Bayes (Naïve or not) Classifiers: Generative Approach

Bayes (Naïve or not) Classifiers: Generative Approach Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg

More information

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall

More information

UNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS

UNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Numercal Computg -I UNIT SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Structure Page Nos..0 Itroducto 6. Objectves 7. Ital Approxmato to a Root 7. Bsecto Method 8.. Error Aalyss 9.4 Regula Fals Method

More information

Lecture 9: Tolerant Testing

Lecture 9: Tolerant Testing Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have

More information

Multiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades

Multiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades STAT 101 Dr. Kar Lock Morga 11/20/12 Exam 2 Grades Multple Regresso SECTIONS 9.2, 10.1, 10.2 Multple explaatory varables (10.1) Parttog varablty R 2, ANOVA (9.2) Codtos resdual plot (10.2) Trasformatos

More information

Analysis of System Performance IN2072 Chapter 5 Analysis of Non Markov Systems

Analysis of System Performance IN2072 Chapter 5 Analysis of Non Markov Systems Char for Network Archtectures ad Servces Prof. Carle Departmet of Computer Scece U Müche Aalyss of System Performace IN2072 Chapter 5 Aalyss of No Markov Systems Dr. Alexader Kle Prof. Dr.-Ig. Georg Carle

More information

Module 7: Probability and Statistics

Module 7: Probability and Statistics Lecture 4: Goodess of ft tests. Itroducto Module 7: Probablty ad Statstcs I the prevous two lectures, the cocepts, steps ad applcatos of Hypotheses testg were dscussed. Hypotheses testg may be used to

More information

To use adaptive cluster sampling we must first make some definitions of the sampling universe:

To use adaptive cluster sampling we must first make some definitions of the sampling universe: 8.3 ADAPTIVE SAMPLING Most of the methods dscussed samplg theory are lmted to samplg desgs hch the selecto of the samples ca be doe before the survey, so that oe of the decsos about samplg deped ay ay

More information

TESTS BASED ON MAXIMUM LIKELIHOOD

TESTS BASED ON MAXIMUM LIKELIHOOD ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal

More information

Unsupervised Learning and Other Neural Networks

Unsupervised Learning and Other Neural Networks CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all

More information

Chapter 14 Logistic Regression Models

Chapter 14 Logistic Regression Models Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as

More information

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632

More information

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:

More information

MEASURES OF DISPERSION

MEASURES OF DISPERSION MEASURES OF DISPERSION Measure of Cetral Tedecy: Measures of Cetral Tedecy ad Dsperso ) Mathematcal Average: a) Arthmetc mea (A.M.) b) Geometrc mea (G.M.) c) Harmoc mea (H.M.) ) Averages of Posto: a) Meda

More information

CS286.2 Lecture 4: Dinur s Proof of the PCP Theorem

CS286.2 Lecture 4: Dinur s Proof of the PCP Theorem CS86. Lecture 4: Dur s Proof of the PCP Theorem Scrbe: Thom Bohdaowcz Prevously, we have prove a weak verso of the PCP theorem: NP PCP 1,1/ (r = poly, q = O(1)). Wth ths result we have the desred costat

More information

Analysis of Variance with Weibull Data

Analysis of Variance with Weibull Data Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad

More information

A tighter lower bound on the circuit size of the hardest Boolean functions

A tighter lower bound on the circuit size of the hardest Boolean functions Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the

More information

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted

More information

Unimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods

Unimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods Malaysa Umodalty Joural Tests of Mathematcal for Global Optmzato Sceces (): of 05 Sgle - 5 Varable (007) Fuctos Usg Statstcal Methods Umodalty Tests for Global Optmzato of Sgle Varable Fuctos Usg Statstcal

More information

ESS Line Fitting

ESS Line Fitting ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here

More information

L5 Polynomial / Spline Curves

L5 Polynomial / Spline Curves L5 Polyomal / Sple Curves Cotets Coc sectos Polyomal Curves Hermte Curves Bezer Curves B-Sples No-Uform Ratoal B-Sples (NURBS) Mapulato ad Represetato of Curves Types of Curve Equatos Implct: Descrbe a

More information

Statistics Descriptive and Inferential Statistics. Instructor: Daisuke Nagakura

Statistics Descriptive and Inferential Statistics. Instructor: Daisuke Nagakura Statstcs Descrptve ad Iferetal Statstcs Istructor: Dasuke Nagakura (agakura@z7.keo.jp) 1 Today s topc Today, I talk about two categores of statstcal aalyses, descrptve statstcs ad feretal statstcs, ad

More information

Outline. Point Pattern Analysis Part I. Revisit IRP/CSR

Outline. Point Pattern Analysis Part I. Revisit IRP/CSR Pot Patter Aalyss Part I Outle Revst IRP/CSR, frst- ad secod order effects What s pot patter aalyss (PPA)? Desty-based pot patter measures Dstace-based pot patter measures Revst IRP/CSR Equal probablty:

More information

Chapter 5 Properties of a Random Sample

Chapter 5 Properties of a Random Sample Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample

More information

best estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best

best estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg

More information

Homework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015

Homework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015 Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts

More information

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information Malaysa Joural of Mathematcal Sceces (): 97- (9) Bayes Estmator for Expoetal Dstrbuto wth Exteso of Jeffery Pror Iformato Hadeel Salm Al-Kutub ad Noor Akma Ibrahm Isttute for Mathematcal Research, Uverst

More information

Chapter 3 Sampling For Proportions and Percentages

Chapter 3 Sampling For Proportions and Percentages Chapter 3 Samplg For Proportos ad Percetages I may stuatos, the characterstc uder study o whch the observatos are collected are qualtatve ature For example, the resposes of customers may marketg surveys

More information

Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)

Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture) CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.

More information

ENGI 3423 Simple Linear Regression Page 12-01

ENGI 3423 Simple Linear Regression Page 12-01 ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable

More information

Multiple Linear Regression Analysis

Multiple Linear Regression Analysis LINEA EGESSION ANALYSIS MODULE III Lecture - 4 Multple Lear egresso Aalyss Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Cofdece terval estmato The cofdece tervals multple

More information

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package

More information

Analysis of Lagrange Interpolation Formula

Analysis of Lagrange Interpolation Formula P IJISET - Iteratoal Joural of Iovatve Scece, Egeerg & Techology, Vol. Issue, December 4. www.jset.com ISS 348 7968 Aalyss of Lagrage Iterpolato Formula Vjay Dahya PDepartmet of MathematcsMaharaja Surajmal

More information

Descriptive Statistics

Descriptive Statistics Page Techcal Math II Descrptve Statstcs Descrptve Statstcs Descrptve statstcs s the body of methods used to represet ad summarze sets of data. A descrpto of how a set of measuremets (for eample, people

More information

(b) By independence, the probability that the string 1011 is received correctly is

(b) By independence, the probability that the string 1011 is received correctly is Soluto to Problem 1.31. (a) Let A be the evet that a 0 s trasmtted. Usg the total probablty theorem, the desred probablty s P(A)(1 ɛ ( 0)+ 1 P(A) ) (1 ɛ 1)=p(1 ɛ 0)+(1 p)(1 ɛ 1). (b) By depedece, the probablty

More information

8.1 Hashing Algorithms

8.1 Hashing Algorithms CS787: Advaced Algorthms Scrbe: Mayak Maheshwar, Chrs Hrchs Lecturer: Shuch Chawla Topc: Hashg ad NP-Completeess Date: September 21 2007 Prevously we looked at applcatos of radomzed algorthms, ad bega

More information

Chapter 2 - Free Vibration of Multi-Degree-of-Freedom Systems - II

Chapter 2 - Free Vibration of Multi-Degree-of-Freedom Systems - II CEE49b Chapter - Free Vbrato of Mult-Degree-of-Freedom Systems - II We ca obta a approxmate soluto to the fudametal atural frequecy through a approxmate formula developed usg eergy prcples by Lord Raylegh

More information

Dimensionality Reduction and Learning

Dimensionality Reduction and Learning CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that

More information

The Mathematical Appendix

The Mathematical Appendix The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.

More information

Investigating Cellular Automata

Investigating Cellular Automata Researcher: Taylor Dupuy Advsor: Aaro Wootto Semester: Fall 4 Ivestgatg Cellular Automata A Overvew of Cellular Automata: Cellular Automata are smple computer programs that geerate rows of black ad whte

More information

Multiple Choice Test. Chapter Adequacy of Models for Regression

Multiple Choice Test. Chapter Adequacy of Models for Regression Multple Choce Test Chapter 06.0 Adequac of Models for Regresso. For a lear regresso model to be cosdered adequate, the percetage of scaled resduals that eed to be the rage [-,] s greater tha or equal to

More information

1. A real number x is represented approximately by , and we are told that the relative error is 0.1 %. What is x? Note: There are two answers.

1. A real number x is represented approximately by , and we are told that the relative error is 0.1 %. What is x? Note: There are two answers. PROBLEMS A real umber s represeted appromately by 63, ad we are told that the relatve error s % What s? Note: There are two aswers Ht : Recall that % relatve error s What s the relatve error volved roudg

More information

C.11 Bang-bang Control

C.11 Bang-bang Control Itroucto to Cotrol heory Iclug Optmal Cotrol Nguye a e -.5 C. Bag-bag Cotrol. Itroucto hs chapter eals wth the cotrol wth restrctos: s boue a mght well be possble to have scotutes. o llustrate some of

More information

Simple Linear Regression

Simple Linear Regression Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato

More information

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions.

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions. Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos

More information

1. BLAST (Karlin Altschul) Statistics

1. BLAST (Karlin Altschul) Statistics Parwse seuece algmet global ad local Multple seuece algmet Substtuto matrces Database searchg global local BLAST Seuece statstcs Evolutoary tree recostructo Gee Fdg Prote structure predcto RNA structure

More information

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971)) art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the

More information

Statistics MINITAB - Lab 5

Statistics MINITAB - Lab 5 Statstcs 10010 MINITAB - Lab 5 PART I: The Correlato Coeffcet Qute ofte statstcs we are preseted wth data that suggests that a lear relatoshp exsts betwee two varables. For example the plot below s of

More information

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ  1 STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ

More information

Third handout: On the Gini Index

Third handout: On the Gini Index Thrd hadout: O the dex Corrado, a tala statstca, proposed (, 9, 96) to measure absolute equalt va the mea dfferece whch s defed as ( / ) where refers to the total umber of dvduals socet. Assume that. The

More information

MA/CSSE 473 Day 27. Dynamic programming

MA/CSSE 473 Day 27. Dynamic programming MA/CSSE 473 Day 7 Dyamc Programmg Bomal Coeffcets Warshall's algorthm (Optmal BSTs) Studet questos? Dyamc programmg Used for problems wth recursve solutos ad overlappg subproblems Typcally, we save (memoze)

More information

Non-uniform Turán-type problems

Non-uniform Turán-type problems Joural of Combatoral Theory, Seres A 111 2005 106 110 wwwelsevercomlocatecta No-uform Turá-type problems DhruvMubay 1, Y Zhao 2 Departmet of Mathematcs, Statstcs, ad Computer Scece, Uversty of Illos at

More information

Lecture Notes Types of economic variables

Lecture Notes Types of economic variables Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte

More information

Lecture 07: Poles and Zeros

Lecture 07: Poles and Zeros Lecture 07: Poles ad Zeros Defto of poles ad zeros The trasfer fucto provdes a bass for determg mportat system respose characterstcs wthout solvg the complete dfferetal equato. As defed, the trasfer fucto

More information

1 Lyapunov Stability Theory

1 Lyapunov Stability Theory Lyapuov Stablty heory I ths secto we cosder proofs of stablty of equlbra of autoomous systems. hs s stadard theory for olear systems, ad oe of the most mportat tools the aalyss of olear systems. It may

More information

Lecture 1 Review of Fundamental Statistical Concepts

Lecture 1 Review of Fundamental Statistical Concepts Lecture Revew of Fudametal Statstcal Cocepts Measures of Cetral Tedecy ad Dsperso A word about otato for ths class: Idvduals a populato are desgated, where the dex rages from to N, ad N s the total umber

More information

MATH 247/Winter Notes on the adjoint and on normal operators.

MATH 247/Winter Notes on the adjoint and on normal operators. MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say

More information

Lecture 3 Probability review (cont d)

Lecture 3 Probability review (cont d) STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto

More information

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The

More information

18.413: Error Correcting Codes Lab March 2, Lecture 8

18.413: Error Correcting Codes Lab March 2, Lecture 8 18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse

More information

6.867 Machine Learning

6.867 Machine Learning 6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though

More information

A Study of the Reproducibility of Measurements with HUR Leg Extension/Curl Research Line

A Study of the Reproducibility of Measurements with HUR Leg Extension/Curl Research Line HUR Techcal Report 000--9 verso.05 / Frak Borg (borgbros@ett.f) A Study of the Reproducblty of Measuremets wth HUR Leg Eteso/Curl Research Le A mportat property of measuremets s that the results should

More information

Some Notes on the Probability Space of Statistical Surveys

Some Notes on the Probability Space of Statistical Surveys Metodološk zvezk, Vol. 7, No., 200, 7-2 ome Notes o the Probablty pace of tatstcal urveys George Petrakos Abstract Ths paper troduces a formal presetato of samplg process usg prcples ad cocepts from Probablty

More information

BIOREPS Problem Set #11 The Evolution of DNA Strands

BIOREPS Problem Set #11 The Evolution of DNA Strands BIOREPS Problem Set #11 The Evoluto of DNA Strads 1 Backgroud I the md 2000s, evolutoary bologsts studyg DNA mutato rates brds ad prmates dscovered somethg surprsg. There were a large umber of mutatos

More information

Algorithms Design & Analysis. Hash Tables

Algorithms Design & Analysis. Hash Tables Algorthms Desg & Aalyss Hash Tables Recap Lower boud Order statstcs 2 Today s topcs Drect-accessble table Hash tables Hash fuctos Uversal hashg Perfect Hashg Ope addressg 3 Symbol-table problem Symbol

More information

A New Family of Transformations for Lifetime Data

A New Family of Transformations for Lifetime Data Proceedgs of the World Cogress o Egeerg 4 Vol I, WCE 4, July - 4, 4, Lodo, U.K. A New Famly of Trasformatos for Lfetme Data Lakhaa Watthaacheewakul Abstract A famly of trasformatos s the oe of several

More information

Can we take the Mysticism Out of the Pearson Coefficient of Linear Correlation?

Can we take the Mysticism Out of the Pearson Coefficient of Linear Correlation? Ca we tae the Mstcsm Out of the Pearso Coeffcet of Lear Correlato? Itroducto As the ttle of ths tutoral dcates, our purpose s to egeder a clear uderstadg of the Pearso coeffcet of lear correlato studets

More information

For combinatorial problems we might need to generate all permutations, combinations, or subsets of a set.

For combinatorial problems we might need to generate all permutations, combinations, or subsets of a set. Addtoal Decrease ad Coquer Algorthms For combatoral problems we mght eed to geerate all permutatos, combatos, or subsets of a set. Geeratg Permutatos If we have a set f elemets: { a 1, a 2, a 3, a } the

More information

AN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET

AN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET AN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET Abstract. The Permaet versus Determat problem s the followg: Gve a matrx X of determates over a feld of characterstc dfferet from

More information

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution Global Joural of Pure ad Appled Mathematcs. ISSN 0973-768 Volume 3, Number 9 (207), pp. 55-528 Research Ida Publcatos http://www.rpublcato.com Comparg Dfferet Estmators of three Parameters for Trasmuted

More information

SPECIAL CONSIDERATIONS FOR VOLUMETRIC Z-TEST FOR PROPORTIONS

SPECIAL CONSIDERATIONS FOR VOLUMETRIC Z-TEST FOR PROPORTIONS SPECIAL CONSIDERAIONS FOR VOLUMERIC Z-ES FOR PROPORIONS Oe s stctve reacto to the questo of whether two percetages are sgfcatly dfferet from each other s to treat them as f they were proportos whch the

More information

Mean is only appropriate for interval or ratio scales, not ordinal or nominal.

Mean is only appropriate for interval or ratio scales, not ordinal or nominal. Mea Same as ordary average Sum all the data values ad dvde by the sample sze. x = ( x + x +... + x Usg summato otato, we wrte ths as x = x = x = = ) x Mea s oly approprate for terval or rato scales, ot

More information

hp calculators HP 30S Statistics Averages and Standard Deviations Average and Standard Deviation Practice Finding Averages and Standard Deviations

hp calculators HP 30S Statistics Averages and Standard Deviations Average and Standard Deviation Practice Finding Averages and Standard Deviations HP 30S Statstcs Averages ad Stadard Devatos Average ad Stadard Devato Practce Fdg Averages ad Stadard Devatos HP 30S Statstcs Averages ad Stadard Devatos Average ad stadard devato The HP 30S provdes several

More information

Multivariate Transformation of Variables and Maximum Likelihood Estimation

Multivariate Transformation of Variables and Maximum Likelihood Estimation Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty

More information

Cubic Nonpolynomial Spline Approach to the Solution of a Second Order Two-Point Boundary Value Problem

Cubic Nonpolynomial Spline Approach to the Solution of a Second Order Two-Point Boundary Value Problem Joural of Amerca Scece ;6( Cubc Nopolyomal Sple Approach to the Soluto of a Secod Order Two-Pot Boudary Value Problem W.K. Zahra, F.A. Abd El-Salam, A.A. El-Sabbagh ad Z.A. ZAk * Departmet of Egeerg athematcs

More information

Chapter 8. Inferences about More Than Two Population Central Values

Chapter 8. Inferences about More Than Two Population Central Values Chapter 8. Ifereces about More Tha Two Populato Cetral Values Case tudy: Effect of Tmg of the Treatmet of Port-We tas wth Lasers ) To vestgate whether treatmet at a youg age would yeld better results tha

More information

Random Variables and Probability Distributions

Random Variables and Probability Distributions Radom Varables ad Probablty Dstrbutos * If X : S R s a dscrete radom varable wth rage {x, x, x 3,. } the r = P (X = xr ) = * Let X : S R be a dscrete radom varable wth rage {x, x, x 3,.}.If x r P(X = x

More information

THE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5

THE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5 THE ROYAL STATISTICAL SOCIETY 06 EAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5 The Socety s provdg these solutos to assst cadtes preparg for the examatos 07. The solutos are teded as learg ads ad should

More information

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty

More information

CIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights

CIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:

More information

Logistic regression (continued)

Logistic regression (continued) STAT562 page 138 Logstc regresso (cotued) Suppose we ow cosder more complex models to descrbe the relatoshp betwee a categorcal respose varable (Y) that takes o two (2) possble outcomes ad a set of p explaatory

More information

Simulation Output Analysis

Simulation Output Analysis Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5

More information