Square Root Law for Communication with Low Probability of Detection on AWGN Channels

Size: px
Start display at page:

Download "Square Root Law for Communication with Low Probability of Detection on AWGN Channels"

Transcription

1 Square Root Law for Commucato wth Low Probablty of Detecto o AWGN Chaels Boulat A. Bash, Des Goeckel, Do Towsley Departmet of Computer Scece, Uversty of Massachusetts, Amherst, Massachusetts Electrcal ad Computer Egeerg Departmet, Uversty of Massachusetts, Amherst, Massachusetts Abstract We preset a square root lmt o low probablty of detecto (LPD) commucato over addtve whte Gaussa ose (AWGN) chaels. Specfcally, f a warde has a AWGN chael to the trasmtter wth o-zero ose power, we prove that o( ) bts ca be set from the trasmtter to the recever AWGN chael uses wth probablty of detecto by the warde less tha ɛ for ay ɛ > 0, ad, f a lower boud o the ose power o the warde s chael s kow, the O( ) bts ca be covertly set chael uses. Coversely, tryg to trasmt more tha O( ) bts ether results detecto by the warde wth probablty oe or a o-zero probablty of decodg error as. Further, we show that LPD commucato o the AWGN chael allows oe to sed a ozero symbol o every chael use, cotrast to what mght be expected from the square root law foud recetly mage-based stegaography. I. INTRODUCTION Securg formato trasmtted over wreless lks s of paramout cocer for cosumer, dustral, ad mltary applcatos. Tradtoal ecrypto ad key exchage protocols secure data from tercepto by a utrusted thrd party. However, there are may real-lfe stuatos where t s mperatve to prevet the trasmsso from beg detected the frst place, as ecrypted data arouses suspco, ad eve the most theoretcally robust ecrypto ca ofte be defeated by a determed adversary usg o-computatoal methods such as sde-chael aalyss. I spte of ts mportace, low probablty of detecto (LPD) commucato has bee relatvely uderexplored. I ths work we exame the fudametal lmtatos of LPD commucato over wreless lks. I our scearo, Alce commucates wth Bob over a chael subject to addtve whte Gaussa ose (AWGN), whle Wlle attempts to detect her trasmsso (wthout actvely jammg Alce s chael). The chael betwee Wlle ad Alce s also subject to AWGN. Alce seds low-power covert sgals to Bob that Wlle attempts to classfy as ether ose Ths research was sposored by the Natoal Scece Foudato uder grats CNS ad CNS , ad by the U.S. Army Research Laboratory ad the U.K. Mstry of Defece uder Agreemet Number W911NF The vews ad coclusos cotaed ths documet are those of the author(s) ad should ot be terpreted as represetg the offcal polces, ether expressed or mpled, of the U.S. Army Research Laboratory, the U.S. Govermet, the U.K. Mstry of Defece or the U.K. Govermet. The U.S. ad U.K. Govermets are authorzed to reproduce ad dstrbute reprts for Govermet purposes otwthstadg ay copyrght otato hereo. o hs chael from Alce or Alce s sgals to Bob. If the ose o the chael betwee Wlle ad Alce has o-zero power, Alce ca commucate wth Bob whle toleratg a certa probablty of detecto, whch she ca drve dow by trasmttg wth low eough power. Our problem s related to the problem of mperfect stegaography, but the two problems are ot the same. Stegaography cosders hdg formato by alterg the propertes of fxed-sze, fte-alphabet covertext objects (lke mages or software bary code) wth mperfect stegaography systems allowg a fxed probablty of detecto of hdde formato. Covertext ca be cosdered a type of lossless fte-alphabet chael. However, the square root law recetly foud ths evromet [1], whch states that O( ) symbols the orgal covertext of sze may safely be modfed to hde a message, s lmted ts scope. The cotuousvalued chael allows us to spread hdde formato across every symbol used the trasmsso, thus showg that a drect applcato of the stegaographc result quckly leads to cotradcto ad demostratg the dstcto betwee the two problems. I fact, our square root law ca be vewed as geeralzg the square root law for mperfect stegaography. We state our ma result that lmts mutual formato o the covert chael betwee Alce ad Bob usg asymptotc otato where f() = O(g()) deotes a asymptotcally tght upper boud o f(), ad f() = o(g()) ad f() = ω(g()) deote upper ad lower bouds, respectvely, that are ot asymptotcally tght [, Ch. 3.1]: Theorem (Square root law). Suppose the chael betwee Alce ad each of Bob ad Wlle expereces depedet addtve whte Gaussa ose (AWGN) wth power σ b > 0 ad σ w > 0, respectvely, where σ b ad σ w are costats. The, for ay ɛ > 0 ad ukow σ w, Alce ca sed o( ) formato bts to Bob chael uses whle matag a probablty of detecto of Alce s trasmsso by Wlle of less tha ɛ. Moreover, f Alce ca lower-boud σ w ˆσ w, she ca sed O( ) bts chael uses whle matag a probablty of detecto of less tha ɛ. Coversely, f Alce attempts to trasmt ω( ) bts chael uses, the, as, ether Wlle detects her wth arbtrary low probablty of error or Bob caot decode her message relably (.e. wth arbtrary low probablty of decodg error).

2 After troducg our dscrete-tme chael model ad hypothess testg backgroud Secto II, we sketch the proofs of the achevablty ad the coverse of the square root law Sectos III ad IV, respectvely. Detaled proofs ad remarks are avalable [3]. We dscuss the mappg to the cotuoustme chael ad the mplcatos of chael fadg o our results, as well as the relatoshp to prevous work Secto V, ad coclude Secto VI. Alce f 1, f,..., f z (w) II. PREREQUISITES z (b) Wlle Bob decode f 1, f,..., f decde z (w) 1, z (w),..., z (w) or f 1 + z (w) 1, f + z (w),..., f + z (w)? Fg. 1. System framework: Alce ecodes formato to a vector of real symbols f = {f } =1 ad trasmts t o a AWGN chael to Bob, whle Wlle attempts to classfy hs vector of observatos of the chael from Alce y w as ether a AWGN vector z w = {z (w) } =1 or a vector {f +z (w) } =1 of trasmssos corrupted by AWGN. A. Chael Model We use the dscrete-tme AWGN chael model wth realvalued symbols depcted Fgure 1 (ad defer dscusso of the mappg to a cotuous-tme chael as well as a fadg chael to Secto V). Alce trasmts a vector of real-valued symbols f y b = {y (b) } =1 where y(b) ad detcally dstrbuted (..d.) z (b) observes vector y w = { } =1 wth..d. z (w) N (0, σ w). B. Hypothess Testg = {f } =1. Bob receves vector = f + z (b) wth a depedet N (0, σb ). Wlle where y(w) = f + z (w), Wlle expects vector y w to be cosstet wth hs chael ose model. He performs a statstcal hypothess test o ths vector, wth the ull hypothess H 0 beg that Alce s ot covertly commucatg. Ths correspods to each sample N (0, σw)..d. The alterate hypothess H 1 s that Alce s trasmttg, whch correspods to samples from a dfferet dstrbuto. Wlle tolerates some cases whe hs test correctly accuses Alce. Followg the stadard omeclature, we deote the probablty of such rejecto of H 0 whe t s true by α [4]. Wlle s test may also accept H 0 whe t s false ad mss Alce s covert trasmsso. We deote the probablty of a mss by β. The sum α + β determes the performace of a hypothess test [4]. III. ACHIEVABILITY OF SQUARE ROOT LAW I our scearo, Alce ad Bob costruct a covert commucatos system, wth all the detals kow to Wlle except for a secret key shared before commucato. Ths follows best practces securty system desg, as ts securty depeds oly o the key [5]. Sce ths work cocers the lmts of covert commucato, key sze s ot a costrat ad we defer the study of key effcecy to future work. Wlle tres to determe whether Alce trasmtted covert data gve the vector of observatos y w of hs chael from Alce. Deote the probablty dstrbuto of Wlle s chael observatos whe H 0 s true as P 0, ad whe H 1 s true as P 1. To stregthe the achevablty result, we assume that Alce s chael put dstrbuto, as well as the dstrbuto of AWGN o the chael betwee Alce ad Wlle are kow to Wlle. The P 0 ad P 1 are kow to Wlle, ad he ca costruct a optmal statstcal hypothess test that mmzes the sum of error probabltes α + β [4, Ch. 13]. The: Fact 1 (Theorem [4]). For the optmal test: α + β = 1 T V (P 0, P 1 ) where T V (P 0, P 1 ) = 1 p 0(x) p 1 (x) dx s the total varato dstace betwee P 0 ad P 1 ad p 0 (x) ad p 1 (x) are destes of P 0 ad P 1, respectvely. Ufortuately, the total varato metrc s uweldy for the products of probablty measures, whch are used the aalyss of the vectors of observatos. We thus use [6, Lemma ]: ( 1 p 0 (x) p 1 (x) dx) D(P 0 P 1 ) (1) where relatve etropy D(P 0 P 1 ) = X p 0(x) l p0(x) p 1(x) dx wth X beg the support of p 1 (x). If P s the dstrbuto of a sequece {X } =1 where each X P s..d., the D(P 0 P 1 ) = D(P 0 P 1 ). Now we are ready to prove the achevablty theorem uder a average power costrat. Theorem 1.1 (Achevablty). Let Wlle s chael be subject to AWGN wth power σ w > 0. The Alce ca mata Wlle s sum of the probabltes of detecto errors α + β 1 ɛ for ay ɛ > 0 whle covertly trasmttg o( ) bts to Bob uses of a AWGN chael f σ w s ukow ad O( ) bts chael uses f she ca lower-boud σ w ˆσ w. Proof: Costructo: Alce s chael ecoder takes put blocks of sze M bts ad ecodes them to codewords of legth at the rate of R = M/ bts/symbol. We employ radom codg argumets ad depedetly geerate R codewords {c(w k ), k = 1,,..., R } from R for messages W k, each accordg to p X (x) = =1 p X(x ), where X N (0, P f ) ad P f s defed later. The codebook s the secret key shared betwee Alce ad Bob, ad s ot revealed to Wlle. 1 However, Wlle kows how t s costructed, cludg the value of P f, ad he uses statstcal hypothess testg o chael readgs y w to decde whether Alce trasmtted. Aalyss: Cosder the case whe Alce trasmts codeword c(w k ). Suppose that Wlle employs a detector that mplemets a optmal hypothess test o hs chael readgs. 1 Wlle s lack of kowledge of the codebook s crtcal to our result, as the sparsty of the codewords mples that, f the codeword s correctly decoded by Wlle, the the trasmsso s detected.

3 Hs ull hypothess H 0 s that he observed ose o hs chael. Hs alterate hypothess H 1 s that Alce trasmtted ad he observed Alce s codeword corrupted by ose. By Fact 1, the sum of the probabltes of Wlle s detector s errors s expressed by α + β = 1 T V (P 0, P 1 ), where the total varato dstace s betwee the dstrbuto P 0 of ose readgs that Wlle expects to observe uder hs ull hypothess ad the dstrbuto P 1 of the covert codeword trasmtted by Alce corrupted by ose. Alce ca lowerboud α + β by upper-boudg T V (P 0, P 1 ) ɛ. The realzatos of ose z (w) vector z w are zero-mea..d. Gaussa radom varables wth varace σw, ad, thus, P 0 = P w where P w = N (0, σw). Sce he does ot kow the codebook, Wlle s probablty dstrbuto of the trasmtted symbols s of zero-mea..d. Gaussa radom varables wth varace P f. Sce ose s depedet of the trasmtted symbols, whe Alce trasmts, Wlle observes y w, where N (0, P f + σw) = P s s..d., ad P 1 = P s. The, usg (1), the propertes of relatve etropy, ad the Taylor seres expaso of D(P w P s ) wth respect to P f aroud P f = 0, we obta the upper boud: T V (P w, P s ) P f σw () Suppose Alce sets her average covert symbol power P f cf(), where c = ɛ. I most practcal scearos Alce ca lower-boud σw ˆσ w ad set f() = ˆσ w. If σw s ukow, select f() such that f() = o(1) ad f() = ω(1/ ) (the latter codto s used to boud Bob s decodg error probablty). I ether case, for large, P f < σw satsfes the Taylor seres covergece crtero for () to be vald, ad Alce upper-bouds T V (P w, P s ) ɛ, lmtg the performace of Wlle s detector. As stadard results for costat symbol power are ot drectly applcable to our system where P f s a fucto of codeword sze, we exame the probablty P e of Bob s decodg error averaged over all possble codebooks. Let Bob employ a maxmum-lkelhood (ML) decoder (.e. mmum dstace) to process the receved vector y b whe c(w k ) was set. The decoder suffers a error evet E (c(w k )) whe y b s closer to aother codeword c(w ), k: [ )] P e = E c(wk ) P ( R =0, ke (c(w k )) R =0, k E c(wk ) [P (E (c(w k )))] (3) where (3) follows from the uo boud ad the learty of expectato. The dstace betwee two codewords s d = c(w k ) c(w ), where s the L orm. Sce codewords are depedet ad Gaussa, c(w k ) c(w ) N (0, P f ) ad d = P f U, where U χ, wth χ deotg the ch-squared dstrbuto wth degrees of freedom. Therefore, by [7, (3.44)]: ( )] P f U E c(wk ) [P (E (c(w k )))] = E U [Q σb where Q(x) = x expectato yelds: e t / π dt 1 / e x [8, (5)]. Takg the E c(wk ) [P (E (c(w k )))] log ( ) 1+ cf() σ b The summad (3) does ot deped o, ad (3) becomes: ( ) 1+ cf() σ b P e R log Sce f() = ω(1/ ( ) ), f rate R = ρ log 1 + cf() for σb a costat ρ < 1, as creases, the probablty of Bob s decodg error decays expoetally to zero ad Bob obtas R = ( ) ρ log 1 + cf() σ covert bts chael b uses. Sce R ρcf() 4σb l, approachg equalty as gets very large, Bob receves R = o( ) bts chael uses, ad R = O( ) bts chael uses f f() = ˆσ w. Implcatos of a peak power costrat Sce most practcal systems are peak-power costraed, we show that the square root law holds for the bary put Gaussa output chael usg a proof smlar to that of Theorem 1.1. Theorem 1. (Achevablty uder a peak power costrat). Suppose Alce s trasmtter s subject to the peak power costrat b ad Wlle s chael s subject to AWGN wth power σ w > 0. The Alce ca mata Wlle s sum of the probabltes of detecto errors α + β 1 ɛ for ay ɛ > 0 whle covertly trasmttg o( ) bts to Bob over uses of a AWGN chael f σ w s ukow ad O( ) bts chael uses f she ca lower-boud σ w ˆσ w. Proof: Costructo: Alce ecodes the put blocks of sze M bts to codewords of legth at the rate R = M/ bts/symbol wth the symbols draw from alphabet { a, a}, where a satsfes the peak power costrat a < b ad s defed later. We depedetly geerate R codewords {c(w k ), k = 1,,..., R } for messages W k from { a, a} accordg to p X (x) = =1 p X(x ), where p X ( a) = p X (a) = 1. As the proof of Theorem 1.1, the codebook s a secret key shared betwee Alce ad Bob, but Wlle kows how t s costructed, cludg the value of a. Aalyss: Whe Alce trasmts a covert symbol durg the th symbol perod, she trasmts a or a equprobably by costructo ad Wlle observes the covert symbol corrupted ( by AWGN. Therefore, P s = 1 N ( a, σ w ) + N (a, σw) ). Aga, usg (1), the propertes of relatve etropy, ad the Taylor seres expaso of D(P w P s ) wth wth respect to a aroud a = 0, we obta the upper boud: T V (P w, P s ) a σ w Sce the power of Alce s covert symbol s a = P f, (4) s detcal to () ad Alce sets a cf(), where c ad f() are defed as Theorem 1.1. The, for large eough, a < σ w satsfes the Taylor seres covergece crtero, ad (4)

4 Alce obtas the upper boud T V (P w, P s ) ɛ, lmtg the performace of Wlle s detector. Lke Theorem 1.1, we caot apply stadard costatpower chael codg results. Thus, we upper-boud Bob s decodg error probablty by aalyzg a suboptmal decodg scheme. Suppose Bob uses a hard-decso devce o each receved { covert symbol y (b) } = f + z (b) va the rule ˆf = a f y (b) 0; a otherwse, ad apples a ML decoder o ts output. The effectve chael for the ecoder/decoder par s a bary symmetrc chael wth cross-over probablty p e = Q(a/σ b ) ad the probablty of the decodg error averaged over all possble codebooks s P e R (1 H(pe)) [9], where H(p) = p log p (1 p) log (1 p) s the bary etropy fucto. Extedg the aalyss [10, Secto I..1], we frst use the Taylor seres expaso of p e wth respect to a aroud a = 0 to upper-boud p e p (UB) e. Sce H(p e ) H(p (UB) e ) o the terval [0, 1 ], we perform Taylor seres expaso of H(p (UB) e ) wth respect to a aroud a = 0 to obta P e R cf() σ b +O(1) π l. As f() = ω(1/ ), f rate R = ρcf() σ bts/symbol for a costat ρ < 1, the b π l probablty of Bob s decodg error decays expoetally to zero as creases ad Bob obtas R = o( ) bts chael uses, ad R = O( ) bts chael uses f f() = ˆσ w. Relatoshp wth Square Root Law Stegaography It has recetly bee show that fte-alphabet mperfect stegaographc systems at most O( ) symbols the orgal covertext of sze may safely be modfed to hde a stegaographc message [1]. From the stegaographc perspectve, our covertext s the ose o Wlle s chael to Alce. But our result does ot obey ther coverse, as we ca modfy all symbols our covertext, hghlghtg the dfferet ature of the problem scearos. To demostrate the rchess of our scearo ad the geeralty of our square root law, we costruct a codebook where roughly τ out of of symbols are used to carry the message ad whe Alce s trasmttg a codeword, the dstrbuto of each of Wlle s observatos s P s = (1 τ)n (0, σw) + τn (0, P f + σw). Aga, usg (1), the propertes of relatve etropy, ad the Taylor seres expaso of D(P w P s ) wth respect to P f aroud P f = 0 yelds the followg boud: T V (P w, P s ) τp f σw (5) The oly dfferece (5) from () s τ the umerator. Thus, f Alce sets the product τp f = cf(), wth c ad f() as prevously defed, she lmts the performace of Wlle s detector. Ths product s the average symbol power used by Alce. It s easy to verfy that the peak power costraed scearo Alce should set product τa = cf() ad that the umber of bts that Alce ca covertly trasmt to Bob obeys the square root bouds. IV. CONVERSE Here, as the achevablty, the chael betwee Alce ad Bob s subject to AWGN of power σb. Alce s objectve s to covertly trasmt a message W k that s M = ω( ) bts log to Bob chael uses wth arbtrarly small probablty of decodg error as gets large. For a upper boud o the reducto etropy, the messages are chose equprobably. Alce ecodes each message W k arbtrarly to symbols at the rate R = M/ symbols/bt. I the coverse that follows Wlle observes all of Alce s chael uses, but, to stregthe the result, he s oblvous to her sgal propertes. Theorem. If over chael uses, Alce attempts to trasmt a covert message to Bob that s ω( ) bts log, the, as, ether Wlle ca detect her wth arbtrarly low sum of error probabltes α + β, or Bob caot decode wth arbtrarly low probablty of error. Proof: To detect Alce s covert trasmssos, Wlle performs the followg hypothess test: H 0 : H 1 : = z (w), = 1,..., = f + z (w), = 1,..., Rejecto of H 0 meas that Alce s covertly commucatg wth Bob. To perform the test, Wlle collects a vector of depedet readgs y w from hs chael to Alce ad geerates the test statstc S = ywyt w where x T deotes traspose of vector x. Uder the ull hypothess H 0 Alce does ot trasmt ad Wlle reads AWGN. Thus, N (0, σw), ad the mea ad the varace of S are: E [S] = σ w (6) var [S] = σ4 w Suppose Alce trasmts codeword c(w k ) wth the average power per symbol P k = c(w k)c T (W k ). The the mea ad varace of S whe Alce trasmts message W k are: (7) E [S] = σ w + P k (8) var [S] = 4P kσw + σw 4 (9) The varace of Wlle s test statstc (9) s computed by addg the varaces codtoed o c(w k ) of the squared dvdual observatos var [ ] y (ad dvdg by ) sce the ose o the dvdual observatos s depedet. Deote P 0 as the dstrbuto whe H 0 holds, ad P (k) 1 whe H 1 holds wth Alce trasmttg message W k. If H 0 s true, the S should be close to (6). Wlle pcks some threshold t ad compares the value of S to σw + t. He accepts H 0 f S < σw + t ad rejects t otherwse. Suppose that he desres false postve probablty α, whch s the probablty that S σw + t whe H 0 s true. We boud t usg (6) ad (7) wth Chebyshev s Iequalty [6, (3.3)]: α P 0 ( S σ w t ) σ4 w t

5 d, where d = σ w α Thus, to obta α, Wlle sets t = s a costat. As creases, t decreases, whch s cosstet wth Wlle gag greater cofdece wth more observatos. Now let s boud the probablty of a mss β gve t, whch s the probablty that S < σw + t whe Alce trasmts W k. We use Chebyshev s Iequalty wth (8) ad (9): β P (k) 1 ( S σ w P k P k t ) 4P kσ w + σ 4 w ( P k d) (10) If P k = ω(1/ ), lm β = 0. Thus, wth eough observatos, Wlle ca detect wth arbtrarly low error probablty Alce s codewords wth average symbol power P k = ω(1/ ). Note that Wlle s detector s oblvous to ay detals of Alce s codebook costructo. By (10), f Alce wats to lower-boud the sum of the probabltes of error of Wlle s statstcal test by α + β ζ > 0, she must use low-power codewords; partcular, a fracto γ > 0 of the codewords must have P U = O(1/ ). Deotg ths set of codewords by U, we ca lower-boud the probablty of Bob s decodg error P e γp (U) e, where P (U) e s the probablty of decodg error whe a message from U s set. Focusg o U, we adapt the proof of the coverse to the codg theorem for Gaussa chaels [6, Ch. 9.] to obta: P (U) e 1 P U/σb + 1/ log γ + R (11) Sce Alce trasmts ω( ) bts chael uses, her rate s R = ω(1/ ) bts/symbol. However, P U = O(1/ ), ad, as, P (U) e s bouded away from zero. Thus, P e s bouded away from zero f Alce tres to beat Wlle s smple hypothess test. V. DISCUSSION A. Mappg to Cotuous-tme Chael Cosder the mappg of the dscrete-tme model employed throughout ths paper to the physcal (cotuous-tme) chael. For a system that has a (basebad) badwdth costrat of W Hz, f Alce employs the optmal badlmted pulse shape sc(w t), all of the formato s extracted from the chael by Wlle (ad Bob) by samplg at rate W samples/secod. Ths results the dscrete-tme model of Secto II, ad the results preseted here apply drectly. Whe the pulse shape s stead chose to have some excess badwdth, the samplg at a rate hgher tha W has utlty for Wlle whe attemptg to detect Alce s trasmsso. Although we fd t ulkely that the asymptotcs cosdered here wll be altered, techques volvg cyclostatoary detecto are applcable ad could potetally mpact practcal system mplemetatos. B. Fadg ad Shadowg Fadg ad shadowg wll mpact both the capacty of the chael from Alce to Bob ad the ablty for Wlle to detect Alce s trasmsso. There are a umber of dfferet models that could be employed to corporate these effects. However, whle these models wll have a mpact as we move toward practcal systems, they have lttle mpact o the asymptotc results preseted here. C. Relatoshp to Prevous Work Aalytcal evaluato of LPD commucato has bee sparse. Hero studes LPD chaels [11] a multple-put multple-output (MIMO) settg. Whle he recogzes that a LPD commucato system s costraed by average power, he does ot aalyze the costrat asymptotcally ad, thus, does ot obta the square root law. Ulke LPD commucato, much aalytcal work has bee doe o stegaography. As oted the remark Secto III, the square root law was foud fte-alphabet mperfect stegaography [1]. However, although ther goal s the same as ours (hdg formato wth low probablty of detecto by Wlle), ther model based o hdg formato fte-alphabet mages s very dfferet from ours. Our scearo s arguably rcher, ad ts addtoal degree of freedom the choce of trasmsso power allows Alce to alter all symbols used trasmsso whle matag a fxed detecto probablty, whch stads cotrast to the fte-alphabet stegaography result. VI. CONCLUSION We proved that the LPD commucato s subject to a square root law that the umber of bts that ca be covertly trasmtted chael uses s O( ). A terestg result our work s the fact that oe ca use all of the symbols wth postve power to trasmt the covert messages. A promsg drecto of future research s the exteso of ths work to a practcal etworked settg. Evetually, we would lke to aswer ths fudametal questo: s t possble to establsh ad mata a shadow wreless etwork the presece of both actve ad passve wardes? REFERENCES [1] T. Fller, A. D. Ker, ad J. Frdrch, The square root law of stegaographc capacty for markov covers, Meda Forescs ad Securty, vol. 754, o. 1, 009. [] T. H. Corme, C. E. Leserso, R. L. Rvest, ad C. Ste, Itroducto to Algorthms, d ed. Cambrdge, Massachusetts: MIT Press, 001. [3] B. A. Bash, D. Goeckel, ad D. Towlsey, Square root law for commucato wth low probablty of detecto o awg chaels, Uversty of Massachusetts, Tech. Rep. UM-CS , submsso to IEEE Trasactos o Wreless Commucatos. [4] E. Lehma ad J. Romao, Testg Statstcal Hypotheses, 3rd ed. New York: Sprger, 005. [5] A. J. Meezes, S. A. Vastoe, ad P. C. V. Oorschot, Hadbook of Appled Cryptography, 1st ed. Boca Rato, FL, USA: CRC Press, Ic., [6] T. M. Cover ad J. A. Thomas, Elemets of Iformato Theory, d ed. Joh Wley & Sos, Hoboke, NJ, 00. [7] U. Madhow, Fudametals of Dgtal Commucato. Cambrdge, UK: Cambrdge Uversty Press, 008. [8] M. Cha, D. Dardar, ad M. K. Smo, New expoetal bouds ad approxmatos for the computato of error probablty fadg chaels, IEEE Trasactos o Wreless Commucatos, vol., o. 4, pp , Jul [9] A. Barg ad G. D. Forey, Jr., Radom codes: mmum dstaces ad error expoets, IEEE Trasactos o Iformato Theory, vol. 48, o. 9, pp , Sep. 00. [10] E. E. Maja, A model for the study of very osy chaels, ad applcatos, Ph.D. dssertato, Calfora Isttute of Techology, [11] A. O. Hero, Secure space-tme commucato, IEEE Trasactos o Iformato Theory, vol. 49, o. 1, pp , Dec. 003.

Summary of the lecture in Biostatistics

Summary of the lecture in Biostatistics Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the

More information

Functions of Random Variables

Functions of Random Variables Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,

More information

Chapter 5 Properties of a Random Sample

Chapter 5 Properties of a Random Sample Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample

More information

CHAPTER VI Statistical Analysis of Experimental Data

CHAPTER VI Statistical Analysis of Experimental Data Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca

More information

best estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best

best estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg

More information

Special Instructions / Useful Data

Special Instructions / Useful Data JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth

More information

Simulation Output Analysis

Simulation Output Analysis Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5

More information

Lecture Notes Types of economic variables

Lecture Notes Types of economic variables Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte

More information

Lecture 3 Probability review (cont d)

Lecture 3 Probability review (cont d) STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto

More information

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632

More information

Econometric Methods. Review of Estimation

Econometric Methods. Review of Estimation Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators

More information

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:

More information

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package

More information

Lecture 9: Tolerant Testing

Lecture 9: Tolerant Testing Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have

More information

CS286.2 Lecture 4: Dinur s Proof of the PCP Theorem

CS286.2 Lecture 4: Dinur s Proof of the PCP Theorem CS86. Lecture 4: Dur s Proof of the PCP Theorem Scrbe: Thom Bohdaowcz Prevously, we have prove a weak verso of the PCP theorem: NP PCP 1,1/ (r = poly, q = O(1)). Wth ths result we have the desred costat

More information

Class 13,14 June 17, 19, 2015

Class 13,14 June 17, 19, 2015 Class 3,4 Jue 7, 9, 05 Pla for Class3,4:. Samplg dstrbuto of sample mea. The Cetral Lmt Theorem (CLT). Cofdece terval for ukow mea.. Samplg Dstrbuto for Sample mea. Methods used are based o CLT ( Cetral

More information

A tighter lower bound on the circuit size of the hardest Boolean functions

A tighter lower bound on the circuit size of the hardest Boolean functions Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the

More information

Lecture 3. Sampling, sampling distributions, and parameter estimation

Lecture 3. Sampling, sampling distributions, and parameter estimation Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called

More information

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur

More information

Parameter, Statistic and Random Samples

Parameter, Statistic and Random Samples Parameter, Statstc ad Radom Samples A parameter s a umber that descrbes the populato. It s a fxed umber, but practce we do ot kow ts value. A statstc s a fucto of the sample data,.e., t s a quatty whose

More information

X ε ) = 0, or equivalently, lim

X ε ) = 0, or equivalently, lim Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece

More information

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The

More information

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted

More information

CIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights

CIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:

More information

Bounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy

Bounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy Bouds o the expected etropy ad KL-dvergece of sampled multomal dstrbutos Brado C. Roy bcroy@meda.mt.edu Orgal: May 18, 2011 Revsed: Jue 6, 2011 Abstract Iformato theoretc quattes calculated from a sampled

More information

THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA

THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA THE ROYAL STATISTICAL SOCIETY EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA PAPER II STATISTICAL THEORY & METHODS The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for

More information

Introduction to local (nonparametric) density estimation. methods

Introduction to local (nonparametric) density estimation. methods Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest

More information

Chapter 8. Inferences about More Than Two Population Central Values

Chapter 8. Inferences about More Than Two Population Central Values Chapter 8. Ifereces about More Tha Two Populato Cetral Values Case tudy: Effect of Tmg of the Treatmet of Port-We tas wth Lasers ) To vestgate whether treatmet at a youg age would yeld better results tha

More information

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution: Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed

More information

Chapter 8: Statistical Analysis of Simulated Data

Chapter 8: Statistical Analysis of Simulated Data Marquette Uversty MSCS600 Chapter 8: Statstcal Aalyss of Smulated Data Dael B. Rowe, Ph.D. Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 08 by Marquette Uversty MSCS600 Ageda 8. The Sample

More information

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971)) art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the

More information

22 Nonparametric Methods.

22 Nonparametric Methods. 22 oparametrc Methods. I parametrc models oe assumes apror that the dstrbutos have a specfc form wth oe or more ukow parameters ad oe tres to fd the best or atleast reasoably effcet procedures that aswer

More information

ECE 559: Wireless Communication Project Report Diversity Multiplexing Tradeoff in MIMO Channels with partial CSIT. Hoa Pham

ECE 559: Wireless Communication Project Report Diversity Multiplexing Tradeoff in MIMO Channels with partial CSIT. Hoa Pham ECE 559: Wreless Commucato Project Report Dversty Multplexg Tradeoff MIMO Chaels wth partal CSIT Hoa Pham. Summary I ths project, I have studed the performace ga of MIMO systems. There are two types of

More information

Wireless Link Properties

Wireless Link Properties Opportustc Ecrypto for Robust Wreless Securty R. Chadramoul ( Moul ) moul@steves.edu Multmeda System, Networkg, ad Commucatos (MSyNC) Laboratory, Departmet of Electrcal ad Computer Egeerg, Steves Isttute

More information

Chapter 13, Part A Analysis of Variance and Experimental Design. Introduction to Analysis of Variance. Introduction to Analysis of Variance

Chapter 13, Part A Analysis of Variance and Experimental Design. Introduction to Analysis of Variance. Introduction to Analysis of Variance Chapter, Part A Aalyss of Varace ad Epermetal Desg Itroducto to Aalyss of Varace Aalyss of Varace: Testg for the Equalty of Populato Meas Multple Comparso Procedures Itroducto to Aalyss of Varace Aalyss

More information

Chapter 14 Logistic Regression Models

Chapter 14 Logistic Regression Models Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as

More information

The Mathematical Appendix

The Mathematical Appendix The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.

More information

Dimensionality Reduction and Learning

Dimensionality Reduction and Learning CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that

More information

Point Estimation: definition of estimators

Point Estimation: definition of estimators Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.

More information

A New Family of Transformations for Lifetime Data

A New Family of Transformations for Lifetime Data Proceedgs of the World Cogress o Egeerg 4 Vol I, WCE 4, July - 4, 4, Lodo, U.K. A New Famly of Trasformatos for Lfetme Data Lakhaa Watthaacheewakul Abstract A famly of trasformatos s the oe of several

More information

Continuous Distributions

Continuous Distributions 7//3 Cotuous Dstrbutos Radom Varables of the Cotuous Type Desty Curve Percet Desty fucto, f (x) A smooth curve that ft the dstrbuto 3 4 5 6 7 8 9 Test scores Desty Curve Percet Probablty Desty Fucto, f

More information

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch

More information

Evaluation of uncertainty in measurements

Evaluation of uncertainty in measurements Evaluato of ucertaty measuremets Laboratory of Physcs I Faculty of Physcs Warsaw Uversty of Techology Warszawa, 05 Itroducto The am of the measuremet s to determe the measured value. Thus, the measuremet

More information

Bayes (Naïve or not) Classifiers: Generative Approach

Bayes (Naïve or not) Classifiers: Generative Approach Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg

More information

Analysis of System Performance IN2072 Chapter 5 Analysis of Non Markov Systems

Analysis of System Performance IN2072 Chapter 5 Analysis of Non Markov Systems Char for Network Archtectures ad Servces Prof. Carle Departmet of Computer Scece U Müche Aalyss of System Performace IN2072 Chapter 5 Aalyss of No Markov Systems Dr. Alexader Kle Prof. Dr.-Ig. Georg Carle

More information

Lecture 1 Review of Fundamental Statistical Concepts

Lecture 1 Review of Fundamental Statistical Concepts Lecture Revew of Fudametal Statstcal Cocepts Measures of Cetral Tedecy ad Dsperso A word about otato for ths class: Idvduals a populato are desgated, where the dex rages from to N, ad N s the total umber

More information

Analysis of Variance with Weibull Data

Analysis of Variance with Weibull Data Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad

More information

Mean is only appropriate for interval or ratio scales, not ordinal or nominal.

Mean is only appropriate for interval or ratio scales, not ordinal or nominal. Mea Same as ordary average Sum all the data values ad dvde by the sample sze. x = ( x + x +... + x Usg summato otato, we wrte ths as x = x = x = = ) x Mea s oly approprate for terval or rato scales, ot

More information

The Occupancy and Coupon Collector problems

The Occupancy and Coupon Collector problems Chapter 4 The Occupacy ad Coupo Collector problems By Sarel Har-Peled, Jauary 9, 08 4 Prelmares [ Defto 4 Varace ad Stadard Devato For a radom varable X, let V E [ X [ µ X deote the varace of X, where

More information

Qualifying Exam Statistical Theory Problem Solutions August 2005

Qualifying Exam Statistical Theory Problem Solutions August 2005 Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),

More information

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions.

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions. Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos

More information

Median as a Weighted Arithmetic Mean of All Sample Observations

Median as a Weighted Arithmetic Mean of All Sample Observations Meda as a Weghted Arthmetc Mea of All Sample Observatos SK Mshra Dept. of Ecoomcs NEHU, Shllog (Ida). Itroducto: Iumerably may textbooks Statstcs explctly meto that oe of the weakesses (or propertes) of

More information

Beam Warming Second-Order Upwind Method

Beam Warming Second-Order Upwind Method Beam Warmg Secod-Order Upwd Method Petr Valeta Jauary 6, 015 Ths documet s a part of the assessmet work for the subject 1DRP Dfferetal Equatos o Computer lectured o FNSPE CTU Prague. Abstract Ths documet

More information

MEASURES OF DISPERSION

MEASURES OF DISPERSION MEASURES OF DISPERSION Measure of Cetral Tedecy: Measures of Cetral Tedecy ad Dsperso ) Mathematcal Average: a) Arthmetc mea (A.M.) b) Geometrc mea (G.M.) c) Harmoc mea (H.M.) ) Averages of Posto: a) Meda

More information

PTAS for Bin-Packing

PTAS for Bin-Packing CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,

More information

Module 7: Probability and Statistics

Module 7: Probability and Statistics Lecture 4: Goodess of ft tests. Itroducto Module 7: Probablty ad Statstcs I the prevous two lectures, the cocepts, steps ad applcatos of Hypotheses testg were dscussed. Hypotheses testg may be used to

More information

Channel Polarization and Polar Codes; Capacity Achieving

Channel Polarization and Polar Codes; Capacity Achieving Chael Polarzato ad Polar Codes; Capacty chevg Peyma Hesam Tutoral of Iformato Theory Course Uversty of otre Dame December, 9, 009 bstract: ew proposed method for costructg codes that acheves the symmetrc

More information

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty

More information

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers

More information

Some Notes on the Probability Space of Statistical Surveys

Some Notes on the Probability Space of Statistical Surveys Metodološk zvezk, Vol. 7, No., 200, 7-2 ome Notes o the Probablty pace of tatstcal urveys George Petrakos Abstract Ths paper troduces a formal presetato of samplg process usg prcples ad cocepts from Probablty

More information

Midterm Exam 1, section 2 (Solution) Thursday, February hour, 15 minutes

Midterm Exam 1, section 2 (Solution) Thursday, February hour, 15 minutes coometrcs, CON Sa Fracsco State Uverst Mchael Bar Sprg 5 Mdterm xam, secto Soluto Thursda, Februar 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes exam.. No calculators of a kd are allowed..

More information

STATISTICAL INFERENCE

STATISTICAL INFERENCE (STATISTICS) STATISTICAL INFERENCE COMPLEMENTARY COURSE B.Sc. MATHEMATICS III SEMESTER ( Admsso) UNIVERSITY OF CALICUT SCHOOL OF DISTANCE EDUCATION CALICUT UNIVERSITY P.O., MALAPPURAM, KERALA, INDIA -

More information

On generalized fuzzy mean code word lengths. Department of Mathematics, Jaypee University of Engineering and Technology, Guna, Madhya Pradesh, India

On generalized fuzzy mean code word lengths. Department of Mathematics, Jaypee University of Engineering and Technology, Guna, Madhya Pradesh, India merca Joural of ppled Mathematcs 04; (4): 7-34 Publshed ole ugust 30, 04 (http://www.scecepublshggroup.com//aam) do: 0.648/.aam.04004.3 ISSN: 330-0043 (Prt); ISSN: 330-006X (Ole) O geeralzed fuzzy mea

More information

Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)

Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture) CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.

More information

Lecture 02: Bounding tail distributions of a random variable

Lecture 02: Bounding tail distributions of a random variable CSCI-B609: A Theorst s Toolkt, Fall 206 Aug 25 Lecture 02: Boudg tal dstrbutos of a radom varable Lecturer: Yua Zhou Scrbe: Yua Xe & Yua Zhou Let us cosder the ubased co flps aga. I.e. let the outcome

More information

Multivariate Transformation of Variables and Maximum Likelihood Estimation

Multivariate Transformation of Variables and Maximum Likelihood Estimation Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty

More information

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution Global Joural of Pure ad Appled Mathematcs. ISSN 0973-768 Volume 3, Number 9 (207), pp. 55-528 Research Ida Publcatos http://www.rpublcato.com Comparg Dfferet Estmators of three Parameters for Trasmuted

More information

f f... f 1 n n (ii) Median : It is the value of the middle-most observation(s).

f f... f 1 n n (ii) Median : It is the value of the middle-most observation(s). CHAPTER STATISTICS Pots to Remember :. Facts or fgures, collected wth a defte pupose, are called Data.. Statstcs s the area of study dealg wth the collecto, presetato, aalyss ad terpretato of data.. The

More information

3. Basic Concepts: Consequences and Properties

3. Basic Concepts: Consequences and Properties : 3. Basc Cocepts: Cosequeces ad Propertes Markku Jutt Overvew More advaced cosequeces ad propertes of the basc cocepts troduced the prevous lecture are derved. Source The materal s maly based o Sectos.6.8

More information

TESTS BASED ON MAXIMUM LIKELIHOOD

TESTS BASED ON MAXIMUM LIKELIHOOD ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal

More information

STK4011 and STK9011 Autumn 2016

STK4011 and STK9011 Autumn 2016 STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto

More information

Basics of Information Theory: Markku Juntti. Basic concepts and tools 1 Introduction 2 Entropy, relative entropy and mutual information

Basics of Information Theory: Markku Juntti. Basic concepts and tools 1 Introduction 2 Entropy, relative entropy and mutual information : Maru Jutt Overvew he propertes of adlmted Gaussa chaels are further studed, parallel Gaussa chaels ad Gaussa chaels wth feedac are solved. Source he materal s maly ased o Sectos.4.6 of the course oo

More information

Lecture 2 - What are component and system reliability and how it can be improved?

Lecture 2 - What are component and system reliability and how it can be improved? Lecture 2 - What are compoet ad system relablty ad how t ca be mproved? Relablty s a measure of the qualty of the product over the log ru. The cocept of relablty s a exteded tme perod over whch the expected

More information

ECE 729 Introduction to Channel Coding

ECE 729 Introduction to Channel Coding chaelcodg.tex May 4, 2006 ECE 729 Itroducto to Chael Codg Cotets Fudametal Cocepts ad Techques. Chaels.....................2 Ecoders.....................2. Code Rates............... 2.3 Decoders....................

More information

ECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity

ECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data

More information

CHAPTER 3 POSTERIOR DISTRIBUTIONS

CHAPTER 3 POSTERIOR DISTRIBUTIONS CHAPTER 3 POSTERIOR DISTRIBUTIONS If scece caot measure the degree of probablt volved, so much the worse for scece. The practcal ma wll stck to hs apprecatve methods utl t does, or wll accept the results

More information

Unimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods

Unimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods Malaysa Umodalty Joural Tests of Mathematcal for Global Optmzato Sceces (): of 05 Sgle - 5 Varable (007) Fuctos Usg Statstcal Methods Umodalty Tests for Global Optmzato of Sgle Varable Fuctos Usg Statstcal

More information

THE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE

THE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE THE ROYAL STATISTICAL SOCIETY 00 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE PAPER I STATISTICAL THEORY The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for the

More information

Simple Linear Regression

Simple Linear Regression Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato

More information

NP!= P. By Liu Ran. Table of Contents. The P vs. NP problem is a major unsolved problem in computer

NP!= P. By Liu Ran. Table of Contents. The P vs. NP problem is a major unsolved problem in computer NP!= P By Lu Ra Table of Cotets. Itroduce 2. Strategy 3. Prelmary theorem 4. Proof 5. Expla 6. Cocluso. Itroduce The P vs. NP problem s a major usolved problem computer scece. Iformally, t asks whether

More information

arxiv: v1 [math.st] 24 Oct 2016

arxiv: v1 [math.st] 24 Oct 2016 arxv:60.07554v [math.st] 24 Oct 206 Some Relatoshps ad Propertes of the Hypergeometrc Dstrbuto Peter H. Pesku, Departmet of Mathematcs ad Statstcs York Uversty, Toroto, Otaro M3J P3, Caada E-mal: pesku@pascal.math.yorku.ca

More information

Random Variables and Probability Distributions

Random Variables and Probability Distributions Radom Varables ad Probablty Dstrbutos * If X : S R s a dscrete radom varable wth rage {x, x, x 3,. } the r = P (X = xr ) = * Let X : S R be a dscrete radom varable wth rage {x, x, x 3,.}.If x r P(X = x

More information

5 Short Proofs of Simplified Stirling s Approximation

5 Short Proofs of Simplified Stirling s Approximation 5 Short Proofs of Smplfed Strlg s Approxmato Ofr Gorodetsky, drtymaths.wordpress.com Jue, 20 0 Itroducto Strlg s approxmato s the followg (somewhat surprsg) approxmato of the factoral,, usg elemetary fuctos:

More information

1. BLAST (Karlin Altschul) Statistics

1. BLAST (Karlin Altschul) Statistics Parwse seuece algmet global ad local Multple seuece algmet Substtuto matrces Database searchg global local BLAST Seuece statstcs Evolutoary tree recostructo Gee Fdg Prote structure predcto RNA structure

More information

Chapter 13 Student Lecture Notes 13-1

Chapter 13 Student Lecture Notes 13-1 Chapter 3 Studet Lecture Notes 3- Basc Busess Statstcs (9 th Edto) Chapter 3 Smple Lear Regresso 4 Pretce-Hall, Ic. Chap 3- Chapter Topcs Types of Regresso Models Determg the Smple Lear Regresso Equato

More information

This lecture and the next. Why Sorting? Sorting Algorithms so far. Why Sorting? (2) Selection Sort. Heap Sort. Heapsort

This lecture and the next. Why Sorting? Sorting Algorithms so far. Why Sorting? (2) Selection Sort. Heap Sort. Heapsort Ths lecture ad the ext Heapsort Heap data structure ad prorty queue ADT Qucksort a popular algorthm, very fast o average Why Sortg? Whe doubt, sort oe of the prcples of algorthm desg. Sortg used as a subroute

More information

Pseudo-random Functions

Pseudo-random Functions Pseudo-radom Fuctos Debdeep Mukhopadhyay IIT Kharagpur We have see the costructo of PRG (pseudo-radom geerators) beg costructed from ay oe-way fuctos. Now we shall cosder a related cocept: Pseudo-radom

More information

STA302/1001-Fall 2008 Midterm Test October 21, 2008

STA302/1001-Fall 2008 Midterm Test October 21, 2008 STA3/-Fall 8 Mdterm Test October, 8 Last Name: Frst Name: Studet Number: Erolled (Crcle oe) STA3 STA INSTRUCTIONS Tme allowed: hour 45 mutes Ads allowed: A o-programmable calculator A table of values from

More information

Lecture 4 Sep 9, 2015

Lecture 4 Sep 9, 2015 CS 388R: Radomzed Algorthms Fall 205 Prof. Erc Prce Lecture 4 Sep 9, 205 Scrbe: Xagru Huag & Chad Voegele Overvew I prevous lectures, we troduced some basc probablty, the Cheroff boud, the coupo collector

More information

Bootstrap Method for Testing of Equality of Several Coefficients of Variation

Bootstrap Method for Testing of Equality of Several Coefficients of Variation Cloud Publcatos Iteratoal Joural of Advaced Mathematcs ad Statstcs Volume, pp. -6, Artcle ID Sc- Research Artcle Ope Access Bootstrap Method for Testg of Equalty of Several Coeffcets of Varato Dr. Navee

More information

LECTURE - 4 SIMPLE RANDOM SAMPLING DR. SHALABH DEPARTMENT OF MATHEMATICS AND STATISTICS INDIAN INSTITUTE OF TECHNOLOGY KANPUR

LECTURE - 4 SIMPLE RANDOM SAMPLING DR. SHALABH DEPARTMENT OF MATHEMATICS AND STATISTICS INDIAN INSTITUTE OF TECHNOLOGY KANPUR amplg Theory MODULE II LECTURE - 4 IMPLE RADOM AMPLIG DR. HALABH DEPARTMET OF MATHEMATIC AD TATITIC IDIA ITITUTE OF TECHOLOGY KAPUR Estmato of populato mea ad populato varace Oe of the ma objectves after

More information

An Introduction to. Support Vector Machine

An Introduction to. Support Vector Machine A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork

More information

Non-uniform Turán-type problems

Non-uniform Turán-type problems Joural of Combatoral Theory, Seres A 111 2005 106 110 wwwelsevercomlocatecta No-uform Turá-type problems DhruvMubay 1, Y Zhao 2 Departmet of Mathematcs, Statstcs, ad Computer Scece, Uversty of Illos at

More information

Homework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015

Homework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015 Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts

More information

The number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter

The number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s

More information

Chapter 5. Presentation. Entropy STATISTICAL CODING

Chapter 5. Presentation. Entropy STATISTICAL CODING Chapter 5 STATISTICAL CODING Presetato Etropy Iformato data codg Iformato data codg coded represetato of formato Ijectve correspodece Message {b } Multples roles of codg Preparg the trasformato message

More information

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall

More information

STA 105-M BASIC STATISTICS (This is a multiple choice paper.)

STA 105-M BASIC STATISTICS (This is a multiple choice paper.) DCDM BUSINESS SCHOOL September Mock Eamatos STA 0-M BASIC STATISTICS (Ths s a multple choce paper.) Tme: hours 0 mutes INSTRUCTIONS TO CANDIDATES Do ot ope ths questo paper utl you have bee told to do

More information

Multiple Linear Regression Analysis

Multiple Linear Regression Analysis LINEA EGESSION ANALYSIS MODULE III Lecture - 4 Multple Lear egresso Aalyss Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Cofdece terval estmato The cofdece tervals multple

More information

ESS Line Fitting

ESS Line Fitting ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here

More information