Tornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003
|
|
- Molly Lang
- 5 years ago
- Views:
Transcription
1 Tornado and Luby Transform Codes Ashsh Khst Presentaton October 22, 2003
2 Background: Erasure Channel Elas[956] studed the Erasure Channel β x x β β x 2 m x 2 k? Capacty of Noseless Erasure Channel s No Feedback s necessary to acheve capacty A random lnear code can acheve capacty. Encodng: O(n 2 ) Decodng: O(n 3 ) Applcatons Communcaton Lnks over the Internet Storage Meda m( β )
3 Classcal MDS Codes c c 2 2 k Features Any set of k co-ordnates s an nformaton set for (n,k,d) MDS Code. The recever knows the codeword once t receves any k symbols and knows ther postons. Capacty achevng codes. c 2 k n symbols Drawbacks Reed Solomon Codes (RS) codes requre O(k 2 ) tme for decodng. Block codes : Need pror knowledge of erasure probablty.
4 Dgtal Fountan Approach A new protocol for bulk data dstrbuton Scenaro: One Server multple Recevers Encodng: Construct encodng symbols on the fly and send them when atleast one recever s lstenng. Decodng: Collect desred number of symbols from the server and reconstruct the orgnal fle Goals: Relable, Effcent, On Demand and Tolerant
5 Tornado Codes Features Correct -R(- ε) errors over BEC. Tme for encodng and decodng s proportonal to Very fast software mplementatons. n log ( ) ε Tradeoffs The assumpton of ndependent erasures s crtcal. Hgh latency. Low Rate Implementatons are less attractve. Block Codes Not sutable for heterogeneous recever populaton.
6 Irregular Bpartte Graph x x 2 c + x x2 Irregular Random Graphs are used for generatng check symbols (x, x 2, x n ) (x x n, c c nβ ) x n Input/Message Symbols c nβ Check Symbols Degree Sequences Left Degree Sequence: (λ, λ 2, λ n ) Rght Degree Sequence: (ρ, ρ 2 ρ m ) Defnton: λ k (ρ k ) s the fracton of edges that are ncdent on a left(rght) node of degree k.
7 Irregular Graphs: Example Gven: (λ, λ 2 ) (/2,/2) (ρ, ρ 2 ) (0,) Number of Edges E 4 2 π 2 3 l number of left nodes of degree r number of rght nodes of degree 2 π λ E 2 E ρ l r 3 ( 2 l, l ) (2,) r, r ) (0,2) ( 2 Random permutaton between edges, nduces a unform dstrbuton over the ensemble.
8 Constructon of Tornado Codes (n) (βn) (β 2 n) (β m+ n) B 0 B ` B m C B : Irregular Graph C : Conventonal Code Code C(B 0,B,B 2 B m,c): Each B s an rregular bpartte graph wth same degree sequences C s a conventonal rate (- β) code wth O(n 2 ) complexty. m s chosen such that Length of C: m + 0 nβ m m+ 2 nβ n nβ + β β Ths s a rate (- β) code wth encodng/decodng complexty of O(n). n
9 Lnear Tme Decodng Algorthm s s 2 s 3.. Fnd a check node c n that s connected to only one source node s k. If no such c n stop and declare error. (a) set s k c n (b) fnd all c n that are neghbors of s k and set c n c n + s k (c) Remove all edges connected to s k 2. Repeat () untl all source nodes are determned s s 2 s s s 2 s
10 What has to be solved? So far Identfed the structure of encoder as a cascade of rregular bpartte graphs. Suggested a canddate decodng algorthm whch has lnear complexty. Goal: Specfy the set of degree sequences (λ, λ 2, λ n ) and (ρ, ρ 2 ρ m ) for whch the ths smple decodng algorthm succeeds. Man Contrbuton of the Paper:. Develop mathematcal condtons on the degree sequences under whch ths decodng scheme succeeds 2. Provde explct degree sequences that acheve the capacty of BEC.
11 Condtons on Degree Sequences Defne: x ρ( ) ρ x x λ( ) λ δ: Erasure Probablty of the memorless channel Necessary Condton: If the decodng algorthm succeeds n recoverng all message symbols then ρ( δλ( x)) > x, x (0,] Approach: Compute the expected value of the fracton of edges wth degree one on the rght and requre that t s > 0 Suffcent Condton: The above condton s also suffcent f we mpose λ λ 2 0. Approach: The proof uses tools from statstcal mechancs to show that the varance n degree dstrbuton s small. x
12 Capacty Achevng Dstrbuton Fx an nteger D > 0, Let 2,3..., ) )( ( + D D H λ.2,3..., )! ( e α ρ α Average degree of left nodes Average degree of rght nodes ) log( / D E E λ λ β α ρ α α ) log( D e e Intuton: Posson dstrbuton s natural f all the edges from the left unformly choose the rght nodes. Ths dstrbuton s preserved when edges are successvely removed from the graph. Heavy tal dstrbuton produces some message nodes of hgh degrees that get decoded frst and remove many edges from the graph.
13 Capacty Achevng Dstrbuton Note that: ρ( x) e α ( x ) λ( x ) log( x) D For the above choce of ρ(x) and λ(x), t s easy to verfy that ρ( δλ( x)) > x, x (0,] whenever, δ < β +/ D Let D /ε. It follows that β(- ε) fracton of erasures can be corrected by ths rate - β code. The maxmum degree log(d), mples that the number of operatons n decodng s proportonal to nlog(/ ε).
14 Lnear Programmng Approach Fx (λ, λ 2, λ n ) and δ. The objectve s to fnd (ρ, ρ 2 ρ m ) for some fxed m. Let x /N, for,2..n. We have the followng constrants: ρ(- δ λ( x )) > -x ρ 0 and ρ() Mnmze N ( ρ( δλ( x ) + x ) The soluton for ρ(x) s feasble f the nequalty holds for all x n (0,] Once a feasble soluton has been found the optmal δ s found by a bnary search. An teratve approach s suggested that uses the dual condton δλ( ρ( y)) < y, y (0,]
15 Practcal Implementatons 640K 320K 60K 60K Tornado Z Code G 0 G G 2 Rate ½ code. Takes 640,000 packets (each 256 byte) as nput. Only three cascades have been used. G 0 and G use heavytal / posson dstrbuton as noted. G 2 cannot use a standard quadratc tme code. It s degree dstrbuton s obtaned through lnear programmng. On a 200MHz Pentum machne, the decodng operaton takes.73 seconds.
16 Issues The assumpton of ndependent erasures s crtcal n desgn of Tornado codes. So deep nterleavng and very long block lengths are necessary. Hgh latency s ncurred n encodng and decodng operatons, snce both encodng and decodng must be delayed by at least one block sze. Heavy memory usage: Decodng each block of Tornado Z requres 32 MB of RAM. Snce they are block codes, they have to be optmzed for a partcular rate. The number of encodng symbols s fxed when the nput block length and rate s fxed.
17 Luby Transform Codes Features k nput symbols can be recovered from any set of symbols wth probablty -δ. k + O( k log 2 ( k / δ )) Encodng tme: O(log(k/ δ)) per encodng symbol. Decodng tme: O(k log(k/ δ)) These codes are rate-less: the number of dstnct encodng symbols that can be generated s extremely large. Encodng symbols are generated on the fly The constructon does not make use of channel erasure probablty and hence can optmally serve heterogeneous recevers
18 Encodng of LT Codes Fx a degree dstrbuton ρ(d) To produce each encodng symbol: Generate the degree D ~ ρ(d) For each of the D edges, randomly pck one nput symbol node. Compute the XOR of all the D neghbors and assgn ths value to the encodng symbol. How does the decoder know the neghbors of an encodng symbol t receves? Ths nformaton can be explctly ncluded as an overhead n each packet. Pseudo-randomness can be exploted to duplcate the encodng process at the recever. The recever has to be gven the seed and/or keys assocated wth the process.
19 Decodng LT Codes The decodng process s vrtually the same as that of Tornado Codes. At the start release all encodng symbols of degree. Ther neghbors are now covered and form a rpple. In each subsequent step, process one message symbol from the rpple s processed. It s removed as a neghbor of the encodng symbols. If any encodng symbol now has a degree one, t s released. If ts neghbor s not already n the rpple, t gets added to the rpple. The process ends when the rpple s empty. If some message symbols reman unprocessed, ths s a decodng falure.
20 LT Analyss- (ρ()) How many encodng symbols (each of degree ) wll guarantee that all message symbol nodes wll be covered wth probablty > - δ? Ans: k log(k/ δ). The probablty that a message node s not covered s k log( k / δ ) ( ) δ / k k By usng the unon bound estmate, the desred result follows. k log(k/ δ) encodng symbols s unacceptable. Snce all edges are randomly ncdent on message nodes, k log(k/ δ) edges are requred to cover all the nodes.
21 LT Analyss-2 Suppose L nput symbols reman unprocessed durng a decodng step. Any encodng symbol s equally lkely to get released ndependent of all other encodng symbols. If an encodng symbols s released, t s equally lkely to cover any of the L symbols. Defne: q(,l) probablty that an encodng symbol of degree s released, when L nput symbols reman unprocessed. 0 ), (.., 2.., ), ( ) (, 2 2 ) ( L q k L k P C C P C L q k q k L L k otherwse
22 LT Analyss-3 Solton Dstrbuton: ρ() / k ρ( ) / ( ), 2.. k r(l) : probablty that a an encodng symbol s released ( k L)! L ( k L ( 2))! r( L) ρ( ) q(, L) k ( k )! ( k )! k Thus at each step, we expect one encodng symbol to be released The sze of the rpple at each step s one.
23 Propertes of Solton Dstrbuton At each step one encodng symbol s released. Only k encodng symbols are needed on average to retreve k nput symbols. It expected number of edges n the graph s k log(k). The deal solton dstrbuton compresses the number of encodng symbols to the mnmum possble value, keepng the number of edges n the graph mnmum. The deal solton dstrbuton does not work well n practce, snce the expected sze of rpple s one. It s extremely senstve to small varatons.
24 Robust Solton Dstrbuton Mantan the sze of Rpple to a larger value R ~ Defne the followng dstrbuton Intuton c k log( k τ() / δ ) R/k for k/r- R log(r/δ)/k for k/r 0 for I k/r+..k The value of τ() s chosen so that R encodng symbols are expected to be released ntally. Ths generates a rpple of sze R When L encodng symbols are unprocessed, the most probable symbols to be released have degree k/l.
25 Robust Solton Dstrbuton (contd.) When L R, we requre that all the unprocessed symbols be covered. Ths s ensured by choosng τ(k/r) R log(r/δ)/k. The probablty any covered symbol gets ncluded n the rpple s (L-R)/L. We need L/(L-R) releases to expect that the sze of rpple remans the same. Thus the fracton of encodng symbols of degree k/l s proportonal to: L L R ( ) k ( )( k R) ( ) + R ( )( k R) ρ ( ) +τ ( )
26 Robust Solton Dstrbuton The robust solton dstrbuton s gven by µ() (τ() + ρ())/ς, where ς (τ() + ρ()). One can show that: Average number of encodng symbols to recover message 2 symbols s k + O( k log ( k / δ )) Decodng takes tme proportonal to O(k log(k/δ)) and encodng takes tme O(log(k/δ)) per symbol The probablty that the decodng algorthm fals to recover the message symbols s less than δ. 26
27 Conclusons Tornado Codes acheve lnear tme encodng and decodng but cannot solve the heterogeneous user case LT Codes can smultaneously serve heterogeneous users, but requre O(k logk) tme. Raptor codes (2000) acheve the best of both worlds and wll be dscussed next week.
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationApplication of Nonbinary LDPC Codes for Communication over Fading Channels Using Higher Order Modulations
Applcaton of Nonbnary LDPC Codes for Communcaton over Fadng Channels Usng Hgher Order Modulatons Rong-Hu Peng and Rong-Rong Chen Department of Electrcal and Computer Engneerng Unversty of Utah Ths work
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationLecture 3: Shannon s Theorem
CSE 533: Error-Correctng Codes (Autumn 006 Lecture 3: Shannon s Theorem October 9, 006 Lecturer: Venkatesan Guruswam Scrbe: Wdad Machmouch 1 Communcaton Model The communcaton model we are usng conssts
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More informationLecture 4: Universal Hash Functions/Streaming Cont d
CSE 5: Desgn and Analyss of Algorthms I Sprng 06 Lecture 4: Unversal Hash Functons/Streamng Cont d Lecturer: Shayan Oves Gharan Aprl 6th Scrbe: Jacob Schreber Dsclamer: These notes have not been subjected
More informationP R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /
Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationComputing Correlated Equilibria in Multi-Player Games
Computng Correlated Equlbra n Mult-Player Games Chrstos H. Papadmtrou Presented by Zhanxang Huang December 7th, 2005 1 The Author Dr. Chrstos H. Papadmtrou CS professor at UC Berkley (taught at Harvard,
More informationLecture 10: May 6, 2013
TTIC/CMSC 31150 Mathematcal Toolkt Sprng 013 Madhur Tulsan Lecture 10: May 6, 013 Scrbe: Wenje Luo In today s lecture, we manly talked about random walk on graphs and ntroduce the concept of graph expander,
More informationLecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem.
Lecture 14 (03/27/18). Channels. Decodng. Prevew of the Capacty Theorem. A. Barg The concept of a communcaton channel n nformaton theory s an abstracton for transmttng dgtal (and analog) nformaton from
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More informationECE559VV Project Report
ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate
More informationOn the Multicriteria Integer Network Flow Problem
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 5, No 2 Sofa 2005 On the Multcrtera Integer Network Flow Problem Vassl Vasslev, Marana Nkolova, Maryana Vassleva Insttute of
More informationIntroduction to Algorithms
Introducton to Algorthms 6.046J/8.40J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) Our focus: effcency of
More informationprinceton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg
prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there
More informationECE 534: Elements of Information Theory. Solutions to Midterm Exam (Spring 2006)
ECE 534: Elements of Informaton Theory Solutons to Mdterm Eam (Sprng 6) Problem [ pts.] A dscrete memoryless source has an alphabet of three letters,, =,, 3, wth probabltes.4,.4, and., respectvely. (a)
More informationEstimation: Part 2. Chapter GREG estimation
Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the
More informationChapter 8 SCALAR QUANTIZATION
Outlne Chapter 8 SCALAR QUANTIZATION Yeuan-Kuen Lee [ CU, CSIE ] 8.1 Overvew 8. Introducton 8.4 Unform Quantzer 8.5 Adaptve Quantzaton 8.6 Nonunform Quantzaton 8.7 Entropy-Coded Quantzaton Ch 8 Scalar
More informationCopyright 2017 by Taylor Enterprises, Inc., All Rights Reserved. Adjusted Control Limits for P Charts. Dr. Wayne A. Taylor
Taylor Enterprses, Inc. Control Lmts for P Charts Copyrght 2017 by Taylor Enterprses, Inc., All Rghts Reserved. Control Lmts for P Charts Dr. Wayne A. Taylor Abstract: P charts are used for count data
More informationSimultaneous Optimization of Berth Allocation, Quay Crane Assignment and Quay Crane Scheduling Problems in Container Terminals
Smultaneous Optmzaton of Berth Allocaton, Quay Crane Assgnment and Quay Crane Schedulng Problems n Contaner Termnals Necat Aras, Yavuz Türkoğulları, Z. Caner Taşkın, Kuban Altınel Abstract In ths work,
More informationEGR 544 Communication Theory
EGR 544 Communcaton Theory. Informaton Sources Z. Alyazcoglu Electrcal and Computer Engneerng Department Cal Poly Pomona Introducton Informaton Source x n Informaton sources Analog sources Dscrete sources
More informationfind (x): given element x, return the canonical element of the set containing x;
COS 43 Sprng, 009 Dsjont Set Unon Problem: Mantan a collecton of dsjont sets. Two operatons: fnd the set contanng a gven element; unte two sets nto one (destructvely). Approach: Canoncal element method:
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More information4 Analysis of Variance (ANOVA) 5 ANOVA. 5.1 Introduction. 5.2 Fixed Effects ANOVA
4 Analyss of Varance (ANOVA) 5 ANOVA 51 Introducton ANOVA ANOVA s a way to estmate and test the means of multple populatons We wll start wth one-way ANOVA If the populatons ncluded n the study are selected
More informationStanford University CS254: Computational Complexity Notes 7 Luca Trevisan January 29, Notes for Lecture 7
Stanford Unversty CS54: Computatonal Complexty Notes 7 Luca Trevsan January 9, 014 Notes for Lecture 7 1 Approxmate Countng wt an N oracle We complete te proof of te followng result: Teorem 1 For every
More informationx = , so that calculated
Stat 4, secton Sngle Factor ANOVA notes by Tm Plachowsk n chapter 8 we conducted hypothess tests n whch we compared a sngle sample s mean or proporton to some hypotheszed value Chapter 9 expanded ths to
More informationLecture Space-Bounded Derandomization
Notes on Complexty Theory Last updated: October, 2008 Jonathan Katz Lecture Space-Bounded Derandomzaton 1 Space-Bounded Derandomzaton We now dscuss derandomzaton of space-bounded algorthms. Here non-trval
More informationLearning Theory: Lecture Notes
Learnng Theory: Lecture Notes Lecturer: Kamalka Chaudhur Scrbe: Qush Wang October 27, 2012 1 The Agnostc PAC Model Recall that one of the constrants of the PAC model s that the data dstrbuton has to be
More informationEntropy Coding. A complete entropy codec, which is an encoder/decoder. pair, consists of the process of encoding or
Sgnal Compresson Sgnal Compresson Entropy Codng Entropy codng s also known as zero-error codng, data compresson or lossless compresson. Entropy codng s wdely used n vrtually all popular nternatonal multmeda
More informationIntroduction to Information Theory, Data Compression,
Introducton to Informaton Theory, Data Compresson, Codng Mehd Ibm Brahm, Laura Mnkova Aprl 5, 208 Ths s the augmented transcrpt of a lecture gven by Luc Devroye on the 3th of March 208 for a Data Structures
More informationA 2D Bounded Linear Program (H,c) 2D Linear Programming
A 2D Bounded Lnear Program (H,c) h 3 v h 8 h 5 c h 4 h h 6 h 7 h 2 2D Lnear Programmng C s a polygonal regon, the ntersecton of n halfplanes. (H, c) s nfeasble, as C s empty. Feasble regon C s unbounded
More informationLOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin
Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationMaximizing the number of nonnegative subsets
Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationWinter 2008 CS567 Stochastic Linear/Integer Programming Guest Lecturer: Xu, Huan
Wnter 2008 CS567 Stochastc Lnear/Integer Programmng Guest Lecturer: Xu, Huan Class 2: More Modelng Examples 1 Capacty Expanson Capacty expanson models optmal choces of the tmng and levels of nvestments
More informationAppendix B: Resampling Algorithms
407 Appendx B: Resamplng Algorthms A common problem of all partcle flters s the degeneracy of weghts, whch conssts of the unbounded ncrease of the varance of the mportance weghts ω [ ] of the partcles
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More informationTOPICS MULTIPLIERLESS FILTER DESIGN ELEMENTARY SCHOOL ALGORITHM MULTIPLICATION
1 2 MULTIPLIERLESS FILTER DESIGN Realzaton of flters wthout full-fledged multplers Some sldes based on support materal by W. Wolf for hs book Modern VLSI Desgn, 3 rd edton. Partly based on followng papers:
More informationBasic Statistical Analysis and Yield Calculations
October 17, 007 Basc Statstcal Analyss and Yeld Calculatons Dr. José Ernesto Rayas Sánchez 1 Outlne Sources of desgn-performance uncertanty Desgn and development processes Desgn for manufacturablty A general
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationThe Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD
he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s
More informationHashing. Alexandra Stefan
Hashng Alexandra Stefan 1 Hash tables Tables Drect access table (or key-ndex table): key => ndex Hash table: key => hash value => ndex Man components Hash functon Collson resoluton Dfferent keys mapped
More informationCHAPTER 17 Amortized Analysis
CHAPTER 7 Amortzed Analyss In an amortzed analyss, the tme requred to perform a sequence of data structure operatons s averaged over all the operatons performed. It can be used to show that the average
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationMin Cut, Fast Cut, Polynomial Identities
Randomzed Algorthms, Summer 016 Mn Cut, Fast Cut, Polynomal Identtes Instructor: Thomas Kesselhem and Kurt Mehlhorn 1 Mn Cuts n Graphs Lecture (5 pages) Throughout ths secton, G = (V, E) s a mult-graph.
More informationReal-Time Systems. Multiprocessor scheduling. Multiprocessor scheduling. Multiprocessor scheduling
Real-Tme Systems Multprocessor schedulng Specfcaton Implementaton Verfcaton Multprocessor schedulng -- -- Global schedulng How are tasks assgned to processors? Statc assgnment The processor(s) used for
More informationMLE and Bayesian Estimation. Jie Tang Department of Computer Science & Technology Tsinghua University 2012
MLE and Bayesan Estmaton Je Tang Department of Computer Scence & Technology Tsnghua Unversty 01 1 Lnear Regresson? As the frst step, we need to decde how we re gong to represent the functon f. One example:
More informationLossy Compression. Compromise accuracy of reconstruction for increased compression.
Lossy Compresson Compromse accuracy of reconstructon for ncreased compresson. The reconstructon s usually vsbly ndstngushable from the orgnal mage. Typcally, one can get up to 0:1 compresson wth almost
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationTHE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY. William A. Pearlman. References: S. Arimoto - IEEE Trans. Inform. Thy., Jan.
THE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY Wllam A. Pearlman 2002 References: S. Armoto - IEEE Trans. Inform. Thy., Jan. 1972 R. Blahut - IEEE Trans. Inform. Thy., July 1972 Recall
More informationMarkov Chain Monte Carlo (MCMC), Gibbs Sampling, Metropolis Algorithms, and Simulated Annealing Bioinformatics Course Supplement
Markov Chan Monte Carlo MCMC, Gbbs Samplng, Metropols Algorthms, and Smulated Annealng 2001 Bonformatcs Course Supplement SNU Bontellgence Lab http://bsnuackr/ Outlne! Markov Chan Monte Carlo MCMC! Metropols-Hastngs
More informationIntroduction to information theory and data compression
Introducton to nformaton theory and data compresson Adel Magra, Emma Gouné, Irène Woo March 8, 207 Ths s the augmented transcrpt of a lecture gven by Luc Devroye on March 9th 207 for a Data Structures
More informationA Simple Inventory System
A Smple Inventory System Lawrence M. Leems and Stephen K. Park, Dscrete-Event Smulaton: A Frst Course, Prentce Hall, 2006 Hu Chen Computer Scence Vrgna State Unversty Petersburg, Vrgna February 8, 2017
More informationLINEAR REGRESSION ANALYSIS. MODULE VIII Lecture Indicator Variables
LINEAR REGRESSION ANALYSIS MODULE VIII Lecture - 7 Indcator Varables Dr. Shalabh Department of Maematcs and Statstcs Indan Insttute of Technology Kanpur Indcator varables versus quanttatve explanatory
More informationParametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010
Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton
More informationCOMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS
Avalable onlne at http://sck.org J. Math. Comput. Sc. 3 (3), No., 6-3 ISSN: 97-537 COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationx i1 =1 for all i (the constant ).
Chapter 5 The Multple Regresson Model Consder an economc model where the dependent varable s a functon of K explanatory varables. The economc model has the form: y = f ( x,x,..., ) xk Approxmate ths by
More informationDC-Free Turbo Coding Scheme Using MAP/SOVA Algorithms
Proceedngs of the 5th WSEAS Internatonal Conference on Telecommuncatons and Informatcs, Istanbul, Turkey, May 27-29, 26 (pp192-197 DC-Free Turbo Codng Scheme Usng MAP/SOVA Algorthms Prof. Dr. M. Amr Mokhtar
More informationOn balancing multiple video streams with distributed QoS control in mobile communications
On balancng multple vdeo streams wth dstrbuted QoS control n moble communcatons Arjen van der Schaaf, José Angel Lso Arellano, and R. (Inald) L. Lagendjk TU Delft, Mekelweg 4, 68 CD Delft, The Netherlands
More informationLecture 12: Classification
Lecture : Classfcaton g Dscrmnant functons g The optmal Bayes classfer g Quadratc classfers g Eucldean and Mahalanobs metrcs g K Nearest Neghbor Classfers Intellgent Sensor Systems Rcardo Guterrez-Osuna
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationCS 798: Homework Assignment 2 (Probability)
0 Sample space Assgned: September 30, 2009 In the IEEE 802 protocol, the congeston wndow (CW) parameter s used as follows: ntally, a termnal wats for a random tme perod (called backoff) chosen n the range
More informationHierarchical State Estimation Using Phasor Measurement Units
Herarchcal State Estmaton Usng Phasor Measurement Unts Al Abur Northeastern Unversty Benny Zhao (CA-ISO) and Yeo-Jun Yoon (KPX) IEEE PES GM, Calgary, Canada State Estmaton Workng Group Meetng July 28,
More informationCompression in the Real World :Algorithms in the Real World. Compression in the Real World. Compression Outline
Compresson n the Real World 5-853:Algorthms n the Real World Data Compresson: Lectures and 2 Generc Fle Compresson Fles: gzp (LZ77), bzp (Burrows-Wheeler), BOA (PPM) Archvers: ARC (LZW), PKZp (LZW+) Fle
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 31 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 6. Rdge regresson The OLSE s the best lnear unbased
More informationCS : Algorithms and Uncertainty Lecture 17 Date: October 26, 2016
CS 29-128: Algorthms and Uncertanty Lecture 17 Date: October 26, 2016 Instructor: Nkhl Bansal Scrbe: Mchael Denns 1 Introducton In ths lecture we wll be lookng nto the secretary problem, and an nterestng
More informationFinding Dense Subgraphs in G(n, 1/2)
Fndng Dense Subgraphs n Gn, 1/ Atsh Das Sarma 1, Amt Deshpande, and Rav Kannan 1 Georga Insttute of Technology,atsh@cc.gatech.edu Mcrosoft Research-Bangalore,amtdesh,annan@mcrosoft.com Abstract. Fndng
More informationA Robust Method for Calculating the Correlation Coefficient
A Robust Method for Calculatng the Correlaton Coeffcent E.B. Nven and C. V. Deutsch Relatonshps between prmary and secondary data are frequently quantfed usng the correlaton coeffcent; however, the tradtonal
More informationRefined Coding Bounds for Network Error Correction
Refned Codng Bounds for Network Error Correcton Shenghao Yang Department of Informaton Engneerng The Chnese Unversty of Hong Kong Shatn, N.T., Hong Kong shyang5@e.cuhk.edu.hk Raymond W. Yeung Department
More informationEconomics 130. Lecture 4 Simple Linear Regression Continued
Economcs 130 Lecture 4 Contnued Readngs for Week 4 Text, Chapter and 3. We contnue wth addressng our second ssue + add n how we evaluate these relatonshps: Where do we get data to do ths analyss? How do
More informationIntroduction to Algorithms
Introducton to Algorthms 6.046J/18.401J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) What data structures
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More informationMachine learning: Density estimation
CS 70 Foundatons of AI Lecture 3 Machne learnng: ensty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square ata: ensty estmaton {.. n} x a vector of attrbute values Objectve: estmate the model of
More informationIntroduction to Vapor/Liquid Equilibrium, part 2. Raoult s Law:
CE304, Sprng 2004 Lecture 4 Introducton to Vapor/Lqud Equlbrum, part 2 Raoult s Law: The smplest model that allows us do VLE calculatons s obtaned when we assume that the vapor phase s an deal gas, and
More informationMore metrics on cartesian products
More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of
More informationError Probability for M Signals
Chapter 3 rror Probablty for M Sgnals In ths chapter we dscuss the error probablty n decdng whch of M sgnals was transmtted over an arbtrary channel. We assume the sgnals are represented by a set of orthonormal
More informationOutline and Reading. Dynamic Programming. Dynamic Programming revealed. Computing Fibonacci. The General Dynamic Programming Technique
Outlne and Readng Dynamc Programmng The General Technque ( 5.3.2) -1 Knapsac Problem ( 5.3.3) Matrx Chan-Product ( 5.3.1) Dynamc Programmng verson 1.4 1 Dynamc Programmng verson 1.4 2 Dynamc Programmng
More informationPricing Network Services by Jun Shu, Pravin Varaiya
Prcng Network Servces by Jun Shu, Pravn Varaya Presented by Hayden So September 25, 2003 Introducton: Two Network Problems Engneerng: A game theoretcal sound congeston control mechansm that s ncentve compatble
More informationStatistical Inference. 2.3 Summary Statistics Measures of Center and Spread. parameters ( population characteristics )
Ismor Fscher, 8//008 Stat 54 / -8.3 Summary Statstcs Measures of Center and Spread Dstrbuton of dscrete contnuous POPULATION Random Varable, numercal True center =??? True spread =???? parameters ( populaton
More informationEmbedded Systems. 4. Aperiodic and Periodic Tasks
Embedded Systems 4. Aperodc and Perodc Tasks Lothar Thele 4-1 Contents of Course 1. Embedded Systems Introducton 2. Software Introducton 7. System Components 10. Models 3. Real-Tme Models 4. Perodc/Aperodc
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 6 Luca Trevisan September 12, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 6 Luca Trevsan September, 07 Scrbed by Theo McKenze Lecture 6 In whch we study the spectrum of random graphs. Overvew When attemptng to fnd n polynomal
More informationDesign and Optimization of Fuzzy Controller for Inverse Pendulum System Using Genetic Algorithm
Desgn and Optmzaton of Fuzzy Controller for Inverse Pendulum System Usng Genetc Algorthm H. Mehraban A. Ashoor Unversty of Tehran Unversty of Tehran h.mehraban@ece.ut.ac.r a.ashoor@ece.ut.ac.r Abstract:
More informationLecture 5 Decoding Binary BCH Codes
Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationLecture 4: November 17, Part 1 Single Buffer Management
Lecturer: Ad Rosén Algorthms for the anagement of Networs Fall 2003-2004 Lecture 4: November 7, 2003 Scrbe: Guy Grebla Part Sngle Buffer anagement In the prevous lecture we taled about the Combned Input
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VIII LECTURE - 34 ANALYSIS OF VARIANCE IN RANDOM-EFFECTS MODEL AND MIXED-EFFECTS EFFECTS MODEL Dr Shalabh Department of Mathematcs and Statstcs Indan
More informationQueueing Networks II Network Performance
Queueng Networks II Network Performance Davd Tpper Assocate Professor Graduate Telecommuncatons and Networkng Program Unversty of Pttsburgh Sldes 6 Networks of Queues Many communcaton systems must be modeled
More informationConnecting Multiple-unicast and Network Error Correction: Reduction and Unachievability
Connectng Multple-uncast and Network Error Correcton: Reducton and Unachevablty Wentao Huang Calforna Insttute of Technology Mchael Langberg Unversty at Buffalo, SUNY Joerg Klewer New Jersey Insttute of
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Logstc Regresson CAP 561: achne Learnng Instructor: Guo-Jun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate class-condtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationGames of Threats. Elon Kohlberg Abraham Neyman. Working Paper
Games of Threats Elon Kohlberg Abraham Neyman Workng Paper 18-023 Games of Threats Elon Kohlberg Harvard Busness School Abraham Neyman The Hebrew Unversty of Jerusalem Workng Paper 18-023 Copyrght 2017
More informationSampling Theory MODULE VII LECTURE - 23 VARYING PROBABILITY SAMPLING
Samplng heory MODULE VII LECURE - 3 VARYIG PROBABILIY SAMPLIG DR. SHALABH DEPARME OF MAHEMAICS AD SAISICS IDIA ISIUE OF ECHOLOGY KAPUR he smple random samplng scheme provdes a random sample where every
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More information