Nonparametric and Parametric Defined This text distinguishes between systems and the sequences (processes) that result when a WN input is applied

Transcription

1 Linear Signal Models Overview Introduction Linear nonparametric vs. parametric models Equivalent representations Spectral flatness measure PZ vs. ARMA models Wold decomposition Introduction Many researchers use signal models to analyze stationary univariate time series Goal: estimate the process from which the signal was generated Called signal modeling Related to, but different from, system identification Popular assumptions: is ergodic and WSS The system is LTI and stable The input signal is WN The input signal is Gaussian The system is a finite-order PZ system J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver. 1.0 Terminology Nonparametric and Parametric Defined This text distinguishes between systems and the sequences (processes) that result when a WN input is applied h min (n) Systems: AZ, AP, PZ Processes Moving Average (MA) Autoregressive Moving-Average (ARMA) Autoregressive (AR) The processes are assumed to have a WN input signal: WN(0,σw) In some cases, the input is assumed to be a sum of sinusoids In this case, the output consists of line spectra Goal: estimate frequencies and magnitudes of spectral components Nonparametric Models: LTI system models that are specified by the impulse response System is completely specified by Even if the system is causal, may be infinitely long in general Requires an infinite number of parameters to specify Parametric Models: system models that can be specified with a finite number of parameters Almost always finite-order AP, AZ, or PZ Easier to deal with in practical applications Constrains and H(z) J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver

2 Nonparametric versus Parametric We will focus on nonparametric estimators These generally have greater variability but less bias than parametric estimators. Why? However, they do not give a compact representation of the process Will discuss parametric models in detail next term Why Assume Minimum-Phase? Recall that from R x (z) alone we cannot distinguish between minimum and non-minimum-phase systems This is true in general For any stable, finite-order ARMA process, {H(z),} where H(z) has no zeros or poles on the unit circle, there exists a white noise process such that X(z) =H min (z) W (z) and H min (z) is minimum-phase If we have no other information but can only observe the signal, there is no reason not to assume is stable and minimum-phase! Very important assumption If is IID, it is not true in general that is IID J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver Nonrecursive Representation Nonrecursive Representation Continued = h(k)w(n k) r x (l) =σ wr h (l) R x (z) =σ wh(z)h (z ) R x (e jω )=σ w H(e jω ) Note that this is a non-recursive representation Any LTI system can be written in this form The shape of r x (l) and R x (e jω ) are completely determined by the system = h(k)w(n k) Since we cannot distinguish between signals produced by causal and non-causal systems, we can assume the system is causal This is identical to an infinite-order (Q = ) AZ model! We cannot distinguish between the system gain and the white noise process power: α αw(n k) Without loss of generality, we can assume h(0) = 1 J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver

3 = = Recursive Representation h I (n) h I (k)x(n k) =+ h I (k)x(n k) h I (k)x(n k) Let us choose (without loss of generality) that h(0) = 1 If we assume the inverse system is causal and stable, then h I (0) = 1 (why?) In this case, the output is a function of the current (unknown) input and all past values of This is identical to an infinite-order (P = ) APmodel! Equivalent representation of the nonrecursive form Innovations Representation If we assume H(z) is minimum-phase (reasonable), both and h I (n) exist and both are causal and stable. = h(k)w(n k) = x(n +1) = w(n +1)+ x(n +1) = w(n +1) }{{} New Information + h(n +1) h(n k)w(k) h(n +1 k)w(k) j= h I (k j)x(j) } {{ } Past values of J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver Comments on Innovations Representation + h(n +1) h I (k j)x(j) x(n +1) = w(n +1) }{{} New Information j= } {{ } Past values of If the system generating is minimum-phase, w(n +1) carries all the new information needed to generate x(n +1) Thus, w(n +1)is sometimes called the innovation All other information can be predicted from past observations of the output Only holds if is minimum-phase In some contexts, is called the synthesis or coloring filter The inverse is called the analysis or whitening filter PZ, AZ, and AP Representations We just saw that a nonparametric model can be represented as either white noise driving an AZ( ) system or a PZ( ) system Any causal PZ, AZ, or PZ system with finite order can be represented as either a causal AZ( ) orpz( ) system If the system is stable, then 0 as n Thus, if the order of the system is sufficiently large, we can represent any of these systems accurately with an AZ(Q) or AP(P ) system We ll see next term that AP(P ) are much easier to estimate than PZ(Q, P )oraz(q) systems Thus, it is very good news that AP(P ) systems can represent any PZ(Q, P )oraz(q) system if P is large enough See Example 4..1 in the text and discussion in preceding paragraph J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver

4 Spectral Factorization R x (e jω ) = σ w H min (e jω ) Regular: Processes that satisfy the Paley-Wiener condition π π ln R x (e jω ) dω < Regular processes can be factored as R x (z) =σwh min (z)hmin(z ) ( 1 π σw =exp ln [ R x (e jω ) ] ) dω π π Cepstrum The cepstrum is the inverse Fourier transform of ln R x (e jω ) c(k) 1 π ln [ R x (e jω ) ] e jkω dω π π The minimum-phase component of the spectrum is the causal part c + (k) 1 c(0) + c(k)u(k 1) h min (n) =F 1 {exp F{c + (k)}} The maximum-phase component is the anticausal part c (k) 1 c(0) + c(k)u( 1) h max (n) =F 1 {exp F{c (k)}} This is rarely used in practice. If R x (z) is a rational function, spectral factorization is straightforward J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver Spectral Flatness ( 1 π exp π π ln [ R x (e jω ) ] dω) SFM x π π R x(e jω )dω 1 π = σ w σ x Single measure of the spectral flatness Bounded: 0 SFM x 1 If SFM x =1,then is a white process Numerator is the geometric mean, denominator is the arithmetic mean Parametric Signal Models a k x(n k) = W (z) = Q b kz B(z) P a = Parametric models have rational (finite-order) system functions Each can be specified by a linear constant-coefficient difference equation To make the specifications unique, we always set a 0 =1and usually b 0 =1 The model is then defined by {a 1,a,...,a P,b 1,...,b Q,σw} J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver

5 + Parametric Signal Models a k x(n k) =+ W (z) = 1+ Q b kz 1+ B(z) P a = The models are generally divided into three categories Moving-Average Model: MA(Q), P =0 Autoregressive Model: AR(P ), Q =0 Autoregressive Moving-Average Model: ARMA(P,Q) All models assume the systems are BIBO stable In general, when estimating, we constrain ourselves to minimum-phase systems + Parametric Signal Model Limitations a k x(n k) =+ W (z) = 1+ Q b kz 1+ B(z) P a = Parametric signal models all have short-memory behavior The autocorrelation decays exponentially AZ systems can have any impulse response (unconstrained) AP and PZ systems are constrained because the poles must be inside the unit circle Long-term behavior is a sum of decaying exponentials Parametric models are useful for creating processes with any continuous spectrum (PSD) J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver Parametric Signal Model Theory h I (n) There are many equivalent representations of WSS processes modeled as WN driving an LTI system Impulse response and σ w Inverse system impulse response h I (n) and σw Output autocorrelation Output PSD R x (e jω ) If we assume a parametric model, we can also specify by {a 1,a,...,a P,b 1,...,b Q,σw} If we consider lattice filters, we have yet another representation All are complete descriptions of the processes WOLD Decomposition WOLD Decomposition: A general stationary random process can be written as =x r (n)+x p (n) where x r (n) is a regular process with a continuous PSD and x p (n) is a predictable process with a discrete spectrum. Further E [ x r (n 1 )x p(n ) ] =0 n 1,n In general, stationary random processes consist of a continuous PSD R x (e jω ) and a discrete power spectrum with DTFS coefficients R x (k) These processes are called mixed Continuous PSD is due to regular processes (unpredictable) Discrete is due to harmonic or almost periodic processes (predictable) The autocorrelation is r x (l) =r xr (l)+r xp (l) Proof is very difficult (see references in text) J. McNames Portland State University ECE 538/638 Signal Models Ver J. McNames Portland State University ECE 538/638 Signal Models Ver