BATCH-FORM SOLUTIONS TO OPTIMAL INPUT SIGNAL RECOVERY IN THE PRESENCE OF NOISES

Transcription

1 (Preprint) AAS BATCH-FORM SOLUTIONS TO OPTIMAL INPUT SIGNAL RECOVERY IN THE PRESENCE OF NOISES P Lin, M Q Phan, and S A Ketcham This paper studies the problem of optimally recovering the input signals to a linear time-invariant system when both input and measurement noises are present We focus on batch-form solutions which are suitable for applications that deal with short-duration events The system, the input and measurement noise covariances, the noise-corrupted output signals are assumed known, and we seek to recover the input signals that enter the system prior to being corrupted by input noise The proposed solution works through a filter to characterize the input and measurement noise statistics The input signal recovery is optimal in the sense that the filter residual is correctly recovered from the given information about the model and the output measurements A weighted least-squares solution is found to be both simple and useful in acoustic signal recovery applications INTRODUCTION In the area of short-duration large-domain acoustic and seismic signal propagation, highly accurate reduced-order models represent an enabling technology, Refs []-[] These techniques can also be applied to study the propagation of vibrations throughout a large flexible structure Such models are derived from HPC-derived data, and can be used for rapid prediction of the dynamic responses without resorting to the time-consuming HPC simulation Significant savings in computational resources and time can be achieved by this strategy, reducing what normally takes hours on a HPC supercomputer to minutes on a laptop Current research efforts are being made to extend the use of reduced-order models beyond output prediction These efforts include the problems of source signal recovery and source localization By taking advantage of the knowledge of the dynamics of the environment represented by these reduced-order models, it is possible to address the source signal recovery and localization problems in a highly complex multi-path environment with non-line-of-sight sensors This paper studies the problem of optimally recovering the input signals when both input and measurement noises are present We particularly focus on batch forms of the solutions for applications that deal with short-duration events, where a typical time duration of the signals to be recovered is in the range of 5 samples The system, the input and measurement noise covariances, and the noise-corrupted output signals, are assumed known, and we seek to recover the input signal that enters the system prior to being corrupted by noise For the given system and the specific input and measurement noise sequences, there is a unique residual associated with Thayer School of Engineering, Dartmouth College, Hanover, NH 755 Thayer School of Engineering, Dartmouth College, Hanover, NH 755 Signature Physics Branch, Cold Regions Research and Engineering Laboratory, Hanover, NH 755

2 this specific noise-free input signal For optimal input recovery, we seek an input signal that reproduces this unique residual The challenge of the problem lies in the fact that this unique residual is not known a priori because we do not assume knowledge of the specific input and measurement noise sequences, but only their covariances Instead of working with the input and measurement noise covariances, our approach works through a filter, because given the model and the input and measurement noise statistics, the filter gain is uniquely determined In fact, such a filter and system model can be identified from input-output data using modern system identification techniques, Refs [5]-[] Two approaches are considered for the input identification problem The first approach is based on an ARX model (Auto-Regressive model with exogenous inputs) The coefficients of this model are related to the state-space model and the filter gain This model is suitable because when the order of the ARX model is sufficiently large, the single additive noise term in the model is the residual which is known to be white and minimized White additive error is especially attractive for least-squares identification problems The input signal is found by minimizing this residual Next, we seek to determine how this solution is related to a simpler noise-free solution that is based on the unit pulse response model This second approach turns out be useful in that it leads to a weighted least-squares solution The improvement of the weighted least-squares solution is due to the fact that it minimizes the correct filter residual, whereas the ordinary least-squares solution minimizes an incorrect colored residual The filter gain is involved in a weighting matrix of the weighted least-squares solution that improves the recovery of the input signals Furthermore, in the absence of noises, the weighted least-square solution reduces to the regular least-squares solution Intuitively appealing implications can also be gained with this approach: It explains when explicit consideration of the noise covariances leads to improved identification results over the regular least-squares solution It also explains why for many problems, the regular least-squares solution, when applied to noisy measurements, is found to be adequate Numerical examples are supplied to demonstrate the effectiveness of the proposed approach PROBLEM STATEMENT Consider a linear time-invariant, finite dimensional system x(k + ) = Ax(k) + Bu(k) + w(k) () y(k) = Cx(k) + Du(k) + v(k) () where u(k) R m is the input signal that we would like to recover, w(k) R n and v(k) R l represent process noise and measurement noise, respectively, both unknown, but assumed to be white and Gaussian Given system matrices A, B, C, D and the covariance matrices of w(k) and v(k), we would like to recover u(k) from the output y(k) R l, k =,,,, s We assume that the number independent outputs is at least equal to the number of independent inputs Furthermore, because we are dealing with short time records in our applications, we are particularly interested in the batch forms of the solution

3 THE KALMAN FILTER Knowing the system model and the covariance matrices of process noise and measurement noise is equivalent to knowing the corresponding filter with gain K, ˆx(k + ) = Aˆx(k) + Bu(k) + Kε(k) () y(k) = C ˆx(k) + Du(k) + ε(k) () where ε(k) is the filter residual, which is minimized and white Instead of dealing with the covariances of the process and measurement noises, we will work with the fitter gain K Not only this approach is mathematically simpler, in practice the system state-space model and the filter gain can be identified directly from input-output data by several available techniques, Refs [5]-[] Another form of the filter is found to be useful, especially in system identification applications Substituting () into () and re-arranging terms produces, ˆx(k + ) = Aˆx(k) + Bu(k) + K (y(k) C ˆx(k) Du(k)) (5) By making the following simplifying definitions, = (A KC) ˆx(k) + (B KD) u(k) + Ky(k) (6) Ā = A KC B = B KD (7) we have another form of the filter where the residual is not explicitly present in the state equation, ˆx(k + ) = Āˆx(k) + Bu(k) + Ky(k) (8) y(k) = C ˆx(k) + Du(k) + ε(k) (9) In the following we will derive various input-output models from these two forms of the filter and use the resultant models for input identification INPUT IDENTIFICATION BY ARX MODEL We derive an ARX model with the aid of the filter, and invert it for input identification Propagating (8), (9) forward in time yields the following ARX model (Auto-Regressive model with exogenous input) The model is valid for k p such that Ā p regardless of the initial state x(), y(k) = α p y(k p) + + α y(k ) + β p u(k p) + + β u(k) + ε(k) () The coefficients α i s and β i s of the ARX model are related to the original state-space system matrices by the following relationship, i =,,, p, α i = CĀi K β i = CĀi B β = D () If the order p of the ARX model is sufficiently large such that Āp, the above ARX model has a single additive error term ε(k) This error term is the output residual, which is white and minimized We aim to find the input signal by minimizing this residual Since we are looking for a batch form of the solution, we need to write the input-output equations for all available time steps, and re-package the results before inversion This step is done in the next section

4 Solution by ARX Model Inversion To separate the known terms (model coefficients and output measurements) from the unknown terms (input values), re-write () as y(k) = [ y(k p) ] α p α + [ u(k p) ] β p β + ε(k) () y(k ) u(k p) where k p Defining the common input and output history vectors, y() u() y() u() y s = y(s ) u = u(s) Writing () for all available time steps starting from k = p and packaging them in a single matrix equation produces, where y ps and ε ps are defined as y ps = () y ps = H α y s + H β u + ε ps () y(p) y(p + ) y(s) ε ps = e(p) e(p + ) e(s) and the ARX model coefficients are arranged in H α and H β, α p α β p β H α = H β = (6) α p α β p β For further simplicity, define Equation (7) can be written succinctly as (5) y α = y ps H α y s (7) y α = H β u + ε ps (8) Because ε ps contains the residual which is minimized We make use of this fact to find u that minimizes the norm of ε ps, [ (H u = H + β y ) T ] α = β H β H T β y α (9) where the + denotes the pseudoinverse computed via the singular value decomposition To correctly recover u from the above solution, the matrix H β needs to have full-column rank If the number of independent outputs is equal to the number of inputs, then H β is "wide" (having more columns than rows) In this case, the input u cannot be correctly solved for The requirement, therefore, is to have the number of independent outputs to be larger than the number of inputs, in which case H β becomes a "tall" matrix (having more rows than columns)

5 Orthogonal constraints The above solution minimizes the residual The residual is also orthogonal to the input and output measurements, and thus one might wish to build these constraints into the optimization problem to solve for u We have taken this approach, and discovered that it is not a viable one for the following reasons: Because the residual ε ps is a linear function of u, the orthogonality condition of ε ps with respect to u becomes quadratic in the elements of u Optimization problems with quadratic equality constraints are known to be very difficult In our applications, we are typically deal with short data records Imposing orthogonality conditions of the residuals on short data records turns out to be overly restrictive, as these orthogonality conditions should be only applied to long data records, as they would be the case in system identification where long input-output records are assumed to be available, and these orthogonality conditions are automatically satisfied by the least-squares solutions to find the ARX model coefficients with or without residual whitening Because ε ps is also orthogonal to output measurements, and the output measurements are known, the orthogonality conditions associated with the output measurements are linear Minimizing the residual ε ps subject to linear constraints is straightforward For this reason, we might replace the quadratic constraints associated with u by additional linear constraints associated with additional output measurements This strategy of replacing nonlinear constraints by linear constraints is theoretically attractive However, they do nothing to avoid the overly restrictive nature of imposing the orthogonality conditions on short data records as discussed above INPUT IDENTIFICATION FROM UNIT PULSE RESPONSE MODEL Because the filter gain K is involved in the coefficients of the ARX model in (), it is not immediately obvious what the solution proposed in (9) reduces to in the absence of noises For this reason, we will now take the approach of first finding a simple noise-free solution, then seeing how this would be biased in the presence of noises We then determine how this solution might be improved in the presence of noises Solution by Ordinary Least-Squares of Unit Pulse Response Model Returning to (), () and setting the noise terms to zero, we have the following deterministic model, x(k + ) = Ax(k) + Bu(k) () y(k) = Cx(k) + Du(k) () For the simplest case where the initial condition is zero, x() =, the relationship between an input history to an output history is y = P u () 5

6 Define y = y() y() y() y(s) u = u() u() u() u(s) P = D CB D CAB CB D CA s B CAB CB D () The coefficients CA i B in P are the Markov parameters of the system, which are also the unit pulse response samples Hence the model embedded in () is the unit pulse response model Given y and P, as long as P is full column rank, the input u can be recovered from: [ (P u = P + y = T P ) ] P T y () where the + again denotes the pseudoinverse computed via the singular value decomposition The above solution is referred to as the ordinary least-squares () solution In the presence of noise, this solution is biased because the additive noise term is colored instead of white To see why this is the case, in the presence of noises, the counterpart of () is derived from (), (), The additive noise term e is defined as y = P u + e (5) e = Qε (6) where Q is given as Q = I CK I CAK CK I CA s K CAK CK I ε = ε() ε() ε() ε(s) (7) Because ε(k) is the residual which is white, Qε is colored A better solution is one that is based on an equation whose additive noise term is white as explained in the next section Solution by Weighted Least-Squares To set up an equation where the additive noise term is white, we need to return to (8), (9) to obtain, Qy = P u + ε (8) where Q and P are defined as I CK I Q = CĀK CK I CĀs K CĀK CK I (9) 6

7 P = D C B D CĀ B C B D CĀs B C Ā B C B D () A key feature of this model is that the additive error term ε is white The ARX model considered in () is subsumed in the present model in (8), which is only valid for zero initial condition The least-squares solution associated with (8) is therefore unbiased and given as, u W LS = P [ ( + ) Q y = P T ] T P P Q y () The above solution is referred to as a weighted least-squares solution () for the reason explained in the next section Relationship between and Solutions To reveal the relationship between the two solutions given in () and (), we need to determine the relationship between the two models given in (5) and (8) Pre-multiplying (5) with Q produces Qy = QP u + QQε () Comparing to (8), it can be shown that P = QP The main diagonal matrices of the product QP are D s, and the off-diagonal matrices match those of P, ( CK) D + I (CB) = C (B KD) = C B ( C ĀK ) D (CK) (CB) + I (CAB) = CĀKD + C (A KC) B = CĀ B, () and QQ = I The main diagonal matrices of the product QQ are identity matrices, and the offdiagonal matrices are zero, ( CK) I + I (CK) = ( C ĀK ) I (CK) (CK) + CAK = CĀK + CĀK =, () Substituting P = QP into () produces u W LS = (P T QT QP ) P T QT Qy = (P T W P ) P T W y, (5) where W = Q T Q serves as a weighting matrix when the solution in (5) is compared to the solution of () We have explained why () which is derived from (8) can be interpreted as a weighted leastsquares solution The solution can also be derived from (5) as follows Pre-multiplying (5) with Q and comparing the result to (8) produces a relationship between ε and e as follows, ε = Qe (6) 7

8 The above relationship can also be established from (6) with Q = Q because QQ = I The residual ε is white and minimized, hence the correct cost function to minimize is J = ε T ε = e T QT Qe = e T W e (7) Substituting e = y P u into the cost function (7) and minimizing it with respect to u produces u W LS = ( P T W P ) P T W y, which is identical to the expression given in (5) We should note here that an important implication of the weighted least-squares solution is that any improvement over the ordinary least-squares solution is due to the weighting matrix W which can be computed based on the model and the filter gain In cases where W is close to an identity matrix, one should not expect any significant difference between the input signals recovered by both methods Finally, the inverse operation in the solutions can be replaced by the Moore-Penrose pseudoinverse if the matrix that needs to be inverted is ill-conditioned The pseudoinverse should be computed via the singular value decomposition where the nearly zero singular values are not inverted In this case, the resultant solution is non-causal because the pseudoinverse contains elements that multiply future output measurements to produce the input signal at the current time step Indeed for many discrete-time models where the pole-zero excess of the original continuous-time system is three or more, it is possible for the discrete-time representation to contain at least one zero outside the unit circle in the complex plane if the sampling interval is sufficiently small, Refs []-[] Causal inverses become unstable, and finding a non-causal inverse is a way to handle such systems, Ref [5] Here in batch forms, incorporation of non-causality in the solution is automatic through the truncation of the small singular values that are associated with these "unstable" zeros of the forward discrete-time model NUMERICAL EXAMPLES Two sets numerical results are presented The first set is based on a realistic -state acoustic propagation model of an office and laboratory complex derived from HPC (High Performance Computing) simulations The second set is based on a fictitious -state model to clarify the results obtained with the HPC model HPC Model of an Office and Laboratory Complex The original HPC model is derived by the following procedure A D finite-difference timedomain (FDTD) computation is used to simulate the propagation of a sound source placed at the center of the complex, Fig This simulation takes approximately 5 hours using 56 cores of a Cray XT with GB of memory per core The FDTD model has just under 7 billion cells, out of which 758 thousand output locations are selected to represent an output field 6-m above the ground surface and building roofs From this simulation data, the inverse FFT method is used to compute Markov parameters that describe the -sample long dynamics from the center source to the 758 thousand output locations The sampling interval is selected to be 5 sec The unit pulse response model, defined by these Markov parameters, is then converted into a -state model via the second form of the superstable representation, Refs [5], [6] From this original state-space model, model reduction is applied to produce a -state model for use in this numerical study For this illustration, we arbitrarily selected two outputs at locations number and 6 Model reduction is further applied to reduce the dimension of the original state-space model to states Thus the HPC model used in this paper is a -state -input -output model 8

9 Input Recovery Based on an HPC Model A random test input signal is applied at the center source Prior to entering the model this test input signal is corrupted by % input noise The outputs at locations number and 6 are recorded, and these outputs are further corrupted by % measurement noise The -time step test input signal is shown in Figure, and the output signals are shown in Figures The recovered input signals are shown in Figure for both the and methods along with the original test input To facilitate the comparison, zoomed-in segments are shown in Figure 5 Careful examination of the results reveal that the test input is recovered rather well up to time step 876 for the following reason In both methods matrix inverses or pseudoinverses are called for In the case of the solution, it s the pseudoinverse of P, and in the case of the solution, the pseudoinverse of P These matrices could be ill-conditioned as seen in this example The singular value decomposition was used to compute the pseudoinverses where the smallest singular values were truncated The singular values of P and P are shown in Figure 6 In these examples we chose to keep 876 singular values in the computation of the pseudoinverses for both methods This choice caused the recovered input signal to start decaying from the 876-th time step, and the recovered input is only valid up to that time step It turned out that the recovered input signal is relatively insensitive to this choice of singular value cut-off as long as a "sufficient" number of singular values is kept The number of singular values retained could be more or less without significantly changing the recovered input signal Next, we computed the output residuals by both methods, and compared them to the optimal residuals The optimal residuals are unique for the specific model, and the specific noise-corrupted input and output time histories This comparison is shown in Figure 7, and their zoomed-in versions in Figure 8 Close examination of these figures suggests that the residuals arevquite similar to the residuals, and both somewhat resemble the optimal residuals Close similarity between the and residuals is a surprise, whereas the weak correlation with the optimal residuals is caused by the ill-conditioning of the problem as revealed by the small singular values in Figure 6 To test this theory, we performed a simulation where the test input signal was restricted in a certain input space so that conditioning of the problem could be improved We used 8 random vectors as basis vectors to build the test input signal, and recover the input signal within this set of basis vectors The recovered input signals by both methods were found to match the test signal very well as shown in Figure 9, and there residuals strongly tracked the optimal residuals as shown in Figure and their zoomed-in versions in Figure This test confirmed that the source of the residual mismatch was due to the ill-conditioning of the model Despite this ill-conditioning, the input signal was recovered rather well One issue remains, however We need to determine the reason why the and solutions are so similar In order to show that the two solutions could be different, a fictitious model was used The results are shown in the next subsection Input Recovery Based on a Fictitious Model Consider the following fictitous model, [ ] [ ] [ ] [ ] 9 A =, B =, C =, D = 9 5 In this example the input noise level was set to be around %, and output noise levels around 5% To eliminate numerical ill-conditioning, we also restricted the input space for the test input 9

10 Figure shows that despite the relatively high noise levels, both methods recovered the input signal extremely well, and the solution is indeed closer to the test input signal than the solution Furthermore, the residuals matched the optimal residuals much better than the residuals (an almost perfect match was observed for the second output) Having seen that the solution is indeed more accurate than the solution, we determine why the two solutions were so similar to each other in the HPC model By examining the weighting matrices, it is possible to see the reason Figure graphically shows the -by- upper portion of these ]weighting matrices In both cases, the weighting matrices are diagonally dominant, but the weighting matrix for the HPC model is closer to an identify matrix when compared to the weighting matrix for the fictitious model This fact explains why the two solutions were similar to each other in the case of the HPC model, but the solution is better than the solution in the case of the fictitious model CONCLUSIONS In this paper we have considered batch-form solutions to the problem of input signal recovery in the presence of input and measurement noises The solutions are suitable for short-duration events in a signal propagation problem, such as the propagation of an acoustic signal in a large domain, or the propagation of a vibration throughout a large structure Instead of working with the input and output noise covariances, we take the approach of working through a filter, which characterizes both the system model and the noise covariances We have considered two working approaches: one that involves the inversion of an ARX model and another that involves the inversion of a pulse response model The second approach turns out to be useful in that it not only subsumes the first approach, it also leads to a simple weighted least squares solution () Advantages of the solution over an ordinary least-squares solution () also become apparent in that the solution minimizes a white residual whereas the solution minimizes a colored residual, which is known to lead to bias in the result Furthermore, by examining the weighting matrix of the solution, it is also possible to determine if the solution is expected to offer significant improvement over the solution in a practical application Another unexpected benefit of the batch forms of the solution lies in the fact that the regular inverse can be replaced by the pseudoinverse computed via the singular value decomposition when the inverse problem is ill-conditioned This type of ill-conditioning can arise when the original continuous-time system that the discrete-time model represents has pole-zero excess of three or more and when the sampling interval is sufficiently small In this case, a casual inverse is unstable The batch-form of the solution is advantageous because it sidesteps this unstable causal-inverse issue by making the inverse solution non-causal through the pseudoinverse operation Non-causality is not an issue for a batch type solution applied after the fact (ie, the source recovery is carried out after measurement of the output signals is complete) For completeness, we also considered ways to explicitly impose the conditions that the residual is also orthogonal to the input and output measurements Because the residual depends on the input signals, and the input signals themselves are not known (eg, to be solved for in this identification problem), orthogonality of the residual to the input signals leads to quadratic constraints in the optimization problem Such quadratic constraints are known to be extremely difficult to solve Various options to bypass this issue were considered, but they all lead to the conclusion that the orthogonality conditions themselves when forced on short data records are overly restrictive, although these orthogonality conditions are routinely achieved on long data

11 records in a typical system identification solution We have studied the applicability of the proposed solution on a realistic high-performance computing (HPC) model of an office and laboratory campus in Hanover, NH We found that the source signals can be recovered well with the proposed solution techniques Due to the nature of the propagation dynamics, the solution offers little improvement over the solution This result is a rather surprising, but not general, because it is system-specific To validate the method, we have also tested the method on a fictitious model where noticeable improvement of the solution over the solution was observed Finally, in both the HPC model and the fictitious model illustrations, the optimal residuals were correctly recovered This fact confirmed the validity of the proposed solution technique REFERENCES [] Anderson, TS, Moran, ML, Ketcham, SA, Lacombe, J: Tracked Vehicle Simulations and Seismic Wavefield Synthesis in Seismic Sensor Systems Computing in Science and Engineering, pp 8 () [] Ketcham, SA, Moran, ML, Lacombe, J, Greenfield, RJ, Anderson, TS: Seismic Source Model for Moving Vehicles IEEE Transactions on Geoscience and Remote Sensing,, No, pp 8 56 (5) [] Ketcham, SA, Wilson, DK, Cudney, H, Parker, M: Spatial Processing of Urban Acoustic Wave Fields From High-Performance Computations ISBN: , Digital Object Identifier: 9/HPCMP UGC768, DoD High Performance Computing Modernization Program Users Group Conference, pp (7) [] Ketcham, SA, Phan, MQ, and Cudney, HH: Reduced-Order Wave Propagation Modelling Using the Eigensystem Realization Algorithm Modeling, Simulation, and Optimization of Complex Process Bock, HG, Phu, HX, Rannacher, R, and Schlöder, JP (editors), Springer-Verlag, pp 8 9 () [5] Juang, J-N, Phan, MQ, Horta, LG, and Longman, RW: Identification of Observer/ Filter Markov Parameters - Theory and Experiments Journal of Guidance, Control, and Dynamics 6, No, 9 (99) [6] Phan, MQ, Horta, LG, Juang, J-N, and Longman, RW: Improvement of Observer/ Filter Identification (OKID) by Residual Whitening Journal of Vibrations and Acoustics 7, 8 (995) [7] Phan, MQ: Interaction Matrices in System Identification and Control Proceedings of the5th Yale Workshop on Adaptive and Learning Systems, New Haven, CT () [8] Lin, P, Phan, MQ, and Ketcham, SA: State-Space Model and Filter Gain Identification by a Superspace Method The 5th International Conference of High Performance Scientific Computing, Hanoi, Vietnam () [9] Juang, J-N: Applied System Identification Prentice-Hall, Upper Saddle River, NJ () [] Van Overchee, P and De Moor, B: Subspace Identification for Linear Systems Kluwer Academic Publishers (996) [] Verhaegen, M and Dewilde P: Subspace Model Identification Part : The Output Error State-Space Model Identification Class of Algorithms International Journal of Control 56, No 5, 87 (99) [] Panomruttanarug, B and Longman, RW: Repetitive Controller Design Using Optimization in the Frequency Domain Proceedings of the AIAA/AAS Astrodynamics Specialist Conference, Providence, RI () [] Longman, RW: On the Theory and Design of Linear Repetitive Control Systems European Journal of Control, 6, No 5, pp 7 96 () [] Longman, RW, Peng, Y-T, Kwon, T, Lus, H, Betti, R, and Juang, J-N:Adaptive Inverse Iterative Learning Control Advances in the Astronautical Sciences, Vol, pp 5- () [5] Brown, HM, Phan, MQ, and Ketcham, SA: A Non-Causal Inverse Model for Source Signal Recovery in Large Domain Wave Propagation The 5th International Conference of High Performance Scientific Computing, Hanoi, Vietnam ()

12 [6] Phan, MQ, Ketcham, SA, Darling, RS, and Cudney, HH: Superstable Models for Short-Duration Large Domain Wave Propagation Modeling, Simulation, and Optimization of Complex Process Bock, HG, Phu, HX, Rannacher, R, and Schlöder, JP (editors), Springer-Verlag, pp () Figure A complex with a center source and various sensor locations -

13 5 Test Input Signal original noise free input additive input noise Figure Test input signal with additive input noise x 5 Output response due to noisy input additive output noise x 5 Output response due to noisy input additive output noise Figure Output signals with additive output noises at locations (left) and 6 (right) 5 Recovered Input Signal vs Test Input Signal test recovered () 5 Recovered Input Signal vs Test Input Signal test recovered () Figure Recovered input signals with 876 singular values kept: (left) and (right)

14 Recovered Input Signal vs Test Input Signal test recovered () Recovered Input Signal vs Test Input Signal test recovered () Figure 5 Zoomed-in portions of the recovered input signals of Figure : (left) and (right) The test input signal is shown in red Singular Values Singular Values X: 876 Y: X: 876 Y: singular value index 6 8 singular value index Figure 6 Singular values of P for (left) and singular values of P for 5 x Residual Comparison (Output ) 5 x Residual Comparison (Output ) Figure 7 Comparison of residuals (blue), residuals (green) vs optimal residuals (red) for Output (left) and Output (right)

15 x 5 Residual Comparison (Output ) x 5 Residual Comparison (Output ) Figure 8 Zoomed-in portions of the residuals of Figure 7: Output (left) and Output (right) Optimal residuals are shown in red Recovered Input Signal vs Test Input Signal test recovered () Recovered Input Signal vs Test Input Signal test recovered () Figure 9 Recovered input signals in specified input space: (left) and (right) The test input signal is shown in red x 8 6 Residual Comparison (Output ) x 8 6 Residual Comparison (Output ) Figure Comparison of residuals (blue), residuals (green) vs optimal residuals (red) for Output (left) and Output (right) for test input in specified input space 5

16 5 x Residual Comparison (Output ) 5 x Residual Comparison (Output ) Figure Zoomed-in portions of Figure : Output (left) and Output (right) The and residuals closely resemble the optimal residuals in red Recovered Input Signal vs Test Input Signal test recovered () Recovered Input Signal vs Test Input Signal test recovered () Figure Recovered input signals for fictitious model: (left) and (right) showing the solution reproduces the test input signal better than the solution The test input signal is shown in red 8 6 Residual Comparison (Output ) 8 6 Residual Comparison (Output ) Figure Comparison of residuals (blue), residuals (green) vs optimal residuals (red) for Output (left) and Output (right) for fictions model zoomed-in portions The results show the residuals are better at matching the optimal residuals than the residuals 6

17 Weighting Matrix W Weighting Matrix W Figure weighting matrices for HPC model (left) and fictitious model (right) 7