NON-NEGATIVELY CONSTRAINED LEAST

Transcription

1 NON-NEGATVELY CONSTRANED LEAST SQUARES AND PARAMETER CHOCE BY THE RESDUAL PERODOGRAM FOR THE NVERSON OF ELECTROCHEMCAL MPEDANCE SPECTROSCOPY DATA Rosemary Renaut, Mathematics, ASU Jakob Hansen, Jarom Hogue and Grant Sander, Math Undergraduates ASU Sudeep Popat Biodesign nstitute ASU OCTOBER 4, 23

2 Outline Application: Electrochemical Spectroscopy for Biofuels Mathematical Model Nonlinear Fitting Linear Fitting Numerical Quadrature ntegration by t ntegration by s Right Preconditioner: Stability Example Models for Simulation Regularization Parameter Choice Nonnegative Least Squares Parameter Choice: NNLS, Stability Augmented Residual Conclusions

3 Physical Experiment: Microbial Electrolysis Cell for anaerobic bacteria (a) Fuel cell (b) Geobacter (c) Microbial electrolysis cell Figure: Geobacter, fuel cells, and waste water: Single-chamber microbial electrolysis cell consisting of a carbon fiber anode with a stainless steel current collector and a stainless steel cathode is used to convert food wastes into hydrogen in a single step.

4 (b) Potential Loss inside ARB cell Motivation: ARB in fuel cell should be electrically self-sufficient (a) Cyclic Voltammograms (CVs) Figure: (a) CVs of an ARB biofilm during its growth phase. Nernst-Monod relationship compared against the CVs (dotted lines). (b) CV explaining the potential profile for an ARB biofilm. The Nernst-Monod relationship (red dotted line) shows that most potential losses occur inside the ARB s cell.

5 Data Collection: Electrochemical mpedance Spectroscopy Figure: Schematic of the microbial electrochemical cells used for Electrochemical mpedance Spectroscopy (ES)

6 Nyquist Plot of ES impedance measurements Figure: Nyquist plot and fitting by equivalent circuit model (ECM) for an ARB biofilm tested at -.35 vs Ag/AgCl.

7 Goals: challenges Given ES measurements can we find. The underlying total resistance R pol 2. The equivalent circuit model - so as to understand the inefficiencies 3. Number and processes and sizes of their resistances R k such that k R k = R pol. 4. Chemical environment that minimizes ARB potential loss The aim is for a self sustaining electrochemical environment Challenges Limited sampling, perhaps 7 measurements No information on the noise in the data Measurements over a broad frequency range

8 Mathematical Model Fredholm ntegral Equation Small input current and measured output voltage leads to mpedance measurements Z(ω) for angular frequency ω Z(ω) = R + R pol g(t) dt, () + i ωt Kernel h(ω, t) = +i ωt - is not square integrable. R and R pol are unknown parameters. Distribution function of relaxation times (DRT) g(t) g(t)dt = and g(t) > is unknown

9 The DRT g(t) Simple Resistance - Capacitance Circuit Cole-Cole or ZARC single element. (literature dependent) g RQ (t) = 2πt The impedance is analytically given sin βπ ( ( )) cosh β ln t t + cos βπ Z RQ (ω) = < β < and distribution of time constants t., (2) + (iωt ) β, (3) Constant Phase element Q and resistance R

10 The DRT g(t) Simple network of resistance - capacitance circuits May also be realized by a lognormal distribution for the DRT ) g LN (t) = ( tσ 2π exp (ln(t) µ)2 2σ 2. (4) No analytic form for the impedance is given. Figure: Three lognormals and a weighted linear combination

11 Comparison of LN and RQ for one process chosen to align 8 7 LN RQ LN RQ.35 LN RQ (a) DRTs: t space 5 5 (b) DRTs: s space (c) Nyquist plot LN RQ LN RQ (d) Components Z (e) Components Z 2 Figure: Simulated exact data measured at 65 logarithmically spaced points in ω. n each case the solid line indicates the RQ functions and the symbols the LN functions. s = log(t).

12 Nonlinear Fitting f analytic Z(ω) is known fitting to Z is possible No analytic form for Z nonlinear fitting integrates the DRT What error is incurred by a wrong choice of model? The residuals are small Mean Fit by RQ Upper 95% bound Lower 95% bound Mean Fit by LN Upper 95% bound Lower 95% bound 6 8 Mean Fit by RQ Upper 95% bound Lower 95% bound Mean Fit by LN Upper 95% bound Lower 95% bound (a) Fitting to the RQ (b) Fitting to the LN Figure: Residual norms fitting one process to the impedance spectrum of a DRT consisting of one process with white noise.

13 The impact 8 7 LN RQ 8 7 LN RQ 8 7 LNfit RQ LN RQfit (a) RQ by LN 2 (b) LN by RQ 5 5 (c) RQ by LN: s space 5 5 (d) LN by RQ: s space Figure: Fitting functions with the consistent mean values obtained over 5 noisy selections.. True RQ β =.72, t =., true LN σ = 3, t =.. RQ to LN gives β =.86 and t = LN to RQ gives σ = and t =.3 Location and height of peak is incorrect Conclude: knowledge of the model is highly relevant for NLS.

14 Linear Formulation: ntegration in t Z(ω) = Z (ω) + iz 2 (ω) ) ( = (R + R pol g(t) ) + ω 2 t 2 dt ir pol ωtg(t) + ω 2 t 2 dt, ) ( ) = (R + R pol h (ω, t)g(t) dt ir pol h 2 (ω, t)g(t) dt. Given N quadrature points and [T min, T max ], Tmax N g(t)h k (ω, t) dt a n g(t n )h k (ω, t) (5) T min n= Matrices (A k ) mn = a n h k (ω m, t n ) m M, n N. Measurements (b ) m Z (ω m ) R and (b 2 ) m Z 2 (ω m ) (x) n g(t n ) satisfies the discrete linear systems A k x b k, k =,..., 3, A 3 = [A ; A 2 ] (6) Definining finer resolution also yields square A 4 = [A e, Ae 2 ]

15 Systems are ll-conditioned whether Simpson s or trapezium 3 N = 65 3 N = 25 2 Atrap Asimp A2trap A2simp A3trap A3simp A4trap A4simp 25 2 Atrap Asimp A2trap A2simp A3trap A3simp A4trap A4simp [,] [,2] [,3] [2,] [2,2] [2,3] [3,] [3,2] [3,3] ndices of Tmin and Tmax (a) N = 65 [,] [,2] [,3] [2,] [2,2] [2,3] [3,] [3,2] [3,3] ndices of Tmin and Tmax (b) N = Figure: The condition of matrices A k, k =... 4 (cond(a k )) plotted against the pairs (l, j) indicating [T min (l), T max (j)] with T min = [5e 2, e 2, 3e 3] and T max = [8e3, 4e4, 2e5] for both Simpson s and trapezium quadrature rules. Values for ω = 2πφ are based on logarithmic spacing on the interval (φ min, φ max ) = (.7, 5 ), consistent with the measured data.

16 Model Error for the LN g(t): integrating in t Truncation error occurs due to semi-infinite integral Quadrature error occurs due to limited sampling Exact measurements ˆb k (ω, t, σ) are contaminated ˆb k (ω, t, σ) = g(t, t, σ)h i (t, ω)dt + E trunc k (ω, T min, T max, t, σ) + E quad k (ω, T min, T max, t, σ, N), Figure: Larger error for h than h 2. Depends on location of peaks g(t) N = 65 N = 5 5 h actual (max, ω) = (.4,.) h estim. (max, ω) = (.37,.) h2 actual (max, ω) = (.5,.585) h2 estim. (max, ω) = (.7,.585) linear omega /omega /omega 2 h actual (max, ω) = (.6,.) h estim. (max, ω) = (.3,.) h2 actual (max, ω) = (.2,.674) h2 estim. (max, ω) = (.6,.674) linear omega /omega /omega 2 5 ω 5 ω

17 Transformation s = ln(t) Let s = ln(t) Z(ω) = h(ω, t)g(t) dt = h(ω, e s )f(s) ds, For the DRTs (2)-(4) we obtain the functions f(s) := tg(t). (7) f RQ (s) = 2π f LN (s) = σ 2π exp sin(βπ) (cosh(β(s ln(t ))) + cos(βπ)) (s µ)2 ( 2σ 2 Quadrature uses constant step in s h(ω, e s )f(s) ds smax s min (8) ), ln(t ) = µ σ 2, (9) h(ω, e s )f(s) ds s N n= h(ω, e sn )f(s n ). ()

18 Reduced Quadrature Error for integration by s than by t h (s) h 2 (s) h (t) h 2 (t) 6 h (s) h 2 (s) h (t) h 2 (t) 6 7 h (s) h 2 (s) h (t) h 2 (t) (c) t =., β =.5. (d) t =., β = (e) t =., β =.5. h (s) h (s) 5 h 2 (s) h (t) 5 h 2 (s) h (t) 5 h 2 (t) h 2 (t) h (s) h (s) 2 h (t) h 2 (t) (f) σ = ln(3) (g) σ = ln(3) (h) σ = ln(3). Figure: Quadrature error for a single RQ (above) LN(below) process with N = 65 for quadrature in s and t as a function of ω, plotted on a log-log scale.

19 mproved Quadrature : End conditions by integrating the kernels h k (ω, e s )f(s) ds s N h k (ω, e sn )f(s n )+ n= f(s N )r k,n (ω, e s N ) + f(s )r k, (ω, e s ). () r k,n (ω, e s N ) := r k, (ω, e s ) := { 2 ln( + (ωes N ) 2 ) k = π 2 tan (ωe s N )) k = 2, { 2 s h (ω, e s ) k = tan (ωe s ). k = 2. yields with H k (s) = h k (ω, s)f(s) and s s f(s) ds δ(f) { ɛ E k (ω) 2 ( s + ln(2)) + δ(f) + (s N s ) 3 (N + ) H 2N 3 (ζ) k = ɛπ + (s N s ) 3 (ζ), k = 2. H 2N 2 2

20 mpact on Conditioning Quad Equation A A 2 A 3 A 4 t (5).5e + 3.4e e e + 2 s () 2.9e e e e + 8 s () 2.8e e e e + 8 Table: Comparing condition numbers with different quadratures for optimal selection of nodes for t n, using t = /ω. Transformation acts as a right preconditioner on the system. Overdetermined system with A 3. Use matrix A 3 if sufficient data for resolution of the solution.

21 Right preconditioning separates frequencies of the basis (a) A : U above and V below (b) A s : U above and V below. Figure: Normalized Cumulative Periodograms (NCPs) and Kolmogorov-Smirnov 95% confidence bounds for white noise for spectral decomposition of matrices A and A s. NCPs for A 2 and A s 2 show a similar separation of frequency content for basis vectors. NCP outside KS bounds is with 95% confidence not white noise

22 Why is the spectral decomposition important? Solution is expressible in terms of the singular value decomposition of matrix A = UΣV T x = n i= u T i b v i σ i. if u i and/or v i are noise contaminated, so is x 2. Picard condition looks at the ratios ut i b σ i but depends on right hand side b to see the impact of noise 3. NCP looks only at the model matrix A. 4. Spectral decomposition is more informative than the condition number

23 Systems are ll-conditioned- the discrete problem is ill-posed [Dataset Rpol R] = [ ] x 5 A x 5 A2 x 4 A3 3 A4 x Figure: An example solution for real data provide no useful information- the solution is sensitive to noise in the data and the floating point arithmetic calculations

24 Consider mapping A which takes solution x to data b. Ax = b. Definition (Well - Posed (Hadamard in 923)) nverse problem of finding x from b is called well-posed if all Existence a solution exists for any data b in data space, Uniqueness the solution x is unique Stability continuous dependence of x on b : the inverse mapping b x is continuous. Definition (ll-posed: according to Hadamard) A problem is ill-posed if it does not satisfy all three conditions for well-posedness. Alternatively. b / range(a) 2. inverse is not unique because more than one image is mapped to the same data, 3. an arbitrarily small change in b can cause an arbitrarily large change in x.

25 Simulations: RQ simulation solid line, LN simulation. LN LN LN LN LN RQ 5 RQ RQ.9.7 RQ RQ (a) A: g(t) (b) tg(t) (c) Nyquist (d) Z (ω) (e) Z 2(ω) LN RQ.35 LN RQ.3 LN RQ LN RQ.3 LN RQ (f) B: g(t) (g) tg(t) (h) Nyquist (i) Z (ω) (j) Z 2(ω) LN RQ LN RQ 5 LN RQ LN RQ 5 LN RQ (k) C: g(t) (l) tg(t) (m) Nyquist (n) Z (ω) (o) Z 2(ω)

26 Regularization mproved matrices are still ill-conditioned: regularization is required. x = arg min{ Ax b 2 + λ 2 Lx 2 } (2) Tikhonov regularization controls smoothness of x λ: regularization parameter, many techniques exist to find λ - some require statistical properties of measurements L - standard choices or L or - derivative operators. An explicit solution is given with respect to basis vectors for GSVD of [A, L] = [UΣ A Z T, V Σ L Z T ], γ i = σi A/σL i n u T i x = b λ 2 γ i λ 2 + γi 2 z i i= and solves the normal equations (A T A + λ 2 L T L)x = A T x

27 Example Solutions and Residuals - range of λ: a real example Solutions : Solutions: L Solutions: Residuals : Residuals : L Residuals :

28 Parameter Choice: L-Curve (e.g. Hansen, Regularization Tools) For a given residual norm there does not exist a solution with smaller semi-norm than the one provided by the L-Curve, in that sense the solution is optimal.

29 Residual Periodogram - uses time series to detect noise,( e.g. Rust and O Leary, Hansen, Kilmer and Kjeldsen) Suppose vector y is a time series indexed by component i. Diagnostic Does histogram of entries of y generate histogram consistent with y N(, )? (i.e. independent normally distributed with mean and variance ) Not practical to automatically look at a histogram and make an assessment Diagnostic 2 Test expectation that y i are selected from white noise time series. Take Fourier transform of y and form cumulative periodogram z from power spectrum c (q = floor(m/2)) c j = (dft(y) j 2, z j = j i= c j q i= c, j =,..., q, i Automatic: Test is line (z j, j/q) close to straight line with slope and length 5/2?

30 Measure Deviation from Straight Line: Residual Vector : optimal black L true L Figure: Low noise: RQA A 4 Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual and is picked automatically

31 Measure Deviation from Straight Line: Residual Vector : optimal black L true L Figure: High noise: LNB A 4 Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual. Nonnegativity is violated

32 s the approach robust? Solutions by LLS using matrix A 4 : Minimum error red line, LC dotted blue and NCP green line (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example LLS solutions..% noise, RQ-B data set, matrix A 4.

33 Observations. Nonnegativity of g(t) not maintained. 2. Boundary effects are observed 3. However RP and LC are robust methods to estimate λ. They find solutions with error near the minimum. 4. Minimal error curve has a flat region, solutions relatively insensitive when picked within minimal error region. 5. But solutions with high noise are lacking features - over smoothing Can we impose additional constraints?

34 Nonnegative LS with regularization The DRT satisfies g(t) >. The transformed DRT, f(s) = tg(t) > also. Hence nonnegative least squares: x = arg min{ Ax b 2 + λ 2 Lx 2, s.t. x }, (3) { ( ) } Ax b 2 = arg min, s.t. x (4) λlx 2 Depends on regularization parameter λ Can be solved using a standard solver - e.g. Matlab lsqnonneg mpose bounds on solution x >. Parameter choice - solve for multiple λ and pick solution as for the LS problem. Residual Periodogram is new in context of NN LS

35 Measure Deviation from Straight Line: Residual Vector : optimal black L true L Figure: Low noise: RQA A 4 Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

36 Measure Deviation from Straight Line: Residual Vector : optimal black L true L Figure: High noise: LNB A 4 Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but smooth

37 Robustness of Parameter Choice: Solutions by NNLS using matrix A 4 : Minimum error red line, LC dotted blue and NCP green line (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example NNLS solutions..% noise, RQ-B data set, matrix A 4.

38 Observations: Results from NNLS with Parameter Choice. Nonnegativity of g(t) is maintained. 2. LN models are more robust for recovery by both LLS and NNLS 3. RP and LC are just as robust for NNLS as for LLS. 4. With high noise all solutions are smoothed (both by LLS and NNLS) Can we use overdetermined system with less resolution: A 3?

39 Measure Deviation from Straight Line: Residual Vector : optimal black L true L Figure: High noise: LNB A 3 Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but smooth

40 Results with matrix A (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example NNLS solutions..% noise. RQ-B data set matrix A 3

41 Observations on matrix A 3. Nonnegativity of g(t) is maintained. 2. Solutions are more noisy - potentially less ability to locate peaks in low noise data. 3. Matrix A 3 would be good for larger data sets that can offer greater resolution 4. Robustness again for determining the parameters by RP and LC 5. High noise is over smoothed

42 Augmented discrepancy principle Morozov discrepancy principle. For b = ˆb + η where noise η has noise variance σ 2. Discrepancy principle finds Ax b 2 = r m σ 2 RP looks at the entire vector r Augmented residual is defined for the entire system (but should be weighted) R = [Ax b; λlx] Discrepancy analysis again finds R 2 follows a χ 2 distribution The RP applies to vector R as for r.

43 Applying the RP for the augmented problem: High noise LNB L true L Figure: A 4 LS

44 Applying the RP for the augmented problem: High noise LNB L true L Figure: A 3 LS

45 Applying the RP for the augmented problem: High noise LNB L true L Figure: A 4 NNLS

46 Applying the RP for the augmented problem: High noise LNB L true L Figure: A 3 NNLS

47 Conclusions Overall ES Small Scale - standard parameter choice applies for NNLS Residual Periodogram extends to NN Residual Periodogram applies to augmented residual Augmented residual generates less smoothing Potential to use RP for stopping rule in iterative methods NL fitting insufficient, exact DRT form must be known Analysis of quadrature demonstrates minimal model error Transformation is a right preconditioning LLS : nformation on noise level would be helpful - allow other methods UPRE, χ 2 etc High noise all solutions over smoothed. Ability to detect multiple processes in presence of noise is challenging

48 Acknowledgements Authors Hansen, Hogue and Sander were supported by NSF CSUMS grant DMS 73587: CSUMS: Undergraduate Research Experiences for Computational Math Sciences Majors at ASU. Renaut was supported by NSF MCTP grant DMS 4877: MCTP: Mathematics Mentoring Partnership Between Arizona State University and the Maricopa County Community College District, NSF grant DMS 2655: Novel Numerical Approximation Techniques for Non-Standard Sampling Regimes, and AFOSR grant 2577 Development and Analysis of Non-Classical Numerical Approximation Methods. Popat was supported by ONR grant N42344: Characterizing electron transport resistances from anode-respiring bacteria using electrochemical techniques. Professor César Torres for discussions concerning the ES modeling.

49 RP for Augmented Residual Vector and LLS: optimal black L true L Figure: Low noise: A 4 LS LNB Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

50 RP for Augmented Residual Vector and LLS: optimal black L true L Figure: High noise: A 4 LS LNB Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but smooth

51 RP for Augmented Residual Vector and LLS: optimal black L true L Figure: Low noise: A 3 LS LNB Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

52 RP for Augmented Residual Vector and LLS: optimal black L true L Figure: High noise: A 3 LS LNB Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but smooth

53 RP for Augmented Residual Vector and NNLS: optimal black L true L Figure: Low noise: A 4 NNLS LNB Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

54 RP for Augmented Residual Vector and NNLS: optimal black L true L Figure: High noise: A 4 NNLS LNB Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but

55 RP for Augmented Residual Vector and NNLS: optimal black L true L Figure: Low noise: A 3 NNLS LNB Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

56 RP for Augmented Residual Vector and NNLS: optimal black L true L Figure: High noise: A 3 NNLS LNB Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but

57 RP for Augmented Residual Vector and LLS: optimal black L true L Figure: Low noise: A 4 LS RQA Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

58 RP for Augmented Residual Vector and LLS: optimal black L true L Figure: High noise: A 4 LS RQA Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but smooth

59 RP for Augmented Residual Vector and LLS: optimal black L true L Figure: Low noise: A 3 LS RQA Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

60 RP for Augmented Residual Vector and LLS: optimal black L true L Figure: High noise: A 3 LS RQA Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but smooth

61 RP for Augmented Residual Vector and NNLS: optimal black L true L Figure: Low noise: A 4 NNLS RQA Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

62 RP for Augmented Residual Vector and NNLS: optimal black L true L Figure: High noise: A 4 NNLS RQA Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but

63 RP for Augmented Residual Vector and NNLS: optimal black L true L Figure: Low noise: A 3 NNLS RQA Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual Solutions are nonnegative

64 RP for Augmented Residual Vector and NNLS: optimal black L true L Figure: High noise: A 3 NNLS RQA Testing for white noise: Calculate the NCP and measure the deviation from the white noise line for several λ. The vector with highest white noise content indicates transfer of noise to the residual: Solutions are nonnegative but

65 Results with matrix A 4 LLS (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example LLS solutions..% noise, LN-B data set, matrix A 4.

66 Results with matrix A 4 NNLS (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example NNLS solutions..% noise, LN-B data set, matrix A 4.

67 Results with matrix A 3 NNLS (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example NNLS solutions..% noise. LN-B data set matrix A 3

68 s the approach robust? Solutions by LLS using matrix A 4 : Minimum error red line, LC dotted blue and NCP green line (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example LLS solutions..% noise, RQ-A data set, matrix A 4.

69 Robustness of Parameter Choice: Solutions by NNLS using matrix A 4 : Minimum error red line, LC dotted blue and NCP green line (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example NNLS solutions..% noise, RQ-A data set, matrix A 4.

70 s the approach robust? Solutions by LLS using matrix A 4 : Minimum error red line, LC dotted blue and NCP green line (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example LLS solutions..% noise, LN-A data set, matrix A 4.

71 Robustness of Parameter Choice: Solutions by NNLS using matrix A 4 : Minimum error red line, LC dotted blue and NCP green line (a) L = (b) L = L (c) L = (d) L = (e) L = L (f) L = Figure: Mean error and example NNLS solutions..% noise, LN-A data set, matrix A 4.