Quis custodiet ipsos custodes?

Transcription

1 Quis custodiet ipsos custodes? James B. Rawlings, Megan Zagrobelny, Luo Ji Dept. of Chemical and Biological Engineering, Univ. of Wisconsin-Madison, WI, USA IFAC Conference on Nonlinear Model Predictive Control 2012 Noordwijkerhout, The Netherlands August 23-27, 2012 Rawlings/Zagrobelny/Ji Control performance monitoring 1 / 42

2 Outline 1 Motivation 2 Approach Distribution of Signals Distribution of Performance Metrics Extensions: Deterministic Disturbance, Plant/Model Mismatch Extension to Nonlinear Systems Variance Estimation 3 Conclusions and Future Work Rawlings/Zagrobelny/Ji Control performance monitoring 2 / 42

3 Motivation The process industries have recently invested millions of dollars in installing advanced Model Predictive Control (MPC) technology. Rawlings/Zagrobelny/Ji Control performance monitoring 3 / 42

4 Motivation The process industries have recently invested millions of dollars in installing advanced Model Predictive Control (MPC) technology. There is no established control theory and control technology, however, that enable practitioners to routinely monitor the operation of these MPC advanced control systems to address the following important, interrelated issues. Rawlings/Zagrobelny/Ji Control performance monitoring 3 / 42

5 Motivation The process industries have recently invested millions of dollars in installing advanced Model Predictive Control (MPC) technology. There is no established control theory and control technology, however, that enable practitioners to routinely monitor the operation of these MPC advanced control systems to address the following important, interrelated issues. How well are the various MPC systems performing, i.e., are they achieving the goals. Rawlings/Zagrobelny/Ji Control performance monitoring 3 / 42

6 Motivation The process industries have recently invested millions of dollars in installing advanced Model Predictive Control (MPC) technology. There is no established control theory and control technology, however, that enable practitioners to routinely monitor the operation of these MPC advanced control systems to address the following important, interrelated issues. How well are the various MPC systems performing, i.e., are they achieving the goals. Provide a rank ordering of up to hundreds of MPC loops in some number of ranked categories from best to worst Rawlings/Zagrobelny/Ji Control performance monitoring 3 / 42

7 Motivation The process industries have recently invested millions of dollars in installing advanced Model Predictive Control (MPC) technology. There is no established control theory and control technology, however, that enable practitioners to routinely monitor the operation of these MPC advanced control systems to address the following important, interrelated issues. How well are the various MPC systems performing, i.e., are they achieving the goals. Provide a rank ordering of up to hundreds of MPC loops in some number of ranked categories from best to worst Have the performances of the MPC systems changed over time Rawlings/Zagrobelny/Ji Control performance monitoring 3 / 42

8 Motivation The process industries have recently invested millions of dollars in installing advanced Model Predictive Control (MPC) technology. There is no established control theory and control technology, however, that enable practitioners to routinely monitor the operation of these MPC advanced control systems to address the following important, interrelated issues. How well are the various MPC systems performing, i.e., are they achieving the goals. Provide a rank ordering of up to hundreds of MPC loops in some number of ranked categories from best to worst Have the performances of the MPC systems changed over time Do some of the MPC systems require maintenance; if so, which ones Rawlings/Zagrobelny/Ji Control performance monitoring 3 / 42

9 Literature Overview Minimum Variance Minimum variance method for SISO systems marks a starting point for modern controller performance monitoring (Harris, 1989). Rawlings/Zagrobelny/Ji Control performance monitoring 4 / 42

10 Literature Overview Minimum Variance Minimum variance method for SISO systems marks a starting point for modern controller performance monitoring (Harris, 1989). Åström (1970) laid the groundwork for the MV method and predicted its use in controller monitoring. Rawlings/Zagrobelny/Ji Control performance monitoring 4 / 42

11 Literature Overview Minimum Variance Minimum variance method for SISO systems marks a starting point for modern controller performance monitoring (Harris, 1989). Åström (1970) laid the groundwork for the MV method and predicted its use in controller monitoring. Kammer et al. (1998) test for LQ optimality by perturbing the closed-loop system and monitoring the input and output. Rawlings/Zagrobelny/Ji Control performance monitoring 4 / 42

12 Literature Overview Minimum Variance Minimum variance method for SISO systems marks a starting point for modern controller performance monitoring (Harris, 1989). Åström (1970) laid the groundwork for the MV method and predicted its use in controller monitoring. Kammer et al. (1998) test for LQ optimality by perturbing the closed-loop system and monitoring the input and output. Extension to multivariable systems attempted: (Harris et al., 1996; Qin and Yu, 2007). Rawlings/Zagrobelny/Ji Control performance monitoring 4 / 42

13 Literature Overview Extensions to Minimum Variance Minimum variance is generally an undesirable control performance goal. Zero penalty on the input leads to overly aggressive control. Rawlings/Zagrobelny/Ji Control performance monitoring 5 / 42

14 Literature Overview Extensions to Minimum Variance Minimum variance is generally an undesirable control performance goal. Zero penalty on the input leads to overly aggressive control. Alternative performance metrics have been developed: Huang and Shah (1998); Xu et al. (2006); Horch and Isaksson (1999) Rawlings/Zagrobelny/Ji Control performance monitoring 5 / 42

15 Literature Overview Extensions to Minimum Variance Minimum variance is generally an undesirable control performance goal. Zero penalty on the input leads to overly aggressive control. Alternative performance metrics have been developed: Huang and Shah (1998); Xu et al. (2006); Horch and Isaksson (1999) Subspace projections provide a means to identify the directions with poorest performance and isolate the effects of measured and unmeasured disturbances (McNabb and Qin, 2003, 2005). Rawlings/Zagrobelny/Ji Control performance monitoring 5 / 42

16 Literature Overview Extensions to Minimum Variance Minimum variance is generally an undesirable control performance goal. Zero penalty on the input leads to overly aggressive control. Alternative performance metrics have been developed: Huang and Shah (1998); Xu et al. (2006); Horch and Isaksson (1999) Subspace projections provide a means to identify the directions with poorest performance and isolate the effects of measured and unmeasured disturbances (McNabb and Qin, 2003, 2005). Yu and Qin (2008a,b) extracted the directions with changes in control performance through generalized eigenvalue analysis and identified the responsible loops by calculating the contribution of each loop to these directions. Rawlings/Zagrobelny/Ji Control performance monitoring 5 / 42

17 Literature Overview Extensions to Minimum Variance Minimum variance is generally an undesirable control performance goal. Zero penalty on the input leads to overly aggressive control. Alternative performance metrics have been developed: Huang and Shah (1998); Xu et al. (2006); Horch and Isaksson (1999) Subspace projections provide a means to identify the directions with poorest performance and isolate the effects of measured and unmeasured disturbances (McNabb and Qin, 2003, 2005). Yu and Qin (2008a,b) extracted the directions with changes in control performance through generalized eigenvalue analysis and identified the responsible loops by calculating the contribution of each loop to these directions. A natural extension to the MV benchmark, the linear-quadratic-gaussian (LQG) benchmark, identifies the minimum output variance possible when the input variance does not exceed a certain threshold (Huang and Shah, 1999, chap. 13). Rawlings/Zagrobelny/Ji Control performance monitoring 5 / 42

18 Literature Overview MPC Monitoring Patwardhan (1999) proposed using the MPC design objective as the performance measure. Using this concept, Julien et al. (2004) developed plots of the best achievable MPC performance. Rawlings/Zagrobelny/Ji Control performance monitoring 6 / 42

19 Literature Overview MPC Monitoring Patwardhan (1999) proposed using the MPC design objective as the performance measure. Using this concept, Julien et al. (2004) developed plots of the best achievable MPC performance. Schafer and Cinar (2004) used a ratio of historic and achieved performance to detect changes in controller performance and diagnosed the cause of any changes by using a ratio of achieved and designed performance for diagnosis. Rawlings/Zagrobelny/Ji Control performance monitoring 6 / 42

20 Literature Overview MPC Monitoring Patwardhan (1999) proposed using the MPC design objective as the performance measure. Using this concept, Julien et al. (2004) developed plots of the best achievable MPC performance. Schafer and Cinar (2004) used a ratio of historic and achieved performance to detect changes in controller performance and diagnosed the cause of any changes by using a ratio of achieved and designed performance for diagnosis. Other research focused on diagnosing the underlying causes of poor MPC performance. Patwardhan and Shah (2002) gave bounds on the effects of constraints, modeling uncertainty, disturbance uncertainty, and nonlinearity on MPC performance. Rawlings/Zagrobelny/Ji Control performance monitoring 6 / 42

21 Building a Monitoring System Three Fundamental Relationships central limit theorem = normal distribution Rawlings/Zagrobelny/Ji Control performance monitoring 7 / 42

22 Building a Monitoring System Three Fundamental Relationships central limit theorem = normal distribution normal + linear system = normal distribution Rawlings/Zagrobelny/Ji Control performance monitoring 7 / 42

23 Building a Monitoring System Three Fundamental Relationships central limit theorem = normal distribution normal + linear system = normal distribution normal + quadratic stage cost = chi-squared distribution Rawlings/Zagrobelny/Ji Control performance monitoring 7 / 42

24 Initial Assumptions central limit theorem = normal distribution Assume process and measurement noises have normal distribution Composed of many random unmodeled disturbances Justified by central limit theorem Rawlings/Zagrobelny/Ji Control performance monitoring 8 / 42

25 Initial Assumptions central limit theorem = normal distribution Assume process and measurement noises have normal distribution Composed of many random unmodeled disturbances Justified by central limit theorem Choose linear input to output process model Convenient for user Close to operating point Identified from data Rawlings/Zagrobelny/Ji Control performance monitoring 8 / 42

26 Normal Distribution of Signals normal + linear system = normal distribution Assume linear model with normally distributed disturbances Rawlings/Zagrobelny/Ji Control performance monitoring 9 / 42

27 Normal Distribution of Signals normal + linear system = normal distribution Assume linear model with normally distributed disturbances Basic statistics: linear transformations of these disturbances are also normally distributed Rawlings/Zagrobelny/Ji Control performance monitoring 9 / 42

28 Normal Distribution of Signals normal + linear system = normal distribution Assume linear model with normally distributed disturbances Basic statistics: linear transformations of these disturbances are also normally distributed For the linear model, all signals in the loop are linear transformations of normal variables Rawlings/Zagrobelny/Ji Control performance monitoring 9 / 42

29 Distribution of Stage Cost normal + quadratic stage cost = chi-squared distribution Want to characterize the distribution of the regulator s stage cost. Why? Optimized by MPC controller Serves as benchmark for monitoring/assessment Rawlings/Zagrobelny/Ji Control performance monitoring 10 / 42

30 Distribution of Stage Cost normal + quadratic stage cost = chi-squared distribution Want to characterize the distribution of the regulator s stage cost. Why? Optimized by MPC controller Serves as benchmark for monitoring/assessment Stage cost is a quadratic function of signals from the process Rawlings/Zagrobelny/Ji Control performance monitoring 10 / 42

31 Distribution of Stage Cost normal + quadratic stage cost = chi-squared distribution Want to characterize the distribution of the regulator s stage cost. Why? Optimized by MPC controller Serves as benchmark for monitoring/assessment Stage cost is a quadratic function of signals from the process All signals are normal variables Rawlings/Zagrobelny/Ji Control performance monitoring 10 / 42

32 Distribution of Stage Cost normal + quadratic stage cost = chi-squared distribution Want to characterize the distribution of the regulator s stage cost. Why? Optimized by MPC controller Serves as benchmark for monitoring/assessment Stage cost is a quadratic function of signals from the process All signals are normal variables Stage cost has generalized chi-squared distribution Rawlings/Zagrobelny/Ji Control performance monitoring 10 / 42

33 Monitoring Framework Define Key Performance Indices (KPI) Sample average of stage cost from data Expectation of (several) ideal stage cost(s) from model Rawlings/Zagrobelny/Ji Control performance monitoring 11 / 42

34 Monitoring Framework Define Key Performance Indices (KPI) Sample average of stage cost from data Expectation of (several) ideal stage cost(s) from model Understand distributions Rawlings/Zagrobelny/Ji Control performance monitoring 11 / 42

35 Monitoring Framework Define Key Performance Indices (KPI) Sample average of stage cost from data Expectation of (several) ideal stage cost(s) from model Understand distributions Develop confidence intervals Rawlings/Zagrobelny/Ji Control performance monitoring 11 / 42

36 Monitoring Framework Define Key Performance Indices (KPI) Sample average of stage cost from data Expectation of (several) ideal stage cost(s) from model Understand distributions Develop confidence intervals Design comparison tests Rawlings/Zagrobelny/Ji Control performance monitoring 11 / 42

37 Offset free control system x + = A x + Bũ (Q, R) regulator u plant y x s u s ˆx estimator y sp, u sp, r sp (Q s, R s ) target selector ˆx ˆd [ ] + x = d [ ] [ A Bd x 0 I d] y = [ ] [ x C C d d (Q w, R v) [ B + u + w 0] MPC controller consisting of: receding horizon regulator, state estimator, and target selector. ] + v Rawlings/Zagrobelny/Ji Control performance monitoring 12 / 42

38 Linear System Model Process x + = Ax + Bu + B d d + Gw d + = d + G d w d y = Cx + C d d + v Estimator ˆx + = Aˆx + AL x ε + Bu + B d ˆd ˆd + = ˆd + L d ε ε = y C ˆx C d ˆd Rawlings/Zagrobelny/Ji Control performance monitoring 13 / 42

39 Linear System Model Process x + = Ax + Bu + B d d + Gw d + = d + G d w d y = Cx + C d d + v Estimator ˆx + = Aˆx + AL x ε + Bu + B d ˆd ˆd + = ˆd + L d ε ε = y C ˆx C d ˆd State x includes past inputs (u k 1 ) Rawlings/Zagrobelny/Ji Control performance monitoring 13 / 42

40 Linear System Model Process x + = Ax + Bu + B d d + Gw d + = d + G d w d y = Cx + C d d + v Estimator ˆx + = Aˆx + AL x ε + Bu + B d ˆd ˆd + = ˆd + L d ε ε = y C ˆx C d ˆd State x includes past inputs (u k 1 ) Process noise, w; measurement noise v Rawlings/Zagrobelny/Ji Control performance monitoring 13 / 42

41 Linear System Model Process x + = Ax + Bu + B d d + Gw d + = d + G d w d y = Cx + C d d + v Estimator ˆx + = Aˆx + AL x ε + Bu + B d ˆd ˆd + = ˆd + L d ε ε = y C ˆx C d ˆd State x includes past inputs (u k 1 ) Process noise, w; measurement noise v Nonstationary disturbance d with noise w d Rawlings/Zagrobelny/Ji Control performance monitoring 13 / 42

42 Linear System Model Process x + = Ax + Bu + B d d + Gw d + = d + G d w d y = Cx + C d d + v Estimator ˆx + = Aˆx + AL x ε + Bu + B d ˆd ˆd + = ˆd + L d ε ε = y C ˆx C d ˆd State x includes past inputs (u k 1 ) Process noise, w; measurement noise v Nonstationary disturbance d with noise w d All noises are normally distributed and independent Rawlings/Zagrobelny/Ji Control performance monitoring 13 / 42

43 Linear Closed Loop Model Apply optimal (linear) feedback law u(k) = K ] [ˆx(k) + Lx ε(k) + u(k 1) ( ) [ ] [0 ] xs I K u s Rawlings/Zagrobelny/Ji Control performance monitoring 14 / 42

44 Linear Closed Loop Model Apply optimal (linear) feedback law u(k) = K ] [ˆx(k) + Lx ε(k) + u(k 1) ( ) [ ] [0 ] xs I K u s Define augmented state and noise ˆx z = ˆd w ˆx x w = w d v ˆd d Any signal in the loop is a linear transformation of z and w Rawlings/Zagrobelny/Ji Control performance monitoring 14 / 42

45 Linear Closed-loop Model Write closed-loop system in terms of augmented state z + = Ãz + G w + z d Rawlings/Zagrobelny/Ji Control performance monitoring 15 / 42

46 Linear Closed-loop Model Write closed-loop system in terms of augmented state z + = Ãz + G w + z d Ã, G depend on System matrices Regulator gains Estimator gains Rawlings/Zagrobelny/Ji Control performance monitoring 15 / 42

47 Linear Closed-loop Model Write closed-loop system in terms of augmented state z + = Ãz + G w + z d Ã, G depend on System matrices Regulator gains Estimator gains z d reflects nonzero setpoints in y or u Rawlings/Zagrobelny/Ji Control performance monitoring 15 / 42

48 Distribution of z Denote distribution of w w N (0, Q w ) Rawlings/Zagrobelny/Ji Control performance monitoring 16 / 42

49 Distribution of z Denote distribution of w w N (0, Q w ) z(k) also has normal distribution z(k) N(m(k), S(k)) m + = Ãm + z d S + = ÃSÃ + GQ w G Rawlings/Zagrobelny/Ji Control performance monitoring 16 / 42

50 Time-Invariant Distribution of z Given z(k) has normal distribution z(k) N(m(k), S(k)) m + = Ãm + z d S + = ÃSÃ + GQ w G Rawlings/Zagrobelny/Ji Control performance monitoring 17 / 42

51 Time-Invariant Distribution of z Given z(k) has normal distribution z(k) N(m(k), S(k)) m + = Ãm + z d S + = ÃSà + GQ w G Assume à is stable Rawlings/Zagrobelny/Ji Control performance monitoring 17 / 42

52 Time-Invariant Distribution of z Given z(k) has normal distribution z(k) N(m(k), S(k)) m + = Ãm + z d S + = ÃSà + GQ w G Assume à is stable For k large enough, z has steady-state distribution z N(m, S ) m = (I Ã) 1 z d S = ÃS à + GQ w G Rawlings/Zagrobelny/Ji Control performance monitoring 17 / 42

53 Stage Cost and KPI Choose stage cost l(k) = l (x(k), u(k)) = y(k) y sp 2 Q y + u(k) u sp 2 R + u(k) u(k 1) 2 S Rawlings/Zagrobelny/Ji Control performance monitoring 18 / 42

54 Stage Cost and KPI Choose stage cost l(k) = l (x(k), u(k)) = y(k) y sp 2 Q y + u(k) u sp 2 R + u(k) u(k 1) 2 S Key Performance Index (KPI) Ideal/theoretical KPI E (l(k)) Calculate from model Rawlings/Zagrobelny/Ji Control performance monitoring 18 / 42

55 Stage Cost and KPI Choose stage cost l(k) = l (x(k), u(k)) = y(k) y sp 2 Q y + u(k) u sp 2 R + u(k) u(k 1) 2 S Key Performance Index (KPI) Ideal/theoretical KPI E (l(k)) Calculate from model Actual/plant KPI l(k) = 1 k 1 k j=0 l(j) Calculate from data Rawlings/Zagrobelny/Ji Control performance monitoring 18 / 42

56 Distribution of Stage Cost Goals Determine theoretical KPI Develop confidence intervals for comparison with plant KPI Rawlings/Zagrobelny/Ji Control performance monitoring 19 / 42

57 Distribution of Stage Cost Goals Determine theoretical KPI Develop confidence intervals for comparison with plant KPI Approach Treat stage cost as quadratic form of normal signal Express distribution as generalized chi-squared Consider sample averages Rawlings/Zagrobelny/Ji Control performance monitoring 19 / 42

58 Distribution of Stage Cost Express stage cost as quadratic form l(k) = y(k) y sp u(k) u sp (u(k) u(k 1)) Q = diag(q y, R, S) Q y(k) y sp u(k) u sp (u(k) u(k 1)) Rawlings/Zagrobelny/Ji Control performance monitoring 20 / 42

59 Distribution of Stage Cost Express stage cost as quadratic form l(k) = y(k) y sp u(k) u sp (u(k) u(k 1)) Q = diag(q y, R, S) Q y(k) y sp u(k) u sp (u(k) u(k 1)) Write in terms of augmented state z and measurement noise v y(k) y sp u(k) u sp = F 1 z(k) + F 2 v(k) + F 3 u d m sp (u(k) u(k 1)) msp is vector of setpoints ud arises from nonzero setpoints F1, F 2, and F 3 from system matrices and regulator gain Rawlings/Zagrobelny/Ji Control performance monitoring 20 / 42

60 Distribution of Stage Cost Signal has normal distribution y(k) y sp u(k) u sp N( m(k), P(k)) u(k) u(k 1) m(k) = F 1 m(k) + F 3 u d m sp P(k) = F 1 S(k)F 1 + F 2 R v F 2 Rawlings/Zagrobelny/Ji Control performance monitoring 21 / 42

61 Distribution of Stage Cost Signal has normal distribution y(k) y sp u(k) u sp N( m(k), P(k)) u(k) u(k 1) m(k) = F 1 m(k) + F 3 u d m sp P(k) = F 1 S(k)F 1 + F 2 R v F 2 Stage cost has generalized chi-squared distribution E(l(k)) = tr( QP(k)) + m(k) Q m(k) var(l(k)) = 2tr( QP(k) QP(k)) + 4 m(k) QP(K) Q m(k) No simple explicit expression for pdf l(k) has steady-state distribution for large k and stable system Rawlings/Zagrobelny/Ji Control performance monitoring 21 / 42

62 Distribution of Plant KPI Sample average of stage costs l(k) = 1 (l(0) + l(1) + + l(k 1)) k Rawlings/Zagrobelny/Ji Control performance monitoring 22 / 42

63 Distribution of Plant KPI Sample average of stage costs l(k) = 1 (l(0) + l(1) + + l(k 1)) k Sample average of random variables Rawlings/Zagrobelny/Ji Control performance monitoring 22 / 42

64 Distribution of Plant KPI Sample average of stage costs l(k) = 1 (l(0) + l(1) + + l(k 1)) k Sample average of random variables Apply central limit theorem Approaches normal distribution Careful! Samples are correlated Unbiased estimate of stage cost: E ( l(k) ) = E (l(k)) Variance depends on variance of stage cost, sample size, and correlation Rawlings/Zagrobelny/Ji Control performance monitoring 22 / 42

65 KPI Convergence KPI Plant KPI Expected KPI Time Theoretical and plant KPI for perfect model Compared simulated data to theoretical KPI As time (points in sample average) increases, KPI converges to expected value Rawlings/Zagrobelny/Ji Control performance monitoring 23 / 42

66 Stage Cost Distribution Frequency l(k) Histogram of stage costs using 1000 samples Mean Variance Sample Theoretical Red lines show standard deviations away from sample average Green lines show bins containing 95% of the points Good agreement between theoretical and sample statistics Rawlings/Zagrobelny/Ji Control performance monitoring 24 / 42

67 Stage Cost Sample Mean Distribution Frequency l(k) Histogram of stage cost sample mean for 1000 simulations Used 1000 independent simulations Calculated KPI (sample average of stage cost) for each simulation Approaches Gaussian distribution Statistics depend on distribution of l(k), sample size, and correlations Rawlings/Zagrobelny/Ji Control performance monitoring 25 / 42

68 Deterministic Disturbance: Model Output is affected by deterministic disturbance p y = Cx + p + v Rawlings/Zagrobelny/Ji Control performance monitoring 26 / 42

69 Deterministic Disturbance: Model Output is affected by deterministic disturbance p y = Cx + p + v Rewrite closed-loop model to account for disturbance z + = Ãz + B p p + Gw + z d Rawlings/Zagrobelny/Ji Control performance monitoring 26 / 42

70 Deterministic Disturbance: Model Output is affected by deterministic disturbance p y = Cx + p + v Rewrite closed-loop model to account for disturbance z + = Ãz + B p p + Gw + z d Affects expectation of z but not its variance m + = Ãm + B p p + z d S + = ÃS Ã + GQ w G Rawlings/Zagrobelny/Ji Control performance monitoring 26 / 42

71 Deterministic Disturbance: Model Output is affected by deterministic disturbance p y = Cx + p + v Rewrite closed-loop model to account for disturbance z + = Ãz + B p p + Gw + z d Affects expectation of z but not its variance m + = Ãm + B p p + z d S + = ÃS Ã + GQ w G No steady-state expectation of z Rawlings/Zagrobelny/Ji Control performance monitoring 26 / 42

72 Deterministic Disturbance: KPI Assume the deterministic disturbance is periodic and average E(l ) = 1 T k+t 1 k E(l(k)) Rawlings/Zagrobelny/Ji Control performance monitoring 27 / 42

73 Deterministic Disturbance: KPI Assume the deterministic disturbance is periodic and average E(l ) = 1 T k+t 1 k E(l(k)) Signal still has normal distribution y(k) y sp u(k) u sp N( m(k), P(k)) u(k) u(k 1) m(k) = F 1 m(k) + F 2 p(k) + F 3 u d m sp P = F 1 S F 1 + F 2 R v F 2 Rawlings/Zagrobelny/Ji Control performance monitoring 27 / 42

74 Deterministic Disturbance: KPI Assume the deterministic disturbance is periodic and average E(l ) = 1 T k+t 1 k E(l(k)) Signal still has normal distribution y(k) y sp u(k) u sp N( m(k), P(k)) u(k) u(k 1) m(k) = F 1 m(k) + F 2 p(k) + F 3 u d m sp P = F 1 S F 1 + F 2 R v F 2 Expected KPI has constant and transient term E(l ) = tr( QP) + 1 T m(k) Q m(k) T k=1 Rawlings/Zagrobelny/Ji Control performance monitoring 27 / 42

75 Deterministic Disturbance Example KPI Plant KPI Expected KPI Expected KPI (with dist) Time Theoretical and plant KPI for deterministic disturbance disturbance Time Deterministic disturbance affecting system Plant KPI increases due to presence of deterministic disturbance Converges to expected value when disturbance is accounted for Rawlings/Zagrobelny/Ji Control performance monitoring 28 / 42

76 Plant-Model Mismatch Assume linear plant and linear model Rawlings/Zagrobelny/Ji Control performance monitoring 29 / 42

77 Plant-Model Mismatch Assume linear plant and linear model What will happen to KPI if model is different from plant? Rawlings/Zagrobelny/Ji Control performance monitoring 29 / 42

78 Plant-Model Mismatch Assume linear plant and linear model What will happen to KPI if model is different from plant? Calculate expected KPI if amount of mismatch is known Rawlings/Zagrobelny/Ji Control performance monitoring 29 / 42

79 Plant-Model Mismatch Assume linear plant and linear model What will happen to KPI if model is different from plant? Calculate expected KPI if amount of mismatch is known Useful for case studies, but cannot calculate in practice! Rawlings/Zagrobelny/Ji Control performance monitoring 29 / 42

80 Plant-Model Mismatch Example KPI Plant KPI Expected KPI Expected KPI (with mismatch) Time Theoretical and plant KPI in presence of plant-model mismatch Plant KPI increases due to presence of mismatch Converges to new theoretical value In practice cannot calculate theoretical value Rawlings/Zagrobelny/Ji Control performance monitoring 30 / 42

81 Constrained and Nonlinear MPC Consider linear system with nonlinear control law (caused by constraints) or nonlinear system central limit theorem = normal distribution normal + linear system = normal distribution normal + quadratic stage cost = chi-squared distribution Rawlings/Zagrobelny/Ji Control performance monitoring 31 / 42

82 Constrained and Nonlinear MPC Consider linear system with nonlinear control law (caused by constraints) or nonlinear system central limit theorem = normal distribution normal + linear system = normal distribution normal + quadratic stage cost = chi-squared distribution Second step does not hold because system is not linear Rawlings/Zagrobelny/Ji Control performance monitoring 31 / 42

83 Constrained and Nonlinear MPC Consider linear system with nonlinear control law (caused by constraints) or nonlinear system central limit theorem = normal distribution normal + linear system = normal distribution normal + quadratic stage cost = chi-squared distribution Second step does not hold because system is not linear Cannot derive convenient formulas Rawlings/Zagrobelny/Ji Control performance monitoring 31 / 42

84 Constrained and Nonlinear MPC Consider linear system with nonlinear control law (caused by constraints) or nonlinear system central limit theorem = normal distribution normal + linear system = normal distribution normal + quadratic stage cost = chi-squared distribution Second step does not hold because system is not linear Cannot derive convenient formulas Instead use model and rapid simulation to compute statistics and confidence intervals for benchmark Rawlings/Zagrobelny/Ji Control performance monitoring 31 / 42

85 Example: Constrained System KPI l(k) ( u 1) l(k) ( u 5) l(k) ( u 10) l(k) (uncons) E(l(k)) (uncons) Time Simulated KPI for linear constrained system KPI increases due to presence of constraints Tightening constraints causes greater increase Converges to constant value in each case Use simulation of model to develop ideal KPI Rawlings/Zagrobelny/Ji Control performance monitoring 32 / 42

86 Variance Estimation and ALS Need to know variance of noises affecting the system in order to develop appropriate benchmark x + = Ax + Gw y = Cx + v Rawlings/Zagrobelny/Ji Control performance monitoring 33 / 42

87 Variance Estimation and ALS Need to know variance of noises affecting the system in order to develop appropriate benchmark x + = Ax + Gw y = Cx + v Autocovariance least squares (ALS); Odelson et al. (2006); Rajamani and Rawlings (2009) Separately identifies process and measurement noise covariances Formulates least squares problem using process model and measurement correlations in data Rawlings/Zagrobelny/Ji Control performance monitoring 33 / 42

88 Variance Estimation and ALS Need to know variance of noises affecting the system in order to develop appropriate benchmark x + = Ax + Gw y = Cx + v Autocovariance least squares (ALS); Odelson et al. (2006); Rajamani and Rawlings (2009) Separately identifies process and measurement noise covariances Formulates least squares problem using process model and measurement correlations in data Alternative: estimate innovation variance and Kalman gain Rawlings/Zagrobelny/Ji Control performance monitoring 33 / 42

89 Obtaining Q and R from Data Xf, Ff X D X 1 X 2 Condenser y 1 Model discretized with t k = k t: d dt X D. X B }{{} x(t) [ ] y 1 (t k ) = y 2 = F (x(t), u(t) ) }{{} X f,f f [ ] 1 0 x(t 0 1 k ) X N 1 X B Reboiler Measurements are only X D, X B at the discretization times y 2 Rawlings/Zagrobelny/Ji Control performance monitoring 34 / 42

90 Obtaining Q and R from Data X D X 1 X 2 Condenser y 1 Model discretized with t k = k t: x k+1 = f (x k, u k ) + g(x k, u k )w k [ ] [ ] y y 2 = x 0 1 k k Xf, Ff w X N 1 X B Reboiler Measurements are only X D, X B at the discretization times Noise w k affects all the states y 2 Rawlings/Zagrobelny/Ji Control performance monitoring 34 / 42

91 Obtaining Q and R from Data X D X 1 X 2 Condenser v 1 y 1 Model discretized with t k = k t: x k+1 = f (x k, u k ) + g(x k, u k )w k [ ] [ ] [ ] y v 1 y 2 = x 0 1 k + v 2 k k Xf, Ff w v 2 y 2 X N 1 X B Reboiler Measurements are only X D, X B at the discretization times Noise w k affects all the states Noise v k corrupts the measurements Rawlings/Zagrobelny/Ji Control performance monitoring 34 / 42

92 Motivation for Using Autocovariances Xf, Ff X D X 1 X 2 Condenser v 1 y 1 Idea of Autocovariances The state noise w k is propagated in time by the state The measurement noise v k appears only at the sample time and is not propagated in time w X N 1 X B Taking autocovariances of data at different time lags gives covariances of w k and v k v 2 Reboiler y 2 Rawlings/Zagrobelny/Ji Control performance monitoring 35 / 42

93 Motivation for Using Autocovariances Xf, Ff X D X 1 X 2 Condenser v 1 y 1 Idea of Autocovariances The state noise w k is propagated in time by the state The measurement noise v k appears only at the sample time and is not propagated in time w X N 1 X B Taking autocovariances of data at different time lags gives covariances of w k and v k v 2 y 2 Reboiler Let w k, v k have zero means and covariances Q and R Rawlings/Zagrobelny/Ji Control performance monitoring 35 / 42

94 Mathematical Formulation of the ALS Linear State-Space Model: x k+1 = Ax k + Gw k w k N(0, Q) y k = Cx k + v k v k N(0, R) Model (A, C, G) known from the linearization, finite set of measurements: {y 0,..., y k } given. Only unknowns are noises w k and v k. Rawlings/Zagrobelny/Ji Control performance monitoring 36 / 42

95 Mathematical Formulation of the ALS Linear State-Space Model: x k+1 = Ax k + Gw k w k N(0, Q) y k = Cx k + v k v k N(0, R) Model (A, C, G) known from the linearization, finite set of measurements: {y 0,..., y k } given. Only unknowns are noises w k and v k. y k = Cx k y k+1 = CAx k + CGw k y k+2 = CA 2 x k +CAGw k +CGw k+1 Rawlings/Zagrobelny/Ji Control performance monitoring 36 / 42

96 Mathematical Formulation of the ALS Linear State-Space Model: x k+1 = Ax k + Gw k w k N(0, Q) y k = Cx k + v k v k N(0, R) Model (A, C, G) known from the linearization, finite set of measurements: {y 0,..., y k } given. Only unknowns are noises w k and v k. y k = Cx k + v k y k+1 = CAx k + CGw k + v k+1 y k+2 = CA 2 x k +CAGw k +CGw k+1 +v k+2 Rawlings/Zagrobelny/Ji Control performance monitoring 36 / 42

97 Mathematical Formulation of the ALS Linear State-Space Model: x k+1 = Ax k + Gw k w k N(0, Q) y k = Cx k + v k v k N(0, R) Model (A, C, G) known from the linearization, finite set of measurements: {y 0,..., y k } given. Only unknowns are noises w k and v k. y k = Cx k + v k y k+1 = CAx k + CGw k + v k+1 E[y k y T k ] = R y k+2 = CA 2 x k +CAGw k +CGw k+1 +v k+2 Rawlings/Zagrobelny/Ji Control performance monitoring 36 / 42

98 Mathematical Formulation of the ALS Linear State-Space Model: x k+1 = Ax k + Gw k w k N(0, Q) y k = Cx k + v k v k N(0, R) Model (A, C, G) known from the linearization, finite set of measurements: {y 0,..., y k } given. Only unknowns are noises w k and v k. y k = Cx k + v k y k+1 = CAx k + CGw k + v k+1 y k+2 = CA 2 x k +CAGw k +CGw k+1 +v k+2 E[y k y T k ] = R E[y k+2 y T k+1 ] = CAGQG T C T Rawlings/Zagrobelny/Ji Control performance monitoring 36 / 42

99 The Autocovariance Least-Squares (ALS) Problem Skipping a lot of algebra, we can write: Autocovariance Least Squares [ ] Φ = min Q,R A (Q)s N ˆb (R) s 2 1 A least-squares problem in a vector of unknowns, Q, R Rawlings/Zagrobelny/Ji Control performance monitoring 37 / 42

100 The Autocovariance Least-Squares (ALS) Problem Skipping a lot of algebra, we can write: Autocovariance Least Squares [ ] Φ = min Q,R A (Q)s N ˆb (R) s 2 1 A least-squares problem in a vector of unknowns, Q, R 2 Form A N from known system matrices Rawlings/Zagrobelny/Ji Control performance monitoring 37 / 42

101 The Autocovariance Least-Squares (ALS) Problem Skipping a lot of algebra, we can write: Autocovariance Least Squares [ ] Φ = min Q,R A (Q)s N ˆb (R) s 2 1 A least-squares problem in a vector of unknowns, Q, R 2 Form A N from known system matrices 3 ˆb is a vector containing the estimated correlations from data ˆb = 1 y T k y T k T. k=1 y k+n 1 yk T s Rawlings/Zagrobelny/Ji Control performance monitoring 37 / 42

102 Conclusions For the linear, unconstrained system we have closed-form probability densities of all variables of interest: manipulated variable, controlled variable, measurement, state (and disturbance) estimates, stage cost, etc. Rawlings/Zagrobelny/Ji Control performance monitoring 38 / 42

103 Conclusions For the linear, unconstrained system we have closed-form probability densities of all variables of interest: manipulated variable, controlled variable, measurement, state (and disturbance) estimates, stage cost, etc. It is then straightforward to derive any statistic of interest for monitoring purposes. Rawlings/Zagrobelny/Ji Control performance monitoring 38 / 42

104 Conclusions For the linear, unconstrained system we have closed-form probability densities of all variables of interest: manipulated variable, controlled variable, measurement, state (and disturbance) estimates, stage cost, etc. It is then straightforward to derive any statistic of interest for monitoring purposes. The estimation of noise variances from data is an essential step in monitoring (as well as state estimation). Rawlings/Zagrobelny/Ji Control performance monitoring 38 / 42

105 Conclusions For the linear, unconstrained system we have closed-form probability densities of all variables of interest: manipulated variable, controlled variable, measurement, state (and disturbance) estimates, stage cost, etc. It is then straightforward to derive any statistic of interest for monitoring purposes. The estimation of noise variances from data is an essential step in monitoring (as well as state estimation). When controller constraints are active or the process model is nonlinear, we lose closed-form probability densities for the variables of interest. Rawlings/Zagrobelny/Ji Control performance monitoring 38 / 42

106 Conclusions For the linear, unconstrained system we have closed-form probability densities of all variables of interest: manipulated variable, controlled variable, measurement, state (and disturbance) estimates, stage cost, etc. It is then straightforward to derive any statistic of interest for monitoring purposes. The estimation of noise variances from data is an essential step in monitoring (as well as state estimation). When controller constraints are active or the process model is nonlinear, we lose closed-form probability densities for the variables of interest. But we can generate (by online computation) statistics of interest for controller monitoring purposes. Rawlings/Zagrobelny/Ji Control performance monitoring 38 / 42

107 Future Outlook The timing may be ideal for vendors to implement these monitoring ideas in products to enhance the value of their MPC software product offerings. Rawlings/Zagrobelny/Ji Control performance monitoring 39 / 42

108 Future Outlook The timing may be ideal for vendors to implement these monitoring ideas in products to enhance the value of their MPC software product offerings. Researchers should remain involved so that the emerging technology takes advantage of the best available supporting theory. Rawlings/Zagrobelny/Ji Control performance monitoring 39 / 42

109 Further reading I K. J. Åström. Introduction to Stochastic Control Theory. Academic Press, San Diego, California, T. J. Harris. Assessment of control loop performance. Can. J. Chem. Eng., 67: , T. J. Harris, F. Boudreau, and J. F. MacGregor. Performance assessment of multivariate feedback controllers. Automatica, 32(11): , A. Horch and A. J. Isaksson. A modified index for control performance assessment. J. Proc. Cont., 9(6): , B. Huang and S. L. Shah. Practical issues in multivariable feedback control performance assessment. J. Proc. Cont., 8(5 6): , B. Huang and S. L. Shah. Performance Assessment of Control Loops. Springer-Verlag, London, R. H. Julien, M. W. Foley, and W. R. Cluett. Performance assessment using a model predictive control benchmark. J. Proc. Cont., 14(4): , L. C. Kammer, R. R. Bitmead, and P. L. Bartlett. Optimal controller properties from closed-loop experiments. Automatica, 34(1):83 91, Rawlings/Zagrobelny/Ji Control performance monitoring 40 / 42

110 Further reading II C. A. McNabb and S. J. Qin. Projection based MIMO control performance monitoring: I-covariance monitoring in state space. J. Proc. Cont., 13(8): , C. A. McNabb and S. J. Qin. Projection based MIMO control performance monitoring: II-measured disturbances and setpoint changes. J. Proc. Cont., 15(1):89 102, B. J. Odelson, M. R. Rajamani, and J. B. Rawlings. A new autocovariance least-squares method for estimating noise covariances. Automatica, 42(2): , February R. S. Patwardhan. Studies in Synthesis and Analysis of Model Predictive Controllers. PhD thesis, University of Alberta, Fall URL http: // R. S. Patwardhan and S. L. Shah. Issues in performance diagnostics of model-based controllers. J. Proc. Cont., 12(3): , S. J. Qin and J. Yu. Recent developments in multivariable controller performance monitoring. J. Proc. Cont., 17(3): , M. R. Rajamani and J. B. Rawlings. Estimation of the disturbance structure from data using semidefinite programming and optimal weighting. Automatica, 45: , J. Schafer and A. Cinar. Multivariable MPC system performance assessment, monitoring, and diagnosis. J. Proc. Cont., 14(2): , Rawlings/Zagrobelny/Ji Control performance monitoring 41 / 42

111 Further reading III F. Xu, K. H. Lee, and B. Huang. Monitoring control performance via sructured closed-loop response subject to output variance/covariance upper bound. J. Proc. Cont., 16(9): , J. Yu and S. J. Qin. Statistical MIMO controller performance monitoring. Part I: Data-driven covariance benchmark. J. Proc. Cont., 18(3 4): , 2008a. J. Yu and S. J. Qin. Statistical MIMO controller performance monitoring. Part II: performance diagnosis. J. Proc. Cont., 18(3 4): , 2008b. Rawlings/Zagrobelny/Ji Control performance monitoring 42 / 42