A Basic Primer on Engineering Control Theory Engineering Control Theory: Can It Impact Adaptive Interventions? *name change awaiting approval by the AZ Board of Regents Daniel E. Rivera School of Mechanical, Aerospace, Chemical and Materials Engineering (MACME) Ira A. Fulton School of Engineering Arizona State University http://csel.asu.edu daniel.rivera@asu.edu Daniel E. Rivera School for the Engineering of Matter, Transport, and Energy (SEMTE)* Ira A. Fulton School of Engineering Arizona State University http://csel.asu.edu daniel.rivera@asu.edu SPR Pre-Conference Workshop # June, SPR Pre-Conference Workshop # June, About this tutorial Tutorial Outline Goal is to discuss how engineering control theory can inform the design and implementation of time-varying adaptive preventive interventions. Talk will be focused on describing important concepts; it will not be a comprehensive survey. Emphasis will be given to establishing connections between prevention/behavioral health, methodology, and engineering, and the opportunities (and challenges) that these present to the behavioral scientist, the methodologist, and the engineer. Fundamentals of adaptive interventions What is meant by control systems engineering, and how can these concepts improve behavioral interventions? - Shower operation as a closed-loop dynamical system. - Analysis and design of a hypothetical time-varying adaptive intervention inspired by the Fast Track program. IF-THEN decision rules Internal Model Control - PID control Summary and conclusions; topics for further study.
Current Projects in Behavioral Health Primary References KDA7*, Control engineering approaches to adaptive interventions for fighting drug abuse, Mentors: L.M. Collins (Penn State) and S.A. Murphy (Michigan). RDA66*, Dynamical systems and related engineering approaches to improving behavioral interventions, NIH Roadmap Initiative Award on Facilitating Interdisciplinary Research Via Methodological and Technological Innovation in the Behavioral and Social Sciences, with L.M. Collins, Penn State, co-pi. *Support from NIDA (National Institute on Drug Abuse) and OBSSR (Office of Behavioral and Social Sciences Research) is gratefully acknowledged. Collins, L.M., S.A. Murphy, and K.L. Bierman, A conceptual framework for adaptive preventive interventions, Prevention Science,, No., pgs. 8-96, Sept.,. Rivera, D.E., M.D. Pew, and L.M. Collins, Using engineering control principles to inform the design of adaptive interventions: a conceptual introduction, Drug and Alcohol Dependence, Special Issue on Adaptive Treatment Strategies, Vol. 88, Supplement, May 7, Pages S-S. Rivera, D.E., M.D. Pew, L.M. Collins, and S.A. Murphy, Engineering control approaches for the design and analysis of adaptive, time-varying interventions, Technical Report -7, The Methodology Center, Penn State University; available electronically from http://methcenter.psu.edu/ or from the ASU- website: http://csel.asu.edu/ Rivera D.E., M. Morari, and S. Skogestad, Internal model control: PID controller design, Ind. Eng. Chem. Proc. Des. Dev., 986, (), -6. 6 Some Similarities Between Prevention and Control Engineering Some Differences... Prevention scientists are interested in developing and delivering interventions that: - are strongly effective, with high levels of adherence. - display uniformity and reproducibility despite heterogeneity of the target population, and inherent variability associated with delivery of the intervention, - are cost-effective in nature. All these goals are compatible with the objectives of control systems engineering... 7 Mathematical skill sets are different; furthermore, methodological perspectives for viewing the world are different (statistical vs. input/ output dynamical systems viewpoints). Terminology can be a barrier; we use many of the same terms to mean very different things. Prevention scientists naturally think in terms of populations; control engineers tend to think in terms of single participants. Experimental design in prevention settings encompasses a larger and broader set of considerations as compared to traditional engineering applications. 8
Basic Components of Adaptive Interventions (Collins, Murphy, and Bierman, Prevention Science,, No., ) The assignment of a particular dosage and/or type of treatment is based on the individual s values on variables that are expected to moderate the effect of the treatment component; these are known as tailoring variables. In a time-varying adaptive intervention, the tailoring variable is assessed periodically, so the intervention is adjusted on an on-going basis. Decision rules translate current and previous values of tailoring variables into choice(s) of treatment and their appropriate dosage. Adaptive Intervention Benefits (Collins, Murphy, and Bierman, Prevention Science,, No., ) An effective adaptive intervention strategy may result in the following advantages over fixed interventions: Reduction of negative effects (i.e., stigma), Reduction of inefficiency and waste, Increased compliance, Enhanced intervention potency. Adaptive interventions can serve as an aid for disseminating efficacious interventions in real-world settings. 9 Adaptive Intervention Simulation (inspired by the Fast Track Program, Conduct Problems Prevention Research Group) A multi-year program designed to prevent conduct disorder in at-risk children. Frequency of home-based counseling visits assigned quarterly to families over a three-year period, based on an assessed level of parental functioning. (the tailoring variable) is used to determine the frequency of home visits (the intervention dosage) according to the following decision rules: - If parental function is very poor then the intervention dosage should correspond to weekly home visits, - If parental function is poor then the intervention dosage should correspond to bi-weekly home visits, - If parental function is below threshold then the intervention dosage should correspond to monthly home visits, - If parental function is at threshold then the intervention dosage should correspond to no home visits. Parental Function - Counselor Home Visits Adaptive Intervention Single Participant Family Illustration The assigned dosage (frequency of counseling visits) decreases as the tailoring variable (parental function) increases, as prescribed by the decision rules. goal No Visit
Control Systems Engineering The field that relies on dynamical models to develop mechanisms for adjusting system variables so that their behavior over time is transformed from undesirable to desirable, Open-loop: refers to system behavior without a controller or decision rules (i.e., MANUAL operation). Closed-loop: refers to system behavior once a controller or decision rule is implemented (i.e., AUTOmatic operation). A well-tuned control system will effectively transfer variability from an expensive system resource to a less expensive one. Control Systems Engineering Control systems engineering is a broadly-applicable field that spans all areas of engineering. Control engineering principles play an important part in many everyday life activities. Some examples of control system applications include: Cruise control and climate control in automobiles The sensor reheat feature in microwave ovens Home heating and cooling The insulin pump for Type-I diabetics Fly-by-wire systems in high-performance aircraft Many, many, more... Open-Loop (Manual) vs. Closed-Loop (Automatic) Control Climate control in automobiles is one of many illustrations of closed-loop control that can be found in daily life. Controlled variables (y): Temperature, water flow The Shower Problem The presence of transportation lag adds delay to the response of this system Disturbances (d): Inlet Water Flows, Temperatures Manual AUTOmatic Manipulated Variables (u): Hot and Cold Water Valve Positions Objective: Adjust hot and cold water flows in response to changes in shower temperature and outlet flow caused by external factors. 6
Signal Definitions Controlled Variables (y; outcomes): system variables that we wish to keep at a reference value (or goal), also known as the setpoint (r). Manipulated Variables (u): system variables whose adjustment influences the response of the controlled variable; their value is determined by the controller/decision policy. Shower Temperature (degrees Celsius) Controlled variable (y): Temperature 9 Open Loop No Disturbance 8 6 8 6 8 The Shower Problem Open Loop Disturbance Response Inlet Flowrate (liters/min).. Disturbance (d): Inlet Water Flow 6 8 6 8. Disturbance Variables (d): system variables that influence the controlled variable response, but cannot be manipulated by the controller; disturbance changes are external to the system. Both manipulated (u) and disturbance (d) variables can be viewed as independent (x) variables; disturbances are exogenous, while manipulated variables can be adjusted by the user. 7 Manipulated Variable (u): Hot Water Valve Position Hot Water Valve (percent open).. Time Consider the change in shower temperature caused by a sudden drop in inlet water flowrate as a result of a disturbance (e.g., sprinklers being activated). 8 Control System Components Feedback Control Strategy Sensors (i.e., assessment instruments): devices needed to measure the controlled and (possibly) the disturbance variables. Actuators: devices needed to achieve desired settings for the manipulated variables Controllers (i.e., clinical decision rules). These relate current and prior controlled variable, manipulated variable, and disturbance measurements to a current value for the manipulated variable. In feedback control: - the measured controlled variable (y) is compared to a goal (also known as a reference setpoint r), - a control error e (= r - y), representing the discrepancy between y and r is calculated. - a control algorithm determines a current value for the manipulated variable (u) based on current and previous values of e and u. 9
The Magic of Feedback (Adapted from K. J. Åström s Challenges in Control Education plenary talk at the 7th IFAC Symposium on Advances in Control Education, Madrid, Spain, June -, 6). Feedback has some amazing properties: can create good systems from bad components, makes a system less sensitive to disturbances and component variations, Shower Temperature (degrees Celsius) 9 Shower Problem: Closed-Loop Feedback Control Controlled: Temperature Closed Loop Open Loop Setpoint 8 6 8 6 8 (Good: Stable, Smooth Responses) T Sensor Controller Inlet Flowrate (liters/min).. Temp. setpoint Disturbance: Inlet Water Flow 6 8 6 8 can stabilize an unstable system, can create desired behavior, for example, linear behavior from nonlinear components. Major drawback: it can cause instability if not properly tuned. Actuator Hot Cold Hot Water Valve (percent open).9.8.7.6..... Manipulated: Hot Water Valve Position Closed Loop No Disturbance 6 8 6 8 Shower Temperature (degrees Celsius) 9 Shower Problem: Closed-Loop Feedback Control Controlled: Temperature Closed Loop Open Loop Setpoint 8 6 8 6 8 (Bad: Unstable, Oscillatory Responses) T Sensor Controller Inlet Flowrate (liters/min).. Temp. setpoint Disturbance: Inlet Water Flow From Open-Loop Operation to Closed-Loop Control (Stochastic Viewpoint) Temperature Deviation (Measured Controlled Variable) Open-Loop (Before Control) 6 8 6 8 Manipulated: Hot Water Valve Position.9.8 Closed Loop No Disturbance Hot Water Valve Adjustment (Manipulated Variable) Closed-Loop Control Actuator Hot Cold Hot Water Valve (percent open).7.6..... 6 8 6 8 The transfer of variance from an expensive resource to a cheaper one is one of the major benefits of control systems engineering
Adaptive Intervention Simulation (inspired by the Fast Track Program, Conduct Problems Prevention Research Group) Parental Function Feedback Loop Block Diagram* (to decide on home visits for families with at-risk children) A multi-year program designed to prevent conduct disorder in at-risk children. Frequency of home-based counseling visits assigned quarterly to families over a three-year period, based on an assessed level of parental functioning. (Open-Loop) (the tailoring variable) is used to determine the frequency of home visits (the intervention dosage) according to the following decision rules: - If parental function is very poor then the intervention dosage should correspond to weekly home visits, - If parental function is poor then the intervention dosage should correspond to bi-weekly home visits, - If parental function is below threshold then the intervention dosage should correspond to monthly home visits, - If parental function is at threshold then the intervention dosage should correspond to no home visits. (Closed-Loop System) From Rivera, D.E., M.D. Pew, and L.M. Collins, Using engineering control principles to inform the design of adaptive interventions: a conceptual introduction, Drug and Alcohol Dependence, Special Issue on Adaptive Treatment Strategies, Vol. 88, Supplement, May 7, Pages S-S. 6 Parental Function - Home Visits Adaptive Intervention as an Inventory Control Problem PF(t) is built up by providing an intervention I(t) (frequency of home visits), that is potentially subject to delay, and is depleted by potentially multiple disturbances (adding up to D(t)). Intervention Dosage (Manipulated Variable) Inflow Controller/ Decision Rules I(t) PF meas (t) Measured Parental Function (Feedback Signal) θ ( Delay Time) Parental Function Target (Setpoint Signal) K I (Gain) Parental Function (Controlled Variable) PF(t + ) = PF(t)+K I I(t θ) D(t) 7 PF(t) PF Goal Exogenous Depletion Effects (Disturbance Variable) D(t) Outflow Parental Function Open Loop Dynamics PF(t+) = PF(t) + K I I(t θ) - D(t) t = time, expressed as an integer reflecting review instance K I = intervention gain D(t) = depletion I(t) = intervention dosage θ = delay time Parental Function (End of Review Instance)! = Parental Function (Start of Review Instance)! + Parental Function Contributed by Intervention! - Parental Function Depletion 8
8 6 Parental Function Dynamics Open Loop Response Parental Function Dynamics Open Loop Response (Continued) 8 6 Participant :(, KI ) = (,. ) Participant :(, KI ) = (,.8 ) Participant :(, KI ) = (,.7 ) Participant :(, KI ) = (,. ) Participant :(, KI ) = (,.7 ) Intervention Dosage Variations in intervention dose response for a single participant family 9 Intervention Dosage Between-participant variability as a result of individual dynamic characteristics D(t)= D(t)= D(t)= D(t)= Depletion (D(t)) Parental Function Dynamics Open Loop (Continued) change as a result of step changes in outflow (the disturbance variable) of varying magnitudes. Connecting ACE and Dynamical Systems Models (Fast Track Example) ACE = E(Y i ) E(Y i ) Y i = Y i = ACE = E N dur t= N dur t= (PF i (t ) + K ii I(t θ i) D i (t)) (PF i (t ) D i (t)) Ndur t= (K ii I(t θ i)) Average Causal Effect (ACE) is a function of model parameters (gain, delay) and intervention dosages assigned during the course of the intervention.
Adaptive Intervention Using IF-THEN Rules Adaptive Intervention Using IF-THEN Rules No Depletion (D(t) = ) goal High Depletion (D(t) = ) goal 6 6 8 6 Intervention 6 6 8 6 Single participant family scenario. The goal is for the family to attain a % proficiency (dashed line) on a parental function scale at the conclusion of the three year intervention. Offset (where parental function fails to meet goal) is more pronounced when high depletion is present. Multiple participant family simulation. The goal is for each family to attain a % proficiency (dashed line) on a parental function scale at the conclusion of the three year intervention. Offset is observed in all participant families. Engineering Control Design Control Design Requirements Based on a knowledge of the open-loop model, an optimized feedback decision algorithm (i.e., the controller ) can be designed for this system. In general, the sophistication of the controller will be a function of the complexity of the model and the desired performance requirements. We consider a tuning rule for a Proportional-Integral Derivative (PID) feedback controller for an integrating system which relies on the concept of Internal Model Control (IMC; Morari and Zafiriou, 987). User supplies the intervention gain (K I ) and delay (θ) and a setting for an adjustable parameter (λ) that defines the speed of response and robustness of the control system. Stability. Many different notions exist, but BIBO stability (bounded inputs resulting in bounded outputs) is usually sufficient. No offset. Control error e = r - y should go to zero (meaning that the controlled variable should reach the goal) at a finite time. Minimal effect of disturbances on controlled variables. Rapid, smooth (i.e., non-oscillatory) responses of controlled variables to setpoint changes. Large variations ( moves ) in the manipulated variables should be avoided. Robustness, that is, performance should display little sensitivity to changes in operating conditions and model parameters. 6
Proportional-Integral-Derivative (PID) with Filter Controller Summary I(t) = I(t-) + K e(t) + K e(t-) + K e(t-) + K (I(t-)-I(t-)) Current Dosage = Previous Dosage + Scaled Corrections using Current and Prior Control Errors + Scaled Previous Dosage Change K, K, K, and K are tuning constants in the controller; e(t) = (PF(t) - PF Goal ), where PF Goal is the setpoint ( goal ) and e(t) is the control error. Internal Model Control-Proportional Integral Derivative (IMC-PID) Controller Tuning Rules (Rivera et al., 986) K c = User supplies open-loop model gain (K I ), delay (θ) and the adjustable parameter (λ); T is the review period I(t) =I(t T )+K e(t)+k e(t T )+K e(t T )+K (I(t T ) I(t T )) β = τ = θ (β+λ)+τ K I (β +βλ+λ ) τ I = (β + λ)+τ τ D = τ(β+λ) (β+λ)+τ τ F = βλ β +βλ+λ The dosage decision I(t) is a continuous value between and %, but for purposes of this example it is quantized into the nearest of the four dosage levels (I weekly, I biweekly, I monthly, ). K = TKc τ F +T + T τ I + τd T K = TKc τ F +T + τ D T K = KcτD τ F +T K = τf τ F +T 7 8 Controller/Decision Rule Comparison, High Depletion Rate (D(t) = ) IF-THEN rules goal IMC-PID control (λ = ; moderate speed) 6 month intervention reviewed at quarterly intervals. Offset problem is eliminated by more judicious assignment of intervention dosages during the course of the intervention. 9 Lambda= goal IMC-PID Controller, High Depletion Rate, Various Controller Speeds (determined by λ) Fast (λ = ) goal Moderate (λ = ) Lambda= goal Slow (λ = ) Lambda= goal
IF-THEN vs. IMC-PID Comparison IF-THEN vs. IMC-PID Comparison (Continued) The intervention dosage is adapted at quarterly intervals over a 6-month time period. The goal is for each family to attain a % proficiency (dashed line) on a parental function scale at the conclusion of the three year intervention. The intervention dosage is adapted at quarterly intervals over a 6-month time period. The goal is for each family to attain a % proficiency (dashed line) on a parental function scale at the conclusion of the three year intervention. IF-THEN Decision Rules IMC-PID control (λ = ) IMC-PID control (λ = ) IMC-PID control (λ =) 6 6 8 6 6 6 8 6 6 6 8 6 6 6 8 6 Intervention 6 6 8 6 Intervention 6 6 8 6 Intervention 6 6 8 6 Intervention 6 6 8 6 The closed-loop response of five participant families is evaluated using a controller model based on an average ( nominal ) effect. The transfer of variance concept is illustrated here. The closed-loop response of five participant families is evaluated using a controller model based on an average ( nominal ) effect. The transfer of variance concept is illustrated here. PROS: Simple; IF-THEN Decision Rules Pros and Cons both measurement and dosage levels are categorical in nature. decision rules are simple, compact, and easy to explain. IMC-PID Controller Pros and Cons PROS: reliance on model-based tuning results in improved outcomes (e.g., no offset, robust stability and performance). continuous measurement of tailoring variables is naturally incorporated, controller/decision policy remains compact; it includes an adjustable parameter (λ) defines a speed of response. CONS: Simplistic; may not lead to desirable outcomes (e.g., offset, instability, and high degree of variability in response among participants may result). CONS: Model parameters (K I and θ) need to be estimated prior to the intervention, Adjustable parameter (λ) needs to be selected systematically ( tuning ), Assigning continuous dosages to categories must be done with care, Controller/decision rules harder to explain to non-experts.
Summary and Conclusions Behavioral interventions constitute uncertain, nonlinear dynamical systems, and can therefore benefit from a control engineering perspective. Applying a dynamical systems approach requires more frequent measurement and a recognition of the input/output nature of phenomena associated with behavioral interventions. A hypothetical adaptive intervention based on Fast Track has been simulated using a rule-based controller ( IF-THEN decision rules) vs. engineeringbased PID (Proportional-Integral-Derivative) decision algorithms. A fluid analogy, based on the principle of mass conservation, was useful in establishing a comparison between these various control systems. A plethora of opportunities exist in applying systems approaches to behavioral interventions which includes experimental design, development of optimal decision policies, and simulation. Additional Topics (not covered) System identification: examines how to empirically obtain dynamical models from data; also enables simplifying other model types (e.g., system dynamics, agent-based models) into forms amenable for control. Feedforward control action: if disturbance variables can be measured, these can be incorporated as tailoring variables in the controller in a feedforward (i.e., anticipative) manner. Model predictive control*: control design paradigm that features advanced adaptive functionality such as constraint handling, decisionmaking involving multiple outcomes, and formal assignment of intervention dosages to discrete-valued categories. *to be discussed during the session entitled, Innovative Methodology for Adaptive Interventions Drawing from Engineering and Computer Science, Grand Ballroom, Thursday, June rd, : - : p.m. 6 Fluid Analogy for Mediation Analysis Path Diagram: T (t) at (t θ ) e (t) Fluid Analogy for the Theory of Planned Behavior* *to be discussed during the session entitled, Innovative Methodology for Adaptive Interventions Drawing from Engineering and Computer Science, Grand Ballroom, Thursday, June rd, : - : p.m. T a M c e dm τ dt dy τ dt b Y e e (t) $##% #% 7 Y (t)!"# Y (t) c T (t θ ) = at(t θ ) M(t)+e (t) M(t) $##% #% bm(t θ ) bm(t) ( b)m(t) = c T (t θ )+bm(t θ ) Y (t)+e (t). Time constant (!) and delay (θ) variables are essential features in this dynamic model representation for mediation.,(-% &%'(')"*+%!"#$% Behavioral belief evaluation of outcome (ξ = b e) Normative belief motivation to comply (ξ = n m) Control belief power of control belief (ξ = c p) γ γ γ Attitude Toward the Behavior (η) Subjective Norms (η) Perceived Behavioral Control (η) ζ β ζ β β ζ Intention (η) β ζ β ξ(t) ζ Behavior (η) γξ(t θ) ## # 8 βη(t θ) ( β)η.%%'%/& (η ) ζ(t) ) (η ) βη(t θ) βη(t θ6) ( β)η ξ(t) ζ(t) γξ(t θ) ## # ## # (6'+78 9(-:; (η )!$%&$%'($ (η!"# ) ( β)η ζ(t) βη(t θ7) ## # ζ(t) βη(t θ8) )&*+,'(- (η ) γξ(t θ) ζ(t) ## # ( β β)η Any path diagram can be expressed into a corresponding fluid analogy described by a system of differential equations. η (t) ξ(t)
Looking to the Future Additional References We have considered the importance of establishing connections between prevention/behavioral health, methodology, and engineering. Some implications of this work (not an exhaustive list): Behavioral scientist: willingness to collect and work with intensive longitudinal data, reconfigure interventions to enable adaptation (i.e., allow dosage changes through the course of the intervention). Methodologist: develop expertise and familiarity with differential equations, dynamical input/output system models, and control theory. Control engineer: work with data sets that may be irregularly sampled, have missing entries, and involve multiple human participants. Explore experimental designs meaningful to problems in behavioral health. Some additional tutorial presentations that may be of interest: Rivera, D.E., An Introduction to Mechanistic Models and Control Theory, tutorial presentation at the SAMSI Summer 7 Program on Challenges in Dynamic Treatment Regimes and Multistage Decision-Making, June 8-9, 7. Can be downloaded from http://csel.asu.edu/controleducation (select item 9). Rivera, D.E., A Brief Introduction to System Identification, Penn State Methodology Center Brown Bag presentation, March, 8. Can be downloaded from http://csel.asu.edu/controleducation (select item ). A free web-based reference, written by two eminent control systems engineers: Åström, K. J. and R. M. Murray. Feedback systems: an introduction for scientists and engineers, http://www.cds.caltech.edu/~murray/amwiki. 9 Acknowledgments Thank you for your attention! Linda M. Collins, Ph.D., The Methodology Center and Dept. of Human Development and Family Studies, Penn State University. Susan D. Murphy, Ph.D., Department of Statistics, Department of Psychiatry, and Institute for Social Research, Univ. of Michigan. R Roadmap Initiative Grant Project Team (N. Nandola, J.E. Navarro, S. Deshpande, ASU; J. Johnson and I. Shani, PSU) Support from NIH-NIDA (National Institute on Drug Abuse) and NIH-OBSSR (Office of Behavioral and Social Sciences Research) through Grants KDA7 and RDA66 http://csel.asu.edu/health