Condensed Table of Contents for Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control by J. C.

Transcription

1 Condensed Table of Contents for Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control by J. C. Spall John Wiley and Sons, Inc., 2003 Preface... xiii 1. Stochastic Search and Optimization: Motivation and Supporting Results Introduction Some Principles of Stochastic Search and Optimization Gradients, Hessians, and Their Connection to Optimization of Smooth Functions Deterministic Search and Optimization: Steepest Descent and Newton Raphson Search Concluding Remarks Exercises Direct Methods for Stochastic Search Introduction Random Search with Noise-Free Loss Measurements Random Search with Noisy Loss Measurements Nonlinear Simplex (Nelder Mead) Algorithm Concluding Remarks Exercises Recursive Estimation for Linear Models Formulation for Estimation with Linear Models Least-Mean-Squares and Recursive-Least-Squares for Static θ LMS, RLS, and Kalman Filter for Time-Varying θ Case Study: Analysis of Oboe Reed Data Concluding Remarks Exercises Stochastic Approximation for Nonlinear Root-Finding Introduction Potpourri of Stochastic Approximation Examples Convergence of Stochastic Approximation Asymptotic Normality and Choice of Gain Sequence Extensions to Basic Stochastic Approximation

2 4.6 Concluding Remarks Exercises Stochastic Gradient Form of Stochastic Approximation Root-Finding Stochastic Approximation as a Stochastic Gradient Method Neural Network Training Discrete-Event Dynamic Systems Image Restoration Concluding Remarks Exercises Stochastic Approximation and the Finite-Difference Method Introduction and Contrast of Gradient-Based and Gradient-Free Algorithms Some Motivating Examples for Gradient-Free Stochastic Approximation Finite-Difference Algorithm Convergence Theory Asymptotic Normality Practical Selection of Gain Sequences Several Finite-Difference Examples Some Extensions and Enhancements to the Finite-Difference Algorithm Concluding Remarks Exercises Simultaneous Perturbation Stochastic Approximation Background Form and Motivation for Standard SPSA Algorithm Basic Assumptions and Supporting Theory for Convergence Asymptotic Normality and Efficiency Analysis Practical Implementation Numerical Examples Some Extensions: Optimal Perturbation Distribution; One-Measurement Form; Global, Discrete, and Constrained Optimization Adaptive SPSA Concluding Remarks Appendix: Conditions for Asymptotic Normality Exercises

3 8. Annealing-Type Algorithms Introduction to Simulated Annealing and Motivation from the Physics of Cooling Simulated Annealing Algorithm Some Examples Global Optimization via Annealing Algorithms Based on Stochastic Approximation Concluding Remarks Appendix: Convergence Theory for Simulated Annealing Based on Stochastic Approximation Exercises Evolutionary Computation I: Genetic Algorithms Introduction Some Historical Perspective and Motivating Applications Coding of Elements for Searching Standard Genetic Algorithm Operations Overview of Basic GA Search Approach Practical Guidance and Extensions: Coefficient Values, Constraints, Noisy Fitness Evaluations, Local Search, and Parent Selection Examples Concluding Remarks Exercises Evolutionary Computation II: General Methods and Theory Introduction Overview of Evolution Strategy and Evolutionary Programming with Comparisons to Genetic Algorithms Schema Theory What Makes a Problem Hard? Convergence Theory No Free Lunch Theorems Concluding Remarks Exercises Reinforcement Learning via Temporal Differences Introduction Delayed Reinforcement and Formulation for Temporal Difference Learning Basic Temporal Difference Algorithm Batch and Online Implementations of TD Learning Some Examples

4 11.6 Connections to Stochastic Approximation Concluding Remarks Exercises Statistical Methods for Optimization in Discrete Problems Introduction to Multiple Comparisons Over a Finite Set Statistical Comparisons Test Without Prior Information Multiple Comparisons Against One Candidate with Known Noise Variance(s) Multiple Comparisons Against One Candidate with Unknown Noise Variance(s) Extensions to Bonferroni Inequality; Ranking and Selection Methods in Optimization Over a Finite Set Concluding Remarks Exercises Model Selection and Statistical Information Bias Variance Tradeoff Model Selection: Cross-Validation The Information Matrix: Applications and Resampling-Based Computation Concluding Remarks Exercises Simulation-Based Optimization I: Regeneration, Common Random Numbers, and Selection Methods Background Regenerative Systems Optimization with Finite-Difference and Simultaneous Perturbation Gradient Estimators Common Random Numbers Selection Methods for Optimization with Discrete-Valued θ Concluding Remarks Exercises Simulation-Based Optimization II: Stochastic Gradient and Sample Path Methods Framework for Gradient Estimation Pure Likelihood Ratio/Score Function and Pure Infinitesimal Perturbation Analysis Gradient Estimation Methods in Root-Finding Stochastic Approximation: The Hybrid LR/SF and IPA Setting Sample Path Optimization

5 15.5 Concluding Remarks Exercises Markov Chain Monte Carlo Background Metropolis Hastings Algorithm Gibbs Sampling Sketch of Theoretical Foundation for Gibbs Sampling Some Examples of Gibbs Sampling Applications in Bayesian Analysis Concluding Remarks Exercises Optimal Design for Experimental Inputs Introduction Linear Models Response Surface Methodology Nonlinear Models Concluding Remarks Appendix: Optimal Design in Dynamic Models Exercises Appendix A. Selected Results from Multivariate Analysis A.1 Multivariate Calculus and Analysis A.2 Some Useful Results in Matrix Theory Exercises Appendix B. Some Basic Tests in Statistics B.1 Standard One-Sample Test B.2 Some Basic Two-Sample Tests B.3 Comments on Other Aspects of Statistical Testing Exercises Appendix C. Probability Theory and Convergence C.1 Basic Properties C.2 Convergence Theory Exercises Appendix D. Random Number Generation D.1 Background and Introduction to Linear Congruential Generators

6 D.2 Transformation of Uniform Random Numbers to Other Distributions Exercises Appendix E. Markov Processes E.1 Background on Markov Processes E.2 Discrete Markov Chains Exercises Answers to Selected Exercises References Frequently Used Notation Index