Chapter 8: Generalization and Function Approximation
|
|
- Rosalind Powers
- 5 years ago
- Views:
Transcription
1 Chapte 8: Genealization and Function Appoximation Objectives of this chapte: Look at how expeience with a limited pat of the state set be used to poduce good behavio ove a much lage pat. Oveview of function appoximation (FA) methods and how they can be adapted to RL
2 Value Pediction with FA As usual: Policy Evaluation (the pediction poblem): fo a given policy π, compute the state-value function V! In ealie chaptes, value functions wee stoed in lookup tables. Hee, the value function estimate at time t, V t, depends on a paamete vecto, and only the paamete vecto is updated. e.g.,! t could be the vecto of connection weights of a neual netwok. R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 2
3 Adapt Supevised Leaning Algoithms Taining Info = desied (taget) outputs Inputs Supevised Leaning System Outputs Taining example = {input, taget output} Eo = (taget output actual output) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 3
4 Backups as Taining Examples e.g., the TD(0) backup : [ ] V(s t )! V(s t ) + " t +1 +# V(s t +1 ) $ V(s t ) As a taining example: desciption of s t, t +1 +! V (s t+1 ) { } input taget output R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 4
5 Any FA Method? In pinciple, yes: atificial neual netwoks decision tees multivaiate egession methods etc. But RL has some special equiements: usually want to lean while inteacting ability to handle nonstationaity othe? R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 5
6 Gadient Descent Methods tanspose = ( (1), (2),K, (n)) T Assume V t is a (sufficiently smooth) diffeentiable function of! t, fo all s "S. Assume, fo now, taining examples of this fom : { desciption of s t, V! (s t )} R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 6
7 Pefomance Measues Many ae applicable but a common and simple one is the mean-squaed eo (MSE) ove a distibution P : MSE( ) = & s%s [ ] P(s) V # (s) $V t (s) Why P? Why minimize MSE? Let us assume that P is always the distibution of states with which backups ae done. The on-policy distibution: the distibution ceated while following the policy being evaluated. Stonge esults ae available fo this distibution. 2 R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 7
8 Gadient Descent Let f be any function of the paamete space. Its gadient at any point in this space is : #" f ( ) = $f ( ) $"(1),$f ( ) $"(2),K,$f ( T % )( ' * & $"(n) )! (2)! t = (! t (1),! t (2)) T Iteatively! t +1 = move down! t "#$ f ( the gadient:!! t )! (1) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 8
9 Gadient Descent Cont. Fo the MSE given above and using the chain ule: +1 = # 1 2 $% " MSE( ) ' P(s) [ V ( (s) #V (s)] 2 s&s = + $ P(s) [ V ( (s) #V t (s)]%" V t(s) = # 1 2 $% ' s&s R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 9
10 Gadient Descent Cont. Use just the sample gadient instead: +1 = # 1 2 $% [ V & (s ) #V t (s t )] 2 = + $ [ V & (s t ) #V t (s t )]%" V (s ), t t Since each sample gadient is an unbiased estimate of the tue gadient, this conveges to a local minimum of the MSE if α deceases appopiately with t. E[ V " (s t ) #V t (s t )]$% V t (s t ) = ' P(s) V " (s) #V t (s) s&s [ ] $ % V t (s) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 10
11 But We Don t have these Tagets Suppose we just have tagets v t instead :! t +1 =! t +" [ v t # V t (s t )]$ V (s )! t t If each v t is an unbiased estimate of V " (s t ), i.e., E{ v t } = V " (s t ), then gadient descent conveges to a local minimum (povided # deceases appopiately). e.g., the Monte Calo taget v t = R t : +1 = + #[ R t $V t (s t )]%" V t (s t ) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 11
12 What about TD(λ) Tagets? +1 = + #[ R $ t %V t (s t )]&" V t (s t ) Not unbiased fo $ <1 But we do it anyway, using the backwads view : +1 = + #$ t e t, whee : $ t = t +1 + % V t (s t +1 ) &V t (s t ), as usual, and e t = % ' e t&1 + ( V (s t ) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 12
13 On-Line Gadient-Descent TD(λ) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 13
14 Linea Methods Repesent states as featue vectos: fo each s " S : # s = (# s (1),# s (2),K,# s (n)) T V t (s) = T " # V t (s) =? n $ i=1 # s = (i)# s (i) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 14
15 Nice Popeties of Linea FA Methods The gadient is vey simple:! V (s) = # s Fo MSE, the eo suface is simple: quadatic suface with a single minumum. Linea gadient descent TD(λ) conveges: Step size deceases appopiately On-line sampling (states sampled fom the on-policy distibution) Conveges to paamete vecto with popety:! " MSE(! " ) # 1 $% & 1 $ % MSE(! ' ) (Tsitsiklis & Van Roy, 1997) best paamete vecto R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 15
16 Coase Coding R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 16
17 Leaning and Coase Coding R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 17
18 Tile Coding Binay featue fo each tile Numbe of featues pesent at any one time is constant Binay featues means weighted sum easy to compute Easy to compute indices of the featues pesent R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 18
19 Tile Coding Cont. Iegula tilings Hashing CMAC Ceebella Model Aithmetic Compute Albus 1971 R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 19
20 Radial Basis Functions (RBFs) e.g., Gaussians % " s (i) = exp # s # c i ' 2 & 2$ i 2 ( * ) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 20
21 Can you beat the cuse of dimensionality? Can you keep the numbe of featues fom going up exponentially with the dimension? Function complexity, not dimensionality, is the poblem. Kaneva coding: Select a bunch of binay pototypes Use hamming distance as distance measue Dimensionality is no longe a poblem, only complexity Lazy leaning schemes: Remembe all the data To get new value, find neaest neighbos and intepolate e.g., locally-weighted egession R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 21
22 Contol with FA Leaning state-action values Taining examples of the fom: { desciption of ( s t, a t ), v } t The geneal gadient-descent ule:! t +1 =! t +" [ v t # Q t (s t,a t )]$! Q(s t,a t ) Gadient-descent Sasa(λ) (backwad view):! t +1 =! t +"# t whee e t # t = t +1 + $ Q t (s t +1, a t +1 ) % Q t (s t,a t ) e t = $ & e t %1 + ' Q t (s t,a t )! R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 22
23 Linea Gadient Descent Sasa(λ) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 23
24 GPI Linea Gadient Descent Watkins Q(λ) R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 24
25 Mountain-Ca Task R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 25
26 Mountain-Ca Results R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 26
27 Baid s Counteexample R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 27
28 Baid s Counteexample Cont. R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 28
29 Should We Bootstap? R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 29
30 Summay Genealization Adapting supevised-leaning function appoximation methods Gadient-descent methods Linea gadient-descent methods Radial basis functions Tile coding Kaneva coding Nonlinea gadient-descent methods? Backpopation? Subleties involving function appoximation, bootstapping and the on-policy/off-policy distinction R. S. Sutton and A. G. Bato: Reinfocement Leaning: An Intoduction 30
Value Prediction with FA. Chapter 8: Generalization and Function Approximation. Adapt Supervised Learning Algorithms. Backups as Training Examples [ ]
Chapte 8: Genealization and Function Appoximation Objectives of this chapte:! Look at how expeience with a limited pat of the state set be used to poduce good behavio ove a much lage pat.! Oveview of function
More informationChapter 8: Generalization and Function Approximation
Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to produce good behavior over a much larger part. Overview
More informationCSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating?
CSE 190: Reinforcement Learning: An Introduction Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to
More informationCSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating?
CSE 190: Reinforcement Learning: An Introduction Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to
More informationCS599 Lecture 2 Function Approximation in RL
CS599 Lecture 2 Function Approximation in RL Look at how experience with a limited part of the state set be used to produce good behavior over a much larger part. Overview of function approximation (FA)
More informationGeneralization and Function Approximation
Generalization and Function Approximation 0 Generalization and Function Approximation Suggested reading: Chapter 8 in R. S. Sutton, A. G. Barto: Reinforcement Learning: An Introduction MIT Press, 1998.
More informationMULTILAYER PERCEPTRONS
Last updated: Nov 26, 2012 MULTILAYER PERCEPTRONS Outline 2 Combining Linea Classifies Leaning Paametes Outline 3 Combining Linea Classifies Leaning Paametes Implementing Logical Relations 4 AND and OR
More informationTemporal-Difference Learning
.997 Decision-Making in Lage-Scale Systems Mach 17 MIT, Sping 004 Handout #17 Lectue Note 13 1 Tempoal-Diffeence Leaning We now conside the poblem of computing an appopiate paamete, so that, given an appoximation
More information4/18/2005. Statistical Learning Theory
Statistical Leaning Theoy Statistical Leaning Theoy A model of supevised leaning consists of: a Envionment - Supplying a vecto x with a fixed but unknown pdf F x (x b Teache. It povides a desied esponse
More informationNumerical Integration
MCEN 473/573 Chapte 0 Numeical Integation Fall, 2006 Textbook, 0.4 and 0.5 Isopaametic Fomula Numeical Integation [] e [ ] T k = h B [ D][ B] e B Jdsdt In pactice, the element stiffness is calculated numeically.
More informationMultiple Experts with Binary Features
Multiple Expets with Binay Featues Ye Jin & Lingen Zhang Decembe 9, 2010 1 Intoduction Ou intuition fo the poect comes fom the pape Supevised Leaning fom Multiple Expets: Whom to tust when eveyone lies
More informationCSCE 478/878 Lecture 4: Experimental Design and Analysis. Stephen Scott. 3 Building a tree on the training set Introduction. Outline.
In Homewok, you ae (supposedly) Choosing a data set 2 Extacting a test set of size > 3 3 Building a tee on the taining set 4 Testing on the test set 5 Repoting the accuacy (Adapted fom Ethem Alpaydin and
More informationDirected Regression. Benjamin Van Roy Stanford University Stanford, CA Abstract
Diected Regession Yi-hao Kao Stanfod Univesity Stanfod, CA 94305 yihaoao@stanfod.edu Benjamin Van Roy Stanfod Univesity Stanfod, CA 94305 bv@stanfod.edu Xiang Yan Stanfod Univesity Stanfod, CA 94305 xyan@stanfod.edu
More informationRegularization. Stephen Scott and Vinod Variyam. Introduction. Outline. Machine. Learning. Problems. Measuring. Performance.
leaning can geneally be distilled to an optimization poblem Choose a classifie (function, hypothesis) fom a set of functions that minimizes an objective function Clealy we want pat of this function to
More informationChapter 3: Theory of Modular Arithmetic 38
Chapte 3: Theoy of Modula Aithmetic 38 Section D Chinese Remainde Theoem By the end of this section you will be able to pove the Chinese Remainde Theoem apply this theoem to solve simultaneous linea conguences
More informationRevision of Lecture Eight
Revision of Lectue Eight Baseband equivalent system and equiements of optimal tansmit and eceive filteing: (1) achieve zeo ISI, and () maximise the eceive SNR Thee detection schemes: Theshold detection
More informationConjugate Gradient Methods. Michael Bader. Summer term 2012
Gadient Methods Outlines Pat I: Quadatic Foms and Steepest Descent Pat II: Gadients Pat III: Summe tem 2012 Pat I: Quadatic Foms and Steepest Descent Outlines Pat I: Quadatic Foms and Steepest Descent
More informationGeometry and statistics in turbulence
Geomety and statistics in tubulence Auoe Naso, Univesity of Twente, Misha Chetkov, Los Alamos, Bois Shaiman, Santa Babaa, Alain Pumi, Nice. Tubulent fluctuations obey a complex dynamics, involving subtle
More information10/04/18. P [P(x)] 1 negl(n).
Mastemath, Sping 208 Into to Lattice lgs & Cypto Lectue 0 0/04/8 Lectues: D. Dadush, L. Ducas Scibe: K. de Boe Intoduction In this lectue, we will teat two main pats. Duing the fist pat we continue the
More informationEM Boundary Value Problems
EM Bounday Value Poblems 10/ 9 11/ By Ilekta chistidi & Lee, Seung-Hyun A. Geneal Desciption : Maxwell Equations & Loentz Foce We want to find the equations of motion of chaged paticles. The way to do
More informationC e f paamete adaptation f (' x) ' ' d _ d ; ; e _e K p K v u ^M() RBF NN ^h( ) _ obot s _ s n W ' f x x xm xm f x xm d Figue : Block diagam of comput
A Neual-Netwok Compensato with Fuzzy Robustication Tems fo Impoved Design of Adaptive Contol of Robot Manipulatos Y.H. FUNG and S.K. TSO Cente fo Intelligent Design, Automation and Manufactuing City Univesity
More informationA Machine Learned Model of a Hybrid Aircraft
1 A Machine Leaned Model of a Hybid Aicaft Bandon Jones, Kevin Jenkins CS229 Machine Leaning, Fall 2016, Stanfod Univesity I. INTRODUCTION Aicaft development pogams ely on aicaft dynamic models fo flight
More informationResearch Design - - Topic 17 Multiple Regression & Multiple Correlation: Two Predictors 2009 R.C. Gardner, Ph.D.
Reseach Design - - Topic 7 Multiple Regession & Multiple Coelation: Two Pedictos 009 R.C. Gadne, Ph.D. Geneal Rationale and Basic Aithmetic fo two pedictos Patial and semipatial coelation Regession coefficients
More informationA Power Method for Computing Square Roots of Complex Matrices
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 13, 39345 1997 ARTICLE NO. AY975517 A Powe Method fo Computing Squae Roots of Complex Matices Mohammed A. Hasan Depatment of Electical Engineeing, Coloado
More informationASTR415: Problem Set #6
ASTR45: Poblem Set #6 Cuan D. Muhlbege Univesity of Mayland (Dated: May 7, 27) Using existing implementations of the leapfog and Runge-Kutta methods fo solving coupled odinay diffeential equations, seveal
More informationx 1 b 1 Consider the midpoint x 0 = 1 2
1 chapte 2 : oot-finding def : Given a function f(), a oot is a numbe satisfying f() = 0. e : f() = 2 3 = ± 3 question : How can we find the oots of a geneal function f()? 2.1 bisection method idea : Find
More informationProbablistically Checkable Proofs
Lectue 12 Pobablistically Checkable Poofs May 13, 2004 Lectue: Paul Beame Notes: Chis Re 12.1 Pobablisitically Checkable Poofs Oveview We know that IP = PSPACE. This means thee is an inteactive potocol
More informationLocalization of Eigenvalues in Small Specified Regions of Complex Plane by State Feedback Matrix
Jounal of Sciences, Islamic Republic of Ian (): - () Univesity of Tehan, ISSN - http://sciencesutaci Localization of Eigenvalues in Small Specified Regions of Complex Plane by State Feedback Matix H Ahsani
More informationReliability analysis examples
Reliability analysis examples Engineeing Risk Analysis Goup, Technische Univesität München. Acisst., 80 Munich, Gemany. May 8, 08 Intoduction In the context of eliability analysis and ae event estimation,
More informationGradient-based Neural Network for Online Solution of Lyapunov Matrix Equation with Li Activation Function
Intenational Confeence on Infomation echnology and Management Innovation (ICIMI 05) Gadient-based Neual Netwok fo Online Solution of Lyapunov Matix Equation with Li Activation unction Shiheng Wang, Shidong
More informationMulti-Objective Optimization Algorithms for Finite Element Model Updating
Multi-Objective Optimization Algoithms fo Finite Element Model Updating E. Ntotsios, C. Papadimitiou Univesity of Thessaly Geece Outline STRUCTURAL IDENTIFICATION USING MEASURED MODAL DATA Weighted Modal
More informationChem 453/544 Fall /08/03. Exam #1 Solutions
Chem 453/544 Fall 3 /8/3 Exam # Solutions. ( points) Use the genealized compessibility diagam povided on the last page to estimate ove what ange of pessues A at oom tempeatue confoms to the ideal gas law
More informationyou of a spring. The potential energy for a spring is given by the parabola U( x)
Small oscillations The theoy of small oscillations is an extemely impotant topic in mechanics. Conside a system that has a potential enegy diagam as below: U B C A x Thee ae thee points of stable equilibium,
More informationLinear Program for Partially Observable Markov Decision Processes. MS&E 339B June 9th, 2004 Erick Delage
Linea Pogam fo Patiall Obsevable Makov Decision Pocesses MS&E 339B June 9th 2004 Eick Delage Intoduction Patiall Obsevable Makov Decision Pocesses Etension of the Makov Decision Pocess to a wold with uncetaint
More informationLINEAR AND NONLINEAR ANALYSES OF A WIND-TUNNEL BALANCE
LINEAR AND NONLINEAR ANALYSES O A WIND-TUNNEL INTRODUCTION BALANCE R. Kakehabadi and R. D. Rhew NASA LaRC, Hampton, VA The NASA Langley Reseach Cente (LaRC) has been designing stain-gauge balances fo utilization
More informationA Deep Convolutional Neural Network Based on Nested Residue Number System
A Deep Convolutional Neual Netwok Based on Nested Residue Numbe System Hioki Nakahaa Ehime Univesity, Japan Tsutomu Sasao Meiji Univesity, Japan Abstact A pe-tained deep convolutional neual netwok (DCNN)
More informationForecasting Agricultural Commodity Prices Using Multivariate Bayesian Machine Learning. Andres M. Ticlavilca, Dillon M. Feuz, and Mac McKee
Foecasting Agicultual Commodity Pices Using Multivaiate Bayesian Machine Leaning Regession by Andes M. Ticlavilca, Dillon M. Feuz, and Mac McKee Suggested citation fomat: Ticlavilca, A. M., Dillon M. Feuz
More informationworking pages for Paul Richards class notes; do not copy or circulate without permission from PGR 2004/11/3 10:50
woking pages fo Paul Richads class notes; do not copy o ciculate without pemission fom PGR 2004/11/3 10:50 CHAPTER7 Solid angle, 3D integals, Gauss s Theoem, and a Delta Function We define the solid angle,
More informationFlux. Area Vector. Flux of Electric Field. Gauss s Law
Gauss s Law Flux Flux in Physics is used to two distinct ways. The fist meaning is the ate of flow, such as the amount of wate flowing in a ive, i.e. volume pe unit aea pe unit time. O, fo light, it is
More informationMethod for Approximating Irrational Numbers
Method fo Appoximating Iational Numbes Eic Reichwein Depatment of Physics Univesity of Califonia, Santa Cuz June 6, 0 Abstact I will put foth an algoithm fo poducing inceasingly accuate ational appoximations
More informationTo Feel a Force Chapter 7 Static equilibrium - torque and friction
To eel a oce Chapte 7 Chapte 7: Static fiction, toque and static equilibium A. Review of foce vectos Between the eath and a small mass, gavitational foces of equal magnitude and opposite diection act on
More informationComputational Methods of Solid Mechanics. Project report
Computational Methods of Solid Mechanics Poject epot Due on Dec. 6, 25 Pof. Allan F. Bowe Weilin Deng Simulation of adhesive contact with molecula potential Poject desciption In the poject, we will investigate
More informationQuantum Fourier Transform
Chapte 5 Quantum Fouie Tansfom Many poblems in physics and mathematics ae solved by tansfoming a poblem into some othe poblem with a known solution. Some notable examples ae Laplace tansfom, Legende tansfom,
More informationElementary Statistics and Inference. Elementary Statistics and Inference. 11. Regression (cont.) 22S:025 or 7P:025. Lecture 14.
Elementay tatistics and Infeence :05 o 7P:05 Lectue 14 1 Elementay tatistics and Infeence :05 o 7P:05 Chapte 10 (cont.) D. Two Regession Lines uppose two vaiables, and ae obtained on 100 students, with
More informationHammerstein Model Identification Based On Instrumental Variable and Least Square Methods
Intenational Jounal of Emeging Tends & Technology in Compute Science (IJETTCS) Volume 2, Issue, Januay Febuay 23 ISSN 2278-6856 Hammestein Model Identification Based On Instumental Vaiable and Least Squae
More informationHydroelastic Analysis of a 1900 TEU Container Ship Using Finite Element and Boundary Element Methods
TEAM 2007, Sept. 10-13, 2007,Yokohama, Japan Hydoelastic Analysis of a 1900 TEU Containe Ship Using Finite Element and Bounday Element Methods Ahmet Egin 1)*, Levent Kaydıhan 2) and Bahadı Uğulu 3) 1)
More informationMachine Learning and Rendering
East Building, Balloom BC nvidia.com/siggaph2018 Machine Leaning and Rendeing Alex Kelle, Diecto of Reseach Machine Leaning and Rendeing Couse web page at https://sites.google.com/site/mlandendeing/ 14:00
More informationRejection Based Face Detection
Rejection Based Face Detection Michael Elad* Scientific Computing and Computational Mathematics Stanfod Univesit The Compute Vision Wokshop Mach 18 th, 00 * Collaboation with Y. Hel-O and R. Keshet 1 Pat
More informationIntroduction to Nuclear Forces
Intoduction to Nuclea Foces One of the main poblems of nuclea physics is to find out the natue of nuclea foces. Nuclea foces diffe fom all othe known types of foces. They cannot be of electical oigin since
More informationA Simple Nonparametric Approach to Estimating the Distribution of Random Coefficients in Structural Models
A Simple Nonpaametic Appoach to Estimating the Distibution of Random Coefficients in Stuctual Models Jeemy T. Fox Rice Univesity & NBER Kyoo il Kim Michigan State Univesity May 2016 Chenyu Yang Univesity
More informationExperience Selection in Deep Reinforcement Learning for Control
Jounal of Machine Leaning Reseach 19 (2018) 1-56 Submitted 3/17; Revised 07/18; Published 08/18 Expeience Selection in Deep Reinfocement Leaning fo Contol Tim de Buin Jens Kobe Cognitive Robotics Depatment
More informationHigh precision computer simulation of cyclotrons KARAMYSHEVA T., AMIRKHANOV I. MALININ V., POPOV D.
High pecision compute simulation of cyclotons KARAMYSHEVA T., AMIRKHANOV I. MALININ V., POPOV D. Abstact Effective and accuate compute simulations ae highly impotant in acceleatos design and poduction.
More information1 Explicit Explore or Exploit (E 3 ) Algorithm
2.997 Decision-Making in Lage-Scale Systems Mach 3 MIT, Sping 2004 Handout #2 Lectue Note 9 Explicit Exploe o Exploit (E 3 ) Algoithm Last lectue, we studied the Q-leaning algoithm: [ ] Q t+ (x t, a t
More information2 x 8 2 x 2 SKILLS Determine whether the given value is a solution of the. equation. (a) x 2 (b) x 4. (a) x 2 (b) x 4 (a) x 4 (b) x 8
5 CHAPTER Fundamentals When solving equations that involve absolute values, we usually take cases. EXAMPLE An Absolute Value Equation Solve the equation 0 x 5 0 3. SOLUTION By the definition of absolute
More information1D2G - Numerical solution of the neutron diffusion equation
DG - Numeical solution of the neuton diffusion equation Y. Danon Daft: /6/09 Oveview A simple numeical solution of the neuton diffusion equation in one dimension and two enegy goups was implemented. Both
More information2. Electrostatics. Dr. Rakhesh Singh Kshetrimayum 8/11/ Electromagnetic Field Theory by R. S. Kshetrimayum
2. Electostatics D. Rakhesh Singh Kshetimayum 1 2.1 Intoduction In this chapte, we will study how to find the electostatic fields fo vaious cases? fo symmetic known chage distibution fo un-symmetic known
More informationFluid flow in curved geometries: Mathematical Modeling and Applications
Fluid flow in cuved geometies: Mathematical Modeling and Applications D. Muhammad Sajid Theoetical Plasma Physics Division PINSTECH, P.O. Niloe, PAEC, Islamabad Mach 01-06, 010 Islamabad, Paistan Pesentation
More informationDonnishJournals
DonnishJounals 041-1189 Donnish Jounal of Educational Reseach and Reviews. Vol 1(1) pp. 01-017 Novembe, 014. http:///dje Copyight 014 Donnish Jounals Oiginal Reseach Pape Vecto Analysis Using MAXIMA Savaş
More informationReasons to Build a Hydraulic Model
Reasons to Build a Hydaulic Model Detemine dischage coefficient fo lage flow measuement stuctue (spillway o wei) Develop effective method fo enegy dissipation at outlet of hydaulic stuctue Development
More informationMAPPING LARGE PARALLEL SIMULATION PROGRAMS TO MULTICOMPUTER SYSTEMS
A.Tentne (ed.): High Pefomance Computing 1994, Poc. of the SCS Simulation Multiconfeence 1994, San Diego, 11.-15. Apil 1994. S. 285-290. MAPPING LARGE PARALLEL SIMULATION PROGRAMS TO MULTICOMPUTER SYSTEMS
More informationElectrostatics (Electric Charges and Field) #2 2010
Electic Field: The concept of electic field explains the action at a distance foce between two chaged paticles. Evey chage poduces a field aound it so that any othe chaged paticle expeiences a foce when
More informationPhys-272 Lecture 17. Motional Electromotive Force (emf) Induced Electric Fields Displacement Currents Maxwell s Equations
Phys-7 Lectue 17 Motional Electomotive Foce (emf) Induced Electic Fields Displacement Cuents Maxwell s Equations Fom Faaday's Law to Displacement Cuent AC geneato Magnetic Levitation Tain Review of Souces
More informationQuantum Mechanics II
Quantum Mechanics II Pof. Bois Altshule Apil 25, 2 Lectue 25 We have been dicussing the analytic popeties of the S-matix element. Remembe the adial wave function was u kl () = R kl () e ik iπl/2 S l (k)e
More informationANALYSIS OF PRESSURE VARIATION OF FLUID IN AN INFINITE ACTING RESERVOIR
Nigeian Jounal of Technology (NIJOTECH) Vol. 36, No. 1, Januay 2017, pp. 80 86 Copyight Faculty of Engineeing, Univesity of Nigeia, Nsukka, Pint ISSN: 0331-8443, Electonic ISSN: 2467-8821 www.nijotech.com
More informationExtra notes for circular motion: Circular motion : v keeps changing, maybe both speed and
Exta notes fo cicula motion: Cicula motion : v keeps changing, maybe both speed and diection ae changing. At least v diection is changing. Hence a 0. Acceleation NEEDED to stay on cicula obit: a cp v /,
More informationGround states of stealthy hyperuniform potentials: I. Entropically favored configurations
Gound states of stealthy hypeunifom potentials: I. Entopically favoed configuations G. Zhang and F. H. Stillinge Depatment of Chemisty, Pinceton Univesity, Pinceton, New Jesey 8544, USA space, finding
More informationObjectives: After finishing this unit you should be able to:
lectic Field 7 Objectives: Afte finishing this unit you should be able to: Define the electic field and explain what detemines its magnitude and diection. Wite and apply fomulas fo the electic field intensity
More informationParticle Systems. University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell
Paticle Systems Univesity of Texas at Austin CS384G - Compute Gaphics Fall 2010 Don Fussell Reading Requied: Witkin, Paticle System Dynamics, SIGGRAPH 97 couse notes on Physically Based Modeling. Witkin
More informationPart V: Closed-form solutions to Loop Closure Equations
Pat V: Closed-fom solutions to Loop Closue Equations This section will eview the closed-fom solutions techniques fo loop closue equations. The following thee cases will be consideed. ) Two unknown angles
More information, the tangent line is an approximation of the curve (and easier to deal with than the curve).
114 Tangent Planes and Linea Appoimations Back in-dimensions, what was the equation of the tangent line of f ( ) at point (, ) f ( )? (, ) ( )( ) = f Linea Appoimation (Tangent Line Appoimation) of f at
More informationRecent Advances in Chemical Engineering, Biochemistry and Computational Chemistry
Themal Conductivity of Oganic Liquids: a New Equation DI NICOLA GIOVANNI*, CIARROCCHI ELEONORA, PIERANTOZZI ARIANO, STRYJEK ROAN 1 DIIS, Univesità Politecnica delle ache, 60131 Ancona, ITALY *coesponding
More informationScientific Computing II
Scientific Computing II Conjugate Gadient Methods Michael Bade Summe 2014 Conjugate Gadient Methods, Summe 2014 1 Families of Iteative Solves elaxation methods: Jacobi-, Gauss-Seidel-Relaxation,... Ove-Relaxation-Methods
More informationPHYS Summer Professor Caillault Homework Solutions. Chapter 5
PHYS 1111 - Summe 2007 - Pofesso Caillault Homewok Solutions Chapte 5 7. Pictue the Poblem: The ball is acceleated hoizontally fom est to 98 mi/h ove a distance of 1.7 m. Stategy: Use equation 2-12 to
More informationFresnel Diffraction. monchromatic light source
Fesnel Diffaction Equipment Helium-Neon lase (632.8 nm) on 2 axis tanslation stage, Concave lens (focal length 3.80 cm) mounted on slide holde, iis mounted on slide holde, m optical bench, micoscope slide
More informationMECHANICAL PULPING REFINER MECHANICAL PULPS
MECHANICAL PULPING REFINER MECHANICAL PULPS Histoy of efine mechanical pulping Fo many yeas all mechanical pulp was made fom stone goundwood (SGW). This equied whole logs. Stating in the 950s, but eally
More informationHINDCASTING OF WIND AND WAVE CLIMATE OF SEAS AROUND RUSSIA
Leonid J. Lopatoukhin, Alexande V. Boukhanovsky, Ecateina S. Chenysheva, Segey V. Ivanov HINDCASTING OF WIND AND WAVE CLIMATE OF SEAS AROUND RUSSIA St. Petesbug State Univesity, Institute fo High Pefomance
More informationCBE Transport Phenomena I Final Exam. December 19, 2013
CBE 30355 Tanspot Phenomena I Final Exam Decembe 9, 203 Closed Books and Notes Poblem. (20 points) Scaling analysis of bounday laye flows. A popula method fo measuing instantaneous wall shea stesses in
More informationFUSE Fusion Utility Sequence Estimator
FUSE Fusion Utility Sequence Estimato Belu V. Dasaathy Dynetics, Inc. P. O. Box 5500 Huntsville, AL 3584-5500 belu.d@dynetics.com Sean D. Townsend Dynetics, Inc. P. O. Box 5500 Huntsville, AL 3584-5500
More informationEstimation of the Correlation Coefficient for a Bivariate Normal Distribution with Missing Data
Kasetsat J. (Nat. Sci. 45 : 736-74 ( Estimation of the Coelation Coefficient fo a Bivaiate Nomal Distibution with Missing Data Juthaphon Sinsomboonthong* ABSTRACT This study poposes an estimato of the
More informationOSCILLATIONS AND GRAVITATION
1. SIMPLE HARMONIC MOTION Simple hamonic motion is any motion that is equivalent to a single component of unifom cicula motion. In this situation the velocity is always geatest in the middle of the motion,
More informationAQI: Advanced Quantum Information Lecture 2 (Module 4): Order finding and factoring algorithms February 20, 2013
AQI: Advanced Quantum Infomation Lectue 2 (Module 4): Ode finding and factoing algoithms Febuay 20, 203 Lectue: D. Mak Tame (email: m.tame@impeial.ac.uk) Intoduction In the last lectue we looked at the
More informationTeachers notes. Beyond the Thrills excursions. Worksheets in this book. Completing the worksheets
Beyond the Thills excusions Teaches notes Physics is the science of how the wold (and Univese) woks. Luna Pak Sydney is a lage hands-on physics laboatoy full of fee falling objects, otating systems and
More informationBayesian Analysis of Topp-Leone Distribution under Different Loss Functions and Different Priors
J. tat. Appl. Po. Lett. 3, No. 3, 9-8 (6) 9 http://dx.doi.og/.8576/jsapl/33 Bayesian Analysis of Topp-Leone Distibution unde Diffeent Loss Functions and Diffeent Pios Hummaa ultan * and. P. Ahmad Depatment
More informationResearch Article On Alzer and Qiu s Conjecture for Complete Elliptic Integral and Inverse Hyperbolic Tangent Function
Abstact and Applied Analysis Volume 011, Aticle ID 697547, 7 pages doi:10.1155/011/697547 Reseach Aticle On Alze and Qiu s Conjectue fo Complete Elliptic Integal and Invese Hypebolic Tangent Function Yu-Ming
More information7.2. Coulomb s Law. The Electric Force
Coulomb s aw Recall that chaged objects attact some objects and epel othes at a distance, without making any contact with those objects Electic foce,, o the foce acting between two chaged objects, is somewhat
More informationChapter 9 Dynamic stability analysis III Lateral motion (Lectures 33 and 34)
Pof. E.G. Tulapukaa Stability and contol Chapte 9 Dynamic stability analysis Lateal motion (Lectues 33 and 34) Keywods : Lateal dynamic stability - state vaiable fom of equations, chaacteistic equation
More informationPulse Neutron Neutron (PNN) tool logging for porosity Some theoretical aspects
Pulse Neuton Neuton (PNN) tool logging fo poosity Some theoetical aspects Intoduction Pehaps the most citicism of Pulse Neuton Neuon (PNN) logging methods has been chage that PNN is to sensitive to the
More informationCHAPTER 3. Section 1. Modeling Population Growth
CHAPTER 3 Section 1. Modeling Population Gowth 1.1. The equation of the Malthusian model is Pt) = Ce t. Apply the initial condition P) = 1. Then 1 = Ce,oC = 1. Next apply the condition P1) = 3. Then 3
More informationAbsolute Specifications: A typical absolute specification of a lowpass filter is shown in figure 1 where:
FIR FILTER DESIGN The design of an digital filte is caied out in thee steps: ) Specification: Befoe we can design a filte we must have some specifications. These ae detemined by the application. ) Appoximations
More informationNumerical solution of diffusion mass transfer model in adsorption systems. Prof. Nina Paula Gonçalves Salau, D.Sc.
Numeical solution of diffusion mass tansfe model in adsoption systems Pof., D.Sc. Agenda Mass Tansfe Mechanisms Diffusion Mass Tansfe Models Solving Diffusion Mass Tansfe Models Paamete Estimation 2 Mass
More informationFE FORMULATIONS FOR PLASTICITY
G These slides ae designed based on the book: Finite Elements in Plasticity Theoy and Pactice, D.R.J. Owen and E. Hinton, 970, Pineidge Pess Ltd., Swansea, UK. Couse Content: A INTRODUCTION AND OVERVIEW
More information6 PROBABILITY GENERATING FUNCTIONS
6 PROBABILITY GENERATING FUNCTIONS Cetain deivations pesented in this couse have been somewhat heavy on algeba. Fo example, detemining the expectation of the Binomial distibution (page 5.1 tuned out to
More informationOn the Poisson Approximation to the Negative Hypergeometric Distribution
BULLETIN of the Malaysian Mathematical Sciences Society http://mathusmmy/bulletin Bull Malays Math Sci Soc (2) 34(2) (2011), 331 336 On the Poisson Appoximation to the Negative Hypegeometic Distibution
More informationHopefully Helpful Hints for Gauss s Law
Hopefully Helpful Hints fo Gauss s Law As befoe, thee ae things you need to know about Gauss s Law. In no paticula ode, they ae: a.) In the context of Gauss s Law, at a diffeential level, the electic flux
More information17.1 Electric Potential Energy. Equipotential Lines. PE = energy associated with an arrangement of objects that exert forces on each other
Electic Potential Enegy, PE Units: Joules Electic Potential, Units: olts 17.1 Electic Potential Enegy Electic foce is a consevative foce and so we can assign an electic potential enegy (PE) to the system
More informationGENERALIZED STATISTICAL METHODS FOR UNSUPERVISED MINORITY CLASS DETECTION IN MIXED DATA SETS. Uwe F. Mayer
GENERALIZED STATISTICAL METHODS FOR UNSUPERVISED MINORITY CLASS DETECTION IN MIXED DATA SETS Cécile Levasseu Jacobs School of Engineeing Univesity of Califonia, San Diego La Jolla, CA, USA clevasseu@ucsd.edu
More informationDiscrete LQ optimal control with integral action: A simple controller on incremental form for MIMO systems
Modeling, Identification and Contol, Vol., No., 1, pp. 5, ISSN 189 18 Discete LQ optimal contol with integal action: A simple contolle on incemental fom fo MIMO systems David Di Ruscio Telemak Univesity
More informationInformation Retrieval Advanced IR models. Luca Bondi
Advanced IR models Luca Bondi Advanced IR models 2 (LSI) Pobabilistic Latent Semantic Analysis (plsa) Vecto Space Model 3 Stating point: Vecto Space Model Documents and queies epesented as vectos in the
More informationMath 301: The Erdős-Stone-Simonovitz Theorem and Extremal Numbers for Bipartite Graphs
Math 30: The Edős-Stone-Simonovitz Theoem and Extemal Numbes fo Bipatite Gaphs May Radcliffe The Edős-Stone-Simonovitz Theoem Recall, in class we poved Tuán s Gaph Theoem, namely Theoem Tuán s Theoem Let
More informationSTABILITY AND PARAMETER SENSITIVITY ANALYSES OF AN INDUCTION MOTOR
HUNGARIAN JOURNAL OF INDUSTRY AND CHEMISTRY VESZPRÉM Vol. 42(2) pp. 109 113 (2014) STABILITY AND PARAMETER SENSITIVITY ANALYSES OF AN INDUCTION MOTOR ATTILA FODOR 1, ROLAND BÁLINT 1, ATTILA MAGYAR 1, AND
More information