Online Learning in High Dimensions. LWPR and it s application

Transcription

1 Lecture 9 LWPR Online Learning in High Dimensions Contents: LWPR and it s application Sethu Vijayakumar, Aaron D'Souza and Stefan Schaal, Incremental Online Learning in High Dimensions, Neural Computation, vol. 17, no. 12, pp (2005). Stefan Klanke, Sethu Vijayakumar and Stefan Schaal, A Library for Locally Weighted Projection Regression, Journal of Machine Learning Research (JMLR), vol. 9, pp (2008). lwpr Lecture 9: RLSC Prof. Sethu Vijayakumar 1

2 Locally Weighted Projection Regression Projection Regression has many benefits: Computationally cheap Numerically robust Ideally suited for incremental addition of projections Typically, fewer projections required than intrinsic dimensionality Based on our analysis of Dimensionality reduction techniques, we will use PLS as our preprocessing step which also gives as the slope of the local linear model in each receptive field. We then plug this into our LWR learning framework to get a very efficient incremental learning system, i.e. LWPR Lecture 9: RLSC Prof. Sethu Vijayakumar 2

3 Locally Linear Models Cleveland 79 Farmer et al. 88 Atkeson 89 Region of Validity 2 k Receptive Field Activation w 1 Linear Model 0 If we can find the tangent (plane) and the region of validity from only local data, the function approximation problem can be solved efficiently Lecture 9: RLSC Prof. Sethu Vijayakumar 3

4 Formalization of Local Models The Linear Model The Kernel Function y x T x 0 T x where x x T 1 Open Parameters T The Prediction w exp 1 2 x c T Dx c y K w i y k i1 K w i i1 where D M T M Lecture 9: RLSC Prof. Sethu Vijayakumar 4

5 LWL in High Dimensional Spaces The Curse of Dimensionality The power of local learning comes from exploiting the discriminative power of local neighborhood relations, but the notion of a local breaks down in high dim. spaces Lecture 9: RLSC Prof. Sethu Vijayakumar 5

6 Human Movement Measure arm movement and full body movement of humans and anthropomorphic robots Perform local dimensionality analysis with a growing variational mixture of factor analyzers Lecture 9: RLSC Prof. Sethu Vijayakumar 6

7 Dimensionality of Full Body Motion Cumulative Variance Global PCA 32 0 / / No. of retained dimensions / / Probabilit y Dimensionality About 8 dimensions in the space formed by joint positions, velocities, and accelerations are needed to model an inverse dynamics model Local FA / / / / 105 Lecture 9: RLSC Prof. Sethu Vijayakumar 7

8 Exploiting Locally Low Dim. Data Use local dimensionality reduction techniques PCA Factor Analysis Partial Least Squares Regression T y T x x 0 T s x where x T s x T 1 Regression Along One Dimensional Projections u X T y Determine Note: projection for spherical from input correlation s Xu Project distributions, the input data PLS performs st y optimally with just one Perform single projection variate regression Iterate multiple times on residual error Lecture 9: RLSC Prof. Sethu Vijayakumar 8

9 Learning the distance metric D n1 D n δj δd J 1 W M r 2 i res, k 1, i 2 i1 k 1 sk, i 2 (1 wi ) T sk Wsk w y N i, j1 D 2 ij Regularization on the distance metric Lecture 9: RLSC Prof. Sethu Vijayakumar 9

10 Incremental Dimensionality Reduction Incremental PLS Sufficient statistics Lambda = Forgetting factor Lecture 9: RLSC Prof. Sethu Vijayakumar 10

11 Incremental Metric Adjustment Incremental Distance Metric Update Lecture 9: RLSC Prof. Sethu Vijayakumar 11

12 Processing Units of LWPR Lecture 9: RLSC Prof. Sethu Vijayakumar 12

13 Summary LWPR algorithm Fit the non linear function with sum of weighted local linear models In each of these local models, do a local dimensionality reduction for regression (orthogonal projections) Learn the locality of the models incrementally Key point: No competition while learning individual local models (unlike mixture models) Lecture 9: RLSC Prof. Sethu Vijayakumar 13

14 Lecture 9: RLSC Prof. Sethu Vijayakumar 14 Example: 1D Fitting y x original training data new training data true y predicted y predicted y after new training data w x c) Learned Organization of Receptive Fields y x b) Local Function Fitting With Receptive Fields a) Global Function Fitting With Sigmoidal Neural Network

15 Empirical Evaluations (Cross Data) Input Dimensionality = 2 (+ 8 or 18 redundant dimensions.) Noise ~ N(0,0.01) # training data = 500 Target function Learned Initial Receptive Fields Learned function Lecture 9: RLSC Prof. Sethu Vijayakumar 15

16 Empirical Evaluations (Cross Data) Low nmse achieved inspite of redundant inputs Lecture 9: RLSC Prof. Sethu Vijayakumar 16

17 Inv. Dynamics Learning (7 DOF) Inverse dynamics of a 7DOF anthropomorphic robot arm SARCOS dexterous arm τ f f (θ, θ, θ ) 21 7 :... We add an additional 29 redundant input dimensions to verify the redundancy handling ability of our algorithm f 50 7 : Lecture 9: RLSC Prof. Sethu Vijayakumar 17

18 Learning results & comparison Lecture 9: RLSC Prof. Sethu Vijayakumar 18

19 Learning Feedforward Controller sensor feedback Lecture 9: RLSC Prof. Sethu Vijayakumar 19

20 Online Learning with 7DOF Robot Arm PD control no dynamics model Learning the Dynamics Model Lecture 9: RLSC Prof. Sethu Vijayakumar 20

21 Online learning with Robot Arm Reaching & pole balancing Learning Trials Lecture 9: RLSC Prof. Sethu Vijayakumar 21

22 Adaptation to Changed Dynamics Learning new dynamics online Adaptation in Humanoid Online Dynamics Adaptation f : Lecture 9: RLSC Prof. Sethu Vijayakumar 22