Linear Regression with mul2ple variables. Mul2ple features. Machine Learning

Transcription

1 Linear Regression with mul2ple variables Mul2ple features Machine Learning

2 Mul4ple features (variables). Size (feet 2 ) Price ($1000)

3 Mul4ple features (variables). Size (feet 2 ) Number of bedrooms Number of floors Age of home (years) Price ($1000) Nota2on: = number of features = input (features) of training example. = value of feature in training example.

4 Hypothesis: Previously:

5 For convenience of nota2on, define. Mul2variate linear regression.

6 Linear Regression with mul2ple variables Gradient descent for mul2ple variables Machine Learning

7 Hypothesis: Parameters: Cost func2on: Gradient descent: Repeat (simultaneously update for every )

8 Gradient Descent Previously (n=1): Repeat New algorithm : Repeat (simultaneously update ) for (simultaneously update )

9 Linear Regression with mul2ple variables Gradient descent in prac2ce I: Feature Scaling Machine Learning

10 Feature Scaling Idea: Make sure features are on a similar scale. E.g. = size ( feet 2 ) = number of bedrooms (1-5) size (feet 2 ) number of bedrooms

11 Feature Scaling Get every feature into approximately a range.

12 Mean normaliza4on Replace with to make features have approximately zero mean (Do not apply to ). E.g.

13 Linear Regression with mul2ple variables Gradient descent in prac2ce II: Learning rate Machine Learning

14 Gradient descent - Debugging : How to make sure gradient descent is working correctly. - How to choose learning rate.

15 Making sure gradient descent is working correctly. Example automa2c convergence test: No. of itera2ons Declare convergence if decreases by less than in one itera2on.

16 Making sure gradient descent is working correctly. Gradient descent not working. Use smaller. No. of itera2ons No. of itera2ons No. of itera2ons - For sufficiently small, should decrease on every itera2on. - But if is too small, gradient descent can be slow to converge.

17 Summary: - If is too small: slow convergence. - If is too large: may not decrease on every itera2on; may not converge. To choose, try

18 Linear Regression with mul2ple variables Features and polynomial regression Machine Learning

19 Housing prices predic4on

20 Polynomial regression Price (y) Size (x)

21 Choice of features Price (y) Size (x)

22 Linear Regression with mul2ple variables Normal equa2on Machine Learning

23 Gradient Descent Normal equa2on: Method to solve for analy2cally.

24 Intui2on: If 1D (for every ) Solve for

25 Examples: Size (feet 2 ) Number of bedrooms Number of floors Age of home (years) Price ($1000)

26 examples ; features. E.g. If

27 is inverse of matrix. Octave: pinv(x *X)*X *y

28 training examples, Gradient Descent features. Normal Equa2on Need to choose. Needs many itera2ons. Works well even when is large. No need to choose. Don t need to iterate. Need to compute Slow if is very large.

29 Linear Regression with mul2ple variables Normal equa2on and non- inver2bility Machine Learning (op2onal)

30 Normal equa2on - What if is non- inver2ble? (singular/ degenerate) - Octave: pinv(x *X)*X *y

31 What if is non- inver2ble? Redundant features (linearly dependent). E.g. size in feet 2 size in m 2 Too many features (e.g. ). - Delete some features, or use regulariza2on.