机器学习笔记-1 Linear Regression with Multiple Variables(week 2)

Posted 2020-09-15

 

1. Multiple Features

note:X0 is equal to 1

2. Feature Scaling

Idea: make sure features are on a similiar scale, approximately a -1<X_i<1 range

For example:

x1 = size (0-2000 feet^2) max-min or standard deviation

x2 = number of bedrooms(1-5)

The contour function of theta1 and theat2 is a very skewed elliptical shape. And 

if you are running gradient descent on this, your gradients may end up a long time

to fjnd the global minimum.

Cure:

x1 = size (0-5000 feet^2)/2000

x2 = number of bedrooms(1-5)/5

so the coutour plots will give a much more direct path to the minimum

Mean normalization:

Replace X_i with X_i  -  U_i  to make features have zero mean(except X0)

Eg:

X1 = (size-1000)/2000

X2= (#bedrooms-2)/5

3. Learning Rate

We can plot the J(theata) vs number of iterations and the J(theata) should

decrease after every iteraion. and we can also see if the gradient descent converges or not.

And if gradient descent is not working, usually means that:

you should use a smaller value of alpha(learning rate)

To choose alpha():

..., 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1...

4. Features

you can try to define new features, for example:

Area = frontage * depth

Polynomial regression:

we can set that x1=size, x2=(size)^2, x3=(size)^3(remember ot feature scaling)

and it becomes linear regression

5. Normal Equations

Idea: method to solve for theta analytically

where  x is m*(n-1) dimensional matrix and y is a m dimensional matrix,

n : number of features, m:number of training example

And feature scaling is not necessary for normal equations

Gradient descent 

1. choose alpha

2. need many iterations

3. works well even have large number of features n.

Normal equation:
1. no need for alpha and iterations

2. need to compute matrix inverse

3. slow for large n (n = 10^6 etc)

Note  is not invertible means that:

1. you have got redundant features(linearly dependent)

2. there are too many features, delete some features, or use regularization