02 Multivariate Linear Regression

Posted qq-1615160629

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了02 Multivariate Linear Regression相关的知识,希望对你有一定的参考价值。

  1. h(x)
    \[
    \beginalign*h_\theta(x) =\beginbmatrix\theta_0 \hspace2em \theta_1 \hspace2em ... \hspace2em \theta_n\endbmatrix\beginbmatrixx_0 \newline x_1 \newline \vdots \newline x_n\endbmatrix= \theta^T x\endalign*, x_0^(i) = 1
    \]

  2. Gradient descent equation
    \[
    \beginalign*& \textrepeat until convergence: \; \lbrace \newline \; & \theta_j := \theta_j - \alpha \frac1m \sum\limits_i=1^m (h_\theta(x^(i)) - y^(i)) \cdot x_j^(i) \; & \textfor j := 0...n\newline \rbrace\endalign*
    \]

  3. 当不同特征的值差距过大\((>10^5)\)时,需要特征缩放(Feature Scaling)

    \[
    x_i := \fracx_i - \mu_is_i
    \]
    Where \(\mu_i\) is the average of all the values for feature(i) and \(s_i\) is the range of values(max - min), or \(s_i\) is the standard deviation.

  4. Learning Rate
    In automatic convergence test, declare convergence if \(J(\theta)\) decreases by less than \(1-^-3\) in one iteration.

  5. Features and Polynomial Regression
    可以将不同的特征值组合来更好的拟合数据,同时因为数据的组合,更加需要特征缩放来加快几何提高精度

  6. Normal Equation 正规方程 不需要特征缩放
    \[
    \theta = (X^TX)^-1X^Ty
    \]

  7. Comparation

    Gradient Descent Normal Equation
    need to choose \(\alpha\) No need to choose \(\alpha\)
    Needs many iterations Don’t need to iterate
    Works well even when n is large (\(>10^4\)) Need to compute \((X^TX)^-1\)
    \(O(kn^2)\) Slow if n is very large \(O(n^3)\)
  8. If \(X^TX\) is noninvertible, the common causes might be having :

    • Redundant features, where two features are very closely related (i.e. they are linearly dependent)
    • Too many features (e.g. m ≤ n). In this case, delete some features or use "regularization" (to be explained in a later lesson).

以上是关于02 Multivariate Linear Regression的主要内容,如果未能解决你的问题,请参考以下文章

Book: The TimeViz Browser

多变量线性回归 matlab

Numpy之高斯分布 multivariate_normal

关于协方差最小化 scipy.stats.multivariate_normal.logpdf

Multivariate Gaussians

调用 scipy.stats.multivariate_normal 后,pylab.plot“无法将浮点 NaN 转换为整数”