吴恩达机器学习笔记

Posted 2022-11-28 未来可期-2018

文章目录

• • Error analysis
  • • Methods to solve over fitting
    • Methods to solve under fitting
    • Recommend approach
    • Error metrics for skewed classes
    • Data for machine learning
  • Support Vector Machine
  • K-means
  • Principle Component Analysis
  • • Data preprocessing
    • Choosing the number of principal components
    • Application of `PCA`
  • Anomaly detection
  • • Examples
    • Anomaly detection algorithm
    • Algorithm evaluation
    • Anomaly detection vs supervised learning
    • Choosing what features to use
    • Multivariate Gaussian distribution
  • Recommender Systems
  • • Content-based recommendations
    • Collaborative filtering
  • Learning with large datasets
  • • Stochastic gradient descent
    • Mini-batch gradient descent
    • Online learning
  • Application example Photo OCR
  • • Ceiling analysis

Error analysis

Methods to solve over fitting

• more training examples
• try smaller sets of features
• try increasing $\lambda$

Methods to solve under fitting

• getting additional features
• try adding polynomial features
• try decreasing $\lambda$

Recommend approach

• start with a simple algorithm that you can implement quickly. Implement it and test it on your cross-validation data
• plot learning curves to decide if more data, more features, etc. are likely to help.
• Error analysis: See if you spot any systematic trend in what type of examples it is making errors on

Error metrics for skewed classes

	1	0
1	True Positive	False Positive
0	False Negative	True Negative

$precision=\frac{TP}{TP+FP}$ 准确率

$recall=\frac{TP}{TP+FN}$ 召回率

$F_{1}score=2\frac{PR}{P+R}$ F值

Data for machine learning

In the following conditions, more data makes sense.

• Assume feature $x\in R^{n+1}$ has sufficient information to predict y accurately.
• Use a learning algorithm with many parameters such as logistic regression or linear regression with many features or neural network with many hidden units.

Support Vector Machine

Logistic regression Cost function
$$mi{n}_{\theta }\frac{1}{m}[\sum _{i=1}^m{y}^{(i)}(-\log h_{\theta }(x^{(i)}))+(1-y^{(i)})(-\log (1-h_{\theta }(x^{(i)})))]+\frac{\lambda }{2m}\sum _{j=1}^n{\theta }_j^2$$
SVM hypothesis
$$mi{n}_{\theta }C\sum _{i=1}^m[y^{(i)}cos{t}_1({\theta }^T{x}^{(i)})+(1-y^{(i)})cos{t}_0({\theta }^T{x}^{(i)})]+\frac{1}{2}\sum _{i=1}^n{\theta }_j^2$$

$$h_{\theta }(x)=\begin{array}{l}0\\ 1\end{array}$$

SVM parameters

$C=\frac{1}{\lambda }$

• large C low bias, high variance
• small C higher bias, low variance

${\sigma }^2$

• large ${\sigma }^2$. feature $f_i$ vary more smoothly. higher bias, lower variance
• small ${\sigma }^2$. feature $f_i$ vary less smoothly. lower bias, higher variance

Kernel function

• no kernel

• 高斯kernel
  $$f=e^{-\frac{||x_1-x_2||}{2{\sigma }^2}}$$

• 多项式kernel $k(x,l)=(x^{T}l+constant{)}^{degree}$

Muti-class classification

train K SVMs, one to distinguish $y=i$ from the rest

Logistic regression vs SVMs

n=number of features $x\in R^{n+1}$，m=number of training examples

• n is large relative to m: use logistic regression or SVM without kernel
• n is small , m is intermediate: use SVM with Gaussian kernel
• n is small and m is large: create or add more features and then use logistic regression or svm without kernel

K-means

Input

• K (number of clusters)
• training set x ( 1 ) , x ( 2 ) , ⋯   , x ( m ) \\x^(1),x^(2),\\cdots,x^(m)\\ x(1),x(2),⋯,吴恩达机器学习笔记

  下载量过百万的吴恩达机器学习和深度学习笔记更新了！（附PDF下载）

  吴恩达《机器学习系列课程》学习笔记

  吴恩达机器学习学习笔记——代价函数

  吴恩达“机器学习”——学习笔记二

  吴恩达《机器学习系列课程》学习笔记：监督学习