udacity机器学习-支持向量机(to be continued)

Posted pandaboy1123

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了udacity机器学习-支持向量机(to be continued)相关的知识,希望对你有一定的参考价值。

Evernote Export

支持向量机(Support Vector Machine)

不适定问题不止一个决策边界
技术分享图片
要找一个决策边界,不仅能将训练集很好的划分,而且提升模型的泛化能力
支持向量机直接将算法放在运行的内部,在不适定的问题中,使用svm去建模是好的
svm是统计学习中非常重要的方法
svm尝试寻找一个最优的决策边界,距离两个类别的最近的样本最远,距离决策边界最近的点称为支撑向量
svm算法要做的就是最大化margin,也就是要找到最大的d

margin=2d

技术分享图片

解析几何,点到直线的距离

(x,y)到Ax+By+C=0的距离A2+B2技术分享图片?技术分享图片Ax+By+C?
扩展到n维空间θTxb?=0-->wTx+b=0-->w技术分享图片wT+b?
w=w12?+w22?+...+wn2?技术分享图片?
w技术分享图片wTxi+b?d 其中?yi=1
w技术分享图片wTxi+b??d 其中?yi=?1
wd技术分享图片wTxi+b?1 其中?yi=1
wd技术分享图片wTxi+b??1 其中?yi=?1
wdT?xi+bd?1其中?yi=1
wdT?xi+bd??1其中?yi=?1
技术分享图片

yi(wTxi+b)1
对于任意支撑向量x有 maxw技术分享图片wTx+b?-->maxw技术分享图片1?-->min2技术分享图片1?w2

Soft Margin和SVM的正则化

Soft Margin SVM yi(wTxi+b)1?ζ
ζi?0
min2技术分享图片1?w2+C?Σi=1m?ζi?
C为超参数,平衡两部分的重要程度

使用SVM需要对数据进行标准化处理
对于SVM上,数据尺度不同,需要对数据进行标准化处理

什么是核函数

技术分享图片

svm的本质:其实就是求解最优化问题,求解最优化过程中,需要变形为数学中最好解决的问题
技术分享图片
核函数思想:K(xi,xk)=xi,xj
K(x,y)=(x?y+1)2

K(x,y)=(i=1n?x?y+1)2=i=1n?(xi2?)(yi2?)+i=2n?j=1i?1?(2技术分享图片?xi?xj?)(2技术分享图片?yi?yj?)+i=1n?(2技术分享图片?xi?)(2技术分享图片?yi?)+1

核函数会降低原算法的复杂度

K(x,y)=(x?y+c)d

d就是函数里的degree

高斯核函数

K(x,y)表示x和y的点乘

K(x,y)=e?γx?y2

正态分布就是高斯函数 g(x)=σ2π技术分享图片?技术分享图片1?e?2技术分享图片1?(σ技术分享图片x?μ?)2
RBF核 Radial Basis Function Kernel
将每一个样本点映射到一个无穷维的特征空间
多项式特征依靠升维是的原本线性不可分的数据线性可分

import numpy as np
import matplotlib.pyplot as plt

from sklearn import datasets
iris = datasets.load_iris()
X = iris.data
y = iris.target
X = X[y < 2, :2]
y = y[y < 2]

plt.scatter(X[y == 0, 0], X[y == 0, 1], color=‘red‘)
plt.scatter(X[y == 1, 0], X[y == 1, 1], color=‘blue‘)
plt.show()

技术分享图片

from sklearn.preprocessing import StandardScaler
standardScaler = StandardScaler()
standardScaler.fit(X)
X_standard = standardScaler.transform(X)
from sklearn.svm import LinearSVC
#SVC使用向量做分类
svc = LinearSVC(C=1e9)
svc.fit(X_standard, y)

def plot_decision_boundary(model, axis):
    x0,x1 = np.meshgrid(
        np.linspace(axis[0],axis[1],int((axis[1]-axis[0])*100)).reshape(-1,1),
        np.linspace(axis[2],axis[3],int((axis[3]-axis[2])*100)).reshape(-1,1),
    )
    X_new = np.c_[x0.ravel(),x1.ravel()]
    y_predict = model.predict(X_new)
    zz = y_predict.reshape(x0.shape)
    from matplotlib.colors import ListedColormap
    custom_camp = ListedColormap([‘#EF9A9A‘,‘#FFF59D‘,‘#90CAF9‘])
    plt.contourf(x0, x1, zz, linewith=5, camp=custom_camp)
    w = model.coef_[0]
    b = model.intercept_[0]
    plot_x = np.linspace(axis[0], axis[1], 200)
    up_y = -w[0]/w[1]*plot_x-b/w[1]+1/w[1]
    down_y = -w[0]/w[1]*plot_x-b/w[1]-1/w[1]
    up_index = (up_y>=axis[2])&(up_y<=axis[3])
    down_index = (down_y>=axis[2])&(down_y<=axis[3])
    plt.plot(plot_x[up_index],up_y[up_index],color=‘black‘)
    plt.plot(plot_x[down_index],down_y[down_index],col

plot_decision_boundary(svc, axis=[-3,3,-3,3])
plt.scatter(X_standard[y==0,0],X_standard[y==0,1])
plt.scatter(X_standard[y==1,0],X_standard[y==1,1])
plt.show()

技术分享图片

技术分享图片

svc2 = LinearSVC(C=0.01)
svc2.fit(X_standard, y)
plot_decision_boundary(svc2, axis=[-3,3,-3,3])
plt.scatter(X_standard[y==0,0],X_standard[y==0,1])
plt.scatter(X_standard[y==1,0],X_standard[y==1,1])
plt.show()

技术分享图片
技术分享图片

#多项式处理非线性问题
import numpy as npimport matplotlib.pyplot as plt
from sklearn import datasets
X, y = datasets.make_moons()
plt.scatter(X[y==0,0],X[y==0,1])plt.scatter(X[y==1,0],X[y==1,1])plt.show()

技术分享图片

#构建数据集
X,y = datasets.make_moons(noise=0.15, random_state=666)
plt.scatter(X[y==0,0],X[y==0,1])plt.scatter(X[y==1,0],X[y==1,1])plt.show()

技术分享图片

from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.svm import LinearSVC
from sklearn.pipeline import Pipeline
def PolynomialSVC(degree,C=1.0):
    return Pipeline([
        ("poly", PolynomialFeatures(degree=degree)),
        ("std_scaler", StandardScaler()),
        ("linearSVC", LinearSVC(C=C)),
    ])
poly_svc = PolynomialSVC(degree=3)
poly_svc.fit(X,y)

def plot_decision_boundary(model, axis):
    x0,x1 = np.meshgrid(
        np.linspace(axis[0],axis[1],int((axis[1]-axis[0])*100)).reshape(-1,1),
        np.linspace(axis[2],axis[3],int((axis[3]-axis[2])*100)).reshape(-1,1),
    )
    X_new = np.c_[x0.ravel(),x1.ravel()]
    y_predict = model.predict(X_new)
    zz = y_predict.reshape(x0.shape)
    from matplotlib.colors import ListedColormap
    custom_camp = ListedColormap([‘#EF9A9A‘,‘#FFF59D‘,‘#90CAF9‘])
plt.contourf(x0, x1, zz, linewith=5, camp=custom_camp)
plot_decision_boundary(poly_svc,axis=[-1.5,2.5,-1.0,1.5])
plt.scatter(X[y==0,0], X[y==0,1])
plt.scatter(X[y==1,0], X[y==1,1])
plt.show()

技术分享图片

#多项式核函数
from sklearn.svm import SVC
def PolynomialKernelSVC(degree, C=1.0):
    return Pipeline([
        ("std_scaler", StandardScaler()),
        ("kernelSVC", SVC(kernel=‘poly‘,degree=degree,C=C)),
    ])
poly_kernel_svc = PolynomialKernelSVC(degree=3)
poly_kernel_svc.fit(X,y)
plot_decision_boundary(poly_kernel_svc,axis=[-1.5,2.5,-1.0,1.5])
plt.scatter(X[y==0,0], X[y==0,1])
plt.scatter(X[y==1,0], X[y==1,1])
plt.show()

技术分享图片

%23%23%23%23%20%E6%94%AF%E6%8C%81%E5%90%91%E9%87%8F%E6%9C%BA(Support%20Vector%20Machine)%0A**%E4%B8%8D%E9%80%82%E5%AE%9A%E9%97%AE%E9%A2%98**%E4%B8%8D%E6%AD%A2%E4%B8%80%E4%B8%AA%E5%86%B3%E7%AD%96%E8%BE%B9%E7%95%8C%0A!%5B0caf6e4b9485ba12d98a1e4a3baab505.png%5D(en-resource%3A%2F%2Fdatabase%2F1248%3A0)%0A%E8%A6%81%E6%89%BE%E4%B8%80%E4%B8%AA%E5%86%B3%E7%AD%96%E8%BE%B9%E7%95%8C%EF%BC%8C%E4%B8%8D%E4%BB%85%E8%83%BD%E5%B0%86%E8%AE%AD%E7%BB%83%E9%9B%86%E5%BE%88%E5%A5%BD%E7%9A%84%E5%88%92%E5%88%86%EF%BC%8C%E8%80%8C%E4%B8%94%E6%8F%90%E5%8D%87%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%B3%9B%E5%8C%96%E8%83%BD%E5%8A%9B%0A%E6%94%AF%E6%8C%81%E5%90%91%E9%87%8F%E6%9C%BA%E7%9B%B4%E6%8E%A5%E5%B0%86%E7%AE%97%E6%B3%95%E6%94%BE%E5%9C%A8%E8%BF%90%E8%A1%8C%E7%9A%84%E5%86%85%E9%83%A8%EF%BC%8C%E5%9C%A8%E4%B8%8D%E9%80%82%E5%AE%9A%E7%9A%84%E9%97%AE%E9%A2%98%E4%B8%AD%EF%BC%8C%E4%BD%BF%E7%94%A8svm%E5%8E%BB%E5%BB%BA%E6%A8%A1%E6%98%AF%E5%A5%BD%E7%9A%84%0Asvm%E6%98%AF%E7%BB%9F%E8%AE%A1%E5%AD%A6%E4%B9%A0%E4%B8%AD%E9%9D%9E%E5%B8%B8%E9%87%8D%E8%A6%81%E7%9A%84%E6%96%B9%E6%B3%95%0A*svm%E5%B0%9D%E8%AF%95%E5%AF%BB%E6%89%BE%E4%B8%80%E4%B8%AA%E6%9C%80%E4%BC%98%E7%9A%84%E5%86%B3%E7%AD%96%E8%BE%B9%E7%95%8C%EF%BC%8C%E8%B7%9D%E7%A6%BB%E4%B8%A4%E4%B8%AA%E7%B1%BB%E5%88%AB%E7%9A%84%E6%9C%80%E8%BF%91%E7%9A%84%E6%A0%B7%E6%9C%AC%E6%9C%80%E8%BF%9C%EF%BC%8C%E8%B7%9D%E7%A6%BB%E5%86%B3%E7%AD%96%E8%BE%B9%E7%95%8C%E6%9C%80%E8%BF%91%E7%9A%84%E7%82%B9%E7%A7%B0%E4%B8%BA%E6%94%AF%E6%92%91%E5%90%91%E9%87%8F*%0A**svm%E7%AE%97%E6%B3%95%E8%A6%81%E5%81%9A%E7%9A%84%E5%B0%B1%E6%98%AF%E6%9C%80%E5%A4%A7%E5%8C%96margin%EF%BC%8C%E4%B9%9F%E5%B0%B1%E6%98%AF%E8%A6%81%E6%89%BE%E5%88%B0%E6%9C%80%E5%A4%A7%E7%9A%84d**%0A%0A%24%24margin%20%3D%202%20d%24%24%0A!%5B12eb0cb0f0748a967f9319b91eb36242.png%5D(en-resource%3A%2F%2Fdatabase%2F1250%3A0)%0A%0A%E8%A7%A3%E6%9E%90%E5%87%A0%E4%BD%95%EF%BC%8C%E7%82%B9%E5%88%B0%E7%9B%B4%E7%BA%BF%E7%9A%84%E8%B7%9D%E7%A6%BB%0A%0A(x%2Cy)%E5%88%B0Ax%2BBy%2BC%3D0%E7%9A%84%E8%B7%9D%E7%A6%BB%24%5Cfrac%7B%7CAx%2BBy%2BC%7C%7D%7B%5Csqrt%7BA%5E2%2BB%5E2%7D%7D%24%0A%E6%89%A9%E5%B1%95%E5%88%B0n%E7%BB%B4%E7%A9%BA%E9%97%B4%24%5Ctheta%5ETx_b%3D0%24--%3E%24w%5ETx%2Bb%3D0%24--%3E%24%5Cfrac%7Bw%5ET%2Bb%7D%7B%7C%7Cw%7C%7C%7D%24%0A%24%7C%7Cw%7C%7C%3D%5Csqrt%7Bw%5E2_1%2Bw_2%5E2%2B...%2Bw%5E2_n%7D%24%0A%24%5Cfrac%7Bw%5ETx%5Ei%2Bb%7D%7B%7C%7Cw%7C%7C%7D%5Cgeq%20d%24%20%20%20%20%20%20%E5%85%B6%E4%B8%AD%24%5Cforall%20y%5Ei%3D1%24%0A%24%5Cfrac%7Bw%5ETx%5Ei%2Bb%7D%7B%7C%7Cw%7C%7C%7D%5Cleq%20-d%24%20%20%20%20%20%20%E5%85%B6%E4%B8%AD%24%5Cforall%20y%5Ei%3D-1%24%0A%24%5Cfrac%7Bw%5ETx%5Ei%2Bb%7D%7B%7C%7Cw%7C%7Cd%7D%5Cgeq%201%24%20%20%20%20%20%20%E5%85%B6%E4%B8%AD%24%5Cforall%20y%5Ei%3D1%24%0A%24%5Cfrac%7Bw%5ETx%5Ei%2Bb%7D%7B%7C%7Cw%7C%7Cd%7D%5Cleq%20-1%24%20%20%20%20%20%20%E5%85%B6%E4%B8%AD%24%5Cforall%20y%5Ei%3D-1%24%0A%24w%5ET_dx%5Ei%2Bb_d%5Cgeq1%24%E5%85%B6%E4%B8%AD%24%5Cforall%20y%5Ei%3D1%24%0A%24w%5ET_dx%5Ei%2Bb_d%5Cleq-1%24%E5%85%B6%E4%B8%AD%24%5Cforall%20y%5Ei%3D-1%24%0A!%5B00ea5f20051de99d61112655ec626c62.png%5D(en-resource%3A%2F%2Fdatabase%2F1256%3A0)%0A%0A%24y%5Ei(w%5ETx%5Ei%2Bb)%5Cgeq1%24%20%0A%E5%AF%B9%E4%BA%8E%E4%BB%BB%E6%84%8F%E6%94%AF%E6%92%91%E5%90%91%E9%87%8Fx%E6%9C%89%20%24max%5Cfrac%7B%7Cw%5ETx%2Bb%7C%7D%7B%7C%7Cw%7C%7C%7D%24--%3E%24max%5Cfrac%7B1%7D%7B%7C%7Cw%7C%7C%7D%24--%3E%24min%5Cfrac%7B1%7D%7B2%7D%7C%7Cw%7C%7C%5E2%24%0A%0A%23%23%23%23%20Soft%20Margin%E5%92%8CSVM%E7%9A%84%E6%AD%A3%E5%88%99%E5%8C%96%0A%0ASoft%20Margin%20SVM%20%24y%5Ei(w%5ETx%5Ei%2Bb)%5Cgeq1-%5Czeta%24%0A%24%5Czeta_i%5Cgeq0%24%0A%24min%5Cfrac%7B1%7D%7B2%7D%7C%7Cw%7C%7C%5E2%2BC%5Ccdot%5CSigma%5Em_%7Bi%3D1%7D%5Czeta_i%24%0AC%E4%B8%BA%E8%B6%85%E5%8F%82%E6%95%B0%EF%BC%8C%E5%B9%B3%E8%A1%A1%E4%B8%A4%E9%83%A8%E5%88%86%E7%9A%84%E9%87%8D%E8%A6%81%E7%A8%8B%E5%BA%A6%0A%0A%E4%BD%BF%E7%94%A8SVM%E9%9C%80%E8%A6%81%E5%AF%B9%E6%95%B0%E6%8D%AE%E8%BF%9B%E8%A1%8C%E6%A0%87%E5%87%86%E5%8C%96%E5%A4%84%E7%90%86%0A%E5%AF%B9%E4%BA%8ESVM%E4%B8%8A%EF%BC%8C%E6%95%B0%E6%8D%AE%E5%B0%BA%E5%BA%A6%E4%B8%8D%E5%90%8C%EF%BC%8C%E9%9C%80%E8%A6%81%E5%AF%B9%E6%95%B0%E6%8D%AE%E8%BF%9B%E8%A1%8C%E6%A0%87%E5%87%86%E5%8C%96%E5%A4%84%E7%90%86%0A%0A%0A%0A%23%23%23%23%20%E4%BB%80%E4%B9%88%E6%98%AF%E6%A0%B8%E5%87%BD%E6%95%B0%0A!%5B5117e2fab8f9aa29439f12627edfde35.png%5D(en-resource%3A%2F%2Fdatabase%2F1258%3A0)%0A%0Asvm%E7%9A%84%E6%9C%AC%E8%B4%A8%EF%BC%9A%E5%85%B6%E5%AE%9E%E5%B0%B1%E6%98%AF%E6%B1%82%E8%A7%A3%E6%9C%80%E4%BC%98%E5%8C%96%E9%97%AE%E9%A2%98%EF%BC%8C%E6%B1%82%E8%A7%A3%E6%9C%80%E4%BC%98%E5%8C%96%E8%BF%87%E7%A8%8B%E4%B8%AD%EF%BC%8C%E9%9C%80%E8%A6%81%E5%8F%98%E5%BD%A2%E4%B8%BA%E6%95%B0%E5%AD%A6%E4%B8%AD%E6%9C%80%E5%A5%BD%E8%A7%A3%E5%86%B3%E7%9A%84%E9%97%AE%E9%A2%98%0A!%5B1cfaf8fbf5fdedf1c8f96c288b5f1f8f.png%5D(en-resource%3A%2F%2Fdatabase%2F1260%3A0)%0A%E6%A0%B8%E5%87%BD%E6%95%B0%E6%80%9D%E6%83%B3%EF%BC%9A%24K(x%5Ei%2Cx%5Ek)%3Dx%5Ei%2Cx%5Ej%24%0A%24K(x%2Cy)%20%3D%20(x%20%5Ccdot%20y%2B1)%5E2%24%0A%24%24K(x%2Cy)%20%3D%20(%5Csum%5En_%7Bi%3D1%7Dx%5Ccdot%20y%2B1)%5E2%0A%20%20%20%20%20%20%20%20%20%20%20%3D%5Csum%5En_%7Bi%3D1%7D(x_i%5E2)(y_i%5E2)%2B%5Csum%5En_%7Bi%3D2%7D%5Csum%5E%7Bi-1%7D_%7Bj%3D1%7D(%5Csqrt%7B2%7Dx_ix_j)(%5Csqrt%7B2%7Dy_iy_j)%2B%5Csum%5En_%7Bi%3D1%7D(%5Csqrt%7B2%7Dx_i)(%5Csqrt%7B2%7Dy_i)%2B1%24%24%0A%E6%A0%B8%E5%87%BD%E6%95%B0%E4%BC%9A%E9%99%8D%E4%BD%8E%E5%8E%9F%E7%AE%97%E6%B3%95%E7%9A%84%E5%A4%8D%E6%9D%82%E5%BA%A6%0A%24%24K(x%2Cy)%20%3D%20(x%5Ccdot%20y%20%2Bc)%5Ed%24%24%0Ad%E5%B0%B1%E6%98%AF%E5%87%BD%E6%95%B0%E9%87%8C%E7%9A%84degree%0A%0A%23%23%23%23%20%E9%AB%98%E6%96%AF%E6%A0%B8%E5%87%BD%E6%95%B0%0AK(x%2Cy)%E8%A1%A8%E7%A4%BAx%E5%92%8Cy%E7%9A%84%E7%82%B9%E4%B9%98%0A%24%24K(x%2Cy)%20%3D%20e%5E%7B-%5Cgamma%7C%7Cx-y%7C%7C%5E2%7D%24%24%0A%E6%AD%A3%E6%80%81%E5%88%86%E5%B8%83%E5%B0%B1%E6%98%AF%E9%AB%98%E6%96%AF%E5%87%BD%E6%95%B0%20%24g(x)%20%3D%20%5Cfrac%7B1%7D%7B%5Csigma%5Csqrt%7B2%5Cpi%7D%7De%5E%7B-%5Cfrac%7B1%7D%7B2%7D(%5Cfrac%7Bx-%5Cmu%7D%7B%5Csigma%7D)%5E2%7D%24%0ARBF%E6%A0%B8%20Radial%20Basis%20Function%20Kernel%0A%E5%B0%86%E6%AF%8F%E4%B8%80%E4%B8%AA%E6%A0%B7%E6%9C%AC%E7%82%B9%E6%98%A0%E5%B0%84%E5%88%B0%E4%B8%80%E4%B8%AA%E6%97%A0%E7%A9%B7%E7%BB%B4%E7%9A%84%E7%89%B9%E5%BE%81%E7%A9%BA%E9%97%B4%0A**%E5%A4%9A%E9%A1%B9%E5%BC%8F%E7%89%B9%E5%BE%81**%E4%BE%9D%E9%9D%A0%E5%8D%87%E7%BB%B4%E6%98%AF%E7%9A%84%E5%8E%9F%E6%9C%AC%E7%BA%BF%E6%80%A7%E4%B8%8D%E5%8F%AF%E5%88%86%E7%9A%84%E6%95%B0%E6%8D%AE%E7%BA%BF%E6%80%A7%E5%8F%AF%E5%88%86%0A%60%60%60python%0Aimport%20numpy%20as%20np%0Aimport%20matplotlib.pyplot%20as%20plt%0A%0Afrom%20sklearn%20import%20datasets%0Airis%20%3D%20datasets.load_iris()%0AX%20%3D%20iris.data%0Ay%20%3D%20iris.target%0AX%20%3D%20X%5By%20%3C%202%2C%20%3A2%5D%0Ay%20%3D%20y%5By%20%3C%202%5D%0A%0Aplt.scatter(X%5By%20%3D%3D%200%2C%200%5D%2C%20X%5By%20%3D%3D%200%2C%201%5D%2C%20color%3D‘red‘)%0Aplt.scatter(X%5By%20%3D%3D%201%2C%200%5D%2C%20X%5By%20%3D%3D%201%2C%201%5D%2C%20color%3D‘blue‘)%0Aplt.show()%0A%60%60%60%0A!%5Bf4e129e8b4c2d4660a312572e135ab40.png%5D(en-resource%3A%2F%2Fdatabase%2F1262%3A0)%0A%60%60%60python%0Afrom%20sklearn.preprocessing%20import%20StandardScaler%0AstandardScaler%20%3D%20StandardScaler()%0AstandardScaler.fit(X)%0AX_standard%20%3D%20standardScaler.transform(X)%0Afrom%20sklearn.svm%20import%20LinearSVC%0A%23SVC%E4%BD%BF%E7%94%A8%E5%90%91%E9%87%8F%E5%81%9A%E5%88%86%E7%B1%BB%0Asvc%20%3D%20LinearSVC(C%3D1e9)%0Asvc.fit(X_standard%2C%20y)%0A%0Adef%20plot_decision_boundary(model%2C%20axis)%3A%0A%20%20%20%20x0%2Cx1%20%3D%20np.meshgrid(%0A%20%20%20%20%20%20%20%20np.linspace(axis%5B0%5D%2Caxis%5B1%5D%2Cint((axis%5B1%5D-axis%5B0%5D)*100)).reshape(-1%2C1)%2C%0A%20%20%20%20%20%20%20%20np.linspace(axis%5B2%5D%2Caxis%5B3%5D%2Cint((axis%5B3%5D-axis%5B2%5D)*100)).reshape(-1%2C1)%2C%0A%20%20%20%20)%0A%20%20%20%20X_new%20%3D%20np.c_%5Bx0.ravel()%2Cx1.ravel()%5D%0A%20%20%20%20y_predict%20%3D%20model.predict(X_new)%0A%20%20%20%20zz%20%3D%20y_predict.reshape(x0.shape)%0A%20%20%20%20from%20matplotlib.colors%20import%20ListedColormap%0A%20%20%20%20custom_camp%20%3D%20ListedColormap(%5B‘%23EF9A9A‘%2C‘%23FFF59D‘%2C‘%2390CAF9‘%5D)%0A%20%20%20%20plt.contourf(x0%2C%20x1%2C%20zz%2C%20linewith%3D5%2C%20camp%3Dcustom_camp)%0A%20%20%20%20w%20%3D%20model.coef_%5B0%5D%0A%20%20%20%20b%20%3D%20model.intercept_%5B0%5D%0A%20%20%20%20plot_x%20%3D%20np.linspace(axis%5B0%5D%2C%20axis%5B1%5D%2C%20200)%0A%20%20%20%20up_y%20%3D%20-w%5B0%5D%2Fw%5B1%5D*plot_x-b%2Fw%5B1%5D%2B1%2Fw%5B1%5D%0A%20%20%20%20down_y%20%3D%20-w%5B0%5D%2Fw%5B1%5D*plot_x-b%2Fw%5B1%5D-1%2Fw%5B1%5D%0A%20%20%20%20up_index%20%3D%20(up_y%3E%3Daxis%5B2%5D)%26(up_y%3C%3Daxis%5B3%5D)%0A%20%20%20%20down_index%20%3D%20(down_y%3E%3Daxis%5B2%5D)%26(down_y%3C%3Daxis%5B3%5D)%0A%20%20%20%20plt.plot(plot_x%5Bup_index%5D%2Cup_y%5Bup_index%5D%2Ccolor%3D‘black‘)%0A%20%20%20%20plt.plot(plot_x%5Bdown_index%5D%2Cdown_y%5Bdown_index%5D%2Ccol%0A%0Aplot_decision_boundary(svc%2C%20axis%3D%5B-3%2C3%2C-3%2C3%5D)%0Aplt.scatter(X_standard%5By%3D%3D0%2C0%5D%2CX_standard%5By%3D%3D0%2C1%5D)%0Aplt.scatter(X_standard%5By%3D%3D1%2C0%5D%2CX_standard%5By%3D%3D1%2C1%5D)%0Aplt.show()%0A%60%60%60%0A!%5B2331a8982ae49eb7cd14410f2d041088.png%5D(en-resource%3A%2F%2Fdatabase%2F1268%3A0)%0A%0A!%5B985b31786a9731b65da134eec7cc6c05.png%5D(en-resource%3A%2F%2Fdatabase%2F1264%3A0)%0A%60%60%60python%0Asvc2%20%3D%20LinearSVC(C%3D0.01)%0Asvc2.fit(X_standard%2C%20y)%0Aplot_decision_boundary(svc2%2C%20axis%3D%5B-3%2C3%2C-3%2C3%5D)%0Aplt.scatter(X_standard%5By%3D%3D0%2C0%5D%2CX_standard%5By%3D%3D0%2C1%5D)%0Aplt.scatter(X_standard%5By%3D%3D1%2C0%5D%2CX_standard%5By%3D%3D1%2C1%5D)%0Aplt.show()%0A%60%60%60%0A!%5Bf0fb80292a4c6f14fc917ff2d0f71e05.png%5D(en-resource%3A%2F%2Fdatabase%2F1266%3A0)%0A!%5Bbabef1fd16618f5aa99c7a510dfe5a32.png%5D(en-resource%3A%2F%2Fdatabase%2F1270%3A0)%0A%0A%60%60%60python%0A%23%E5%A4%9A%E9%A1%B9%E5%BC%8F%E5%A4%84%E7%90%86%E9%9D%9E%E7%BA%BF%E6%80%A7%E9%97%AE%E9%A2%98%0Aimport%20numpy%20as%20npimport%20matplotlib.pyplot%20as%20plt%0Afrom%20sklearn%20import%20datasets%0AX%2C%20y%20%3D%20datasets.make_moons()%0Aplt.scatter(X%5By%3D%3D0%2C0%5D%2CX%5By%3D%3D0%2C1%5D)plt.scatter(X%5By%3D%3D1%2C0%5D%2CX%5By%3D%3D1%2C1%5D)plt.show()%0A%0A%60%60%60%0A!%5B484ddeb4dcfa3f08a8e9d7c25431ad28.png%5D(en-resource%3A%2F%2Fdatabase%2F1272%3A0)%0A%0A%60%60%60python%0A%23%E6%9E%84%E5%BB%BA%E6%95%B0%E6%8D%AE%E9%9B%86%0AX%2Cy%20%3D%20datasets.make_moons(noise%3D0.15%2C%20random_state%3D666)%0Aplt.scatter(X%5By%3D%3D0%2C0%5D%2CX%5By%3D%3D0%2C1%5D)plt.scatter(X%5By%3D%3D1%2C0%5D%2CX%5By%3D%3D1%2C1%5D)plt.show()%0A%0A%60%60%60%0A!%5B513f4ea41345e2ca60b2305c16ed4a40.png%5D(en-resource%3A%2F%2Fdatabase%2F1274%3A0)%0A%0A%60%60%60python%0Afrom%20sklearn.preprocessing%20import%20PolynomialFeatures%2C%20StandardScaler%0Afrom%20sklearn.svm%20import%20LinearSVC%0Afrom%20sklearn.pipeline%20import%20Pipeline%0Adef%20PolynomialSVC(degree%2CC%3D1.0)%3A%0A%20%20%20%20return%20Pipeline(%5B%0A%20%20%20%20%20%20%20%20(%22poly%22%2C%20PolynomialFeatures(degree%3Ddegree))%2C%0A%20%20%20%20%20%20%20%20(%22std_scaler%22%2C%20StandardScaler())%2C%0A%20%20%20%20%20%20%20%20(%22linearSVC%22%2C%20LinearSVC(C%3DC))%2C%0A%20%20%20%20%5D)%0Apoly_svc%20%3D%20PolynomialSVC(degree%3D3)%0Apoly_svc.fit(X%2Cy)%0A%0Adef%20plot_decision_boundary(model%2C%20axis)%3A%0A%20%20%20%20x0%2Cx1%20%3D%20np.meshgrid(%0A%20%20%20%20%20%20%20%20np.linspace(axis%5B0%5D%2Caxis%5B1%5D%2Cint((axis%5B1%5D-axis%5B0%5D)*100)).reshape(-1%2C1)%2C%0A%20%20%20%20%20%20%20%20np.linspace(axis%5B2%5D%2Caxis%5B3%5D%2Cint((axis%5B3%5D-axis%5B2%5D)*100)).reshape(-1%2C1)%2C%0A%20%20%20%20)%0A%20%20%20%20X_new%20%3D%20np.c_%5Bx0.ravel()%2Cx1.ravel()%5D%0A%20%20%20%20y_predict%20%3D%20model.predict(X_new)%0A%20%20%20%20zz%20%3D%20y_predict.reshape(x0.shape)%0A%20%20%20%20from%20matplotlib.colors%20import%20ListedColormap%0A%20%20%20%20custom_camp%20%3D%20ListedColormap(%5B‘%23EF9A9A‘%2C‘%23FFF59D‘%2C‘%2390CAF9‘%5D)%0Aplt.contourf(x0%2C%20x1%2C%20zz%2C%20linewith%3D5%2C%20camp%3Dcustom_camp)%0Aplot_decision_boundary(poly_svc%2Caxis%3D%5B-1.5%2C2.5%2C-1.0%2C1.5%5D)%0Aplt.scatter(X%5By%3D%3D0%2C0%5D%2C%20X%5By%3D%3D0%2C1%5D)%0Aplt.scatter(X%5By%3D%3D1%2C0%5D%2C%20X%5By%3D%3D1%2C1%5D)%0Aplt.show()%0A%0A%60%60%60%0A!%5B3f85011b23e5c25bb832e83b7d02b7e0.png%5D(en-resource%3A%2F%2Fdatabase%2F1276%3A0)%0A%60%60%60python%0A%23%E5%A4%9A%E9%A1%B9%E5%BC%8F%E6%A0%B8%E5%87%BD%E6%95%B0%0Afrom%20sklearn.svm%20import%20SVC%0Adef%20PolynomialKernelSVC(degree%2C%20C%3D1.0)%3A%0A%20%20%20%20return%20Pipeline(%5B%0A%20%20%20%20%20%20%20%20(%22std_scaler%22%2C%20StandardScaler())%2C%0A%20%20%20%20%20%20%20%20(%22kernelSVC%22%2C%20SVC(kernel%3D‘poly‘%2Cdegree%3Ddegree%2CC%3DC))%2C%0A%20%20%20%20%5D)%0Apoly_kernel_svc%20%3D%20PolynomialKernelSVC(degree%3D3)%0Apoly_kernel_svc.fit(X%2Cy)%0Aplot_decision_boundary(poly_kernel_svc%2Caxis%3D%5B-1.5%2C2.5%2C-1.0%2C1.5%5D)%0Aplt.scatter(X%5By%3D%3D0%2C0%5D%2C%20X%5By%3D%3D0%2C1%5D)%0Aplt.scatter(X%5By%3D%3D1%2C0%5D%2C%20X%5By%3D%3D1%2C1%5D)%0Aplt.show()%0A%0A%60%60%60%0A!%5B8f3a1d36c569485a2434667ff571c129.png%5D(en-resource%3A%2F%2Fdatabase%2F1278%3A0)%0A



























以上是关于udacity机器学习-支持向量机(to be continued)的主要内容,如果未能解决你的问题,请参考以下文章

机器学习分类算法之支持向量机

机器学习100天(四十):040 线性支持向量机-公式推导

支持向量机的机器学习网格搜索

python大战机器学习——支持向量机

机器学习:支持向量机

机器学习:支持向量机SVM