如何不让一个超平面影响多类线性核 SVM 中的决策？

Posted 2023-03-12

【中文标题】如何不让一个超平面影响多类线性核 SVM 中的决策？

【英文标题】：How not to let one hyperplane affect decision in multiclass linear kernal SVM?

【发布时间】：2019-10-02 06:52:50

【问题描述】：

我正在使用 scikit 中的 linearsvc 来处理 3 类数据集。使用 one-vs-rest（默认）策略，我得到了 3 个超平面权重向量，每个向量的大小为 number_of_features_in_dataset。现在，最终预测是基于所有 3 个超平面系数的组合，但我想要排除第二个超平面对最终决策做出任何贡献。

我搜索并发现内部多个超平面投票并进行最终分类，在平局的情况下，考虑与单个超平面的距离。

clf = LinearSVC()
clf.fit(x_train,y_train)
y_predict = clf.predict(x_test)
print(clf.coef_) # This prints 3xnos_of_features, where each row represents hyperplane weights
#I want to exclude say 2nd hyperplane from affecting decision made in in line 3

【问题讨论】：

您可以使用超平面距离进行预测，然后手动覆盖其中一个超平面。

@BenjaminBreton 我不确定他们如何在内部聚合距离，我猜他们使用 Platt Scaling 或类似的东西，但不确切。

【参考方案1】：

您可以手动为每个超平面添加一个偏差，以支持其中一个类：

from sklearn.svm import LinearSVC
from sklearn.preprocessing import LabelEncoder
import numpy as np

import warnings
warnings.filterwarnings(module='sklearn*', action='ignore', category=DeprecationWarning)


class BiasedSVC(LinearSVC):

    def __init__(self, penalty='l2', loss='squared_hinge', dual=True, tol=1e-4,
                 C=1.0, multi_class='ovr', fit_intercept=True,
                 intercept_scaling=1, class_weight=None, verbose=0,
                 random_state=None, max_iter=1000, classes=None, biases=None
                 ):
        """
        Same as LinearSVC: (https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/svm/classes.py)
        But allowing to bias the hyperplane to favor a class over another. Works for multiclass classification
        :param classes: list of the classes (all the classes myst be present in y during training).
        :type classes: list of strings
        :param biases: list of biases in the alphabetical order of the classes (ex: [0.0, +0.1, -0.1]) or dict
        containing the weights by class (ex: "class_1": 0.0, "class_2": +0.1, "class_3": -0.1)
        :type biases: list of floats or dict
        """

        super().__init__(penalty, loss, dual, tol, C, multi_class, fit_intercept, intercept_scaling, class_weight,
                         verbose, random_state, max_iter)

        # Define new variables
        self.classes = classes
        self.biases = biases

        # Transtype Biases
        self._biases = self.get_biases(self.biases)

        # Create Norm variable
        self._w_norm = None

        # Create LabelEncoder
        self._le = LabelEncoder()

    def get_biases(self, biases):
        """ Transtype the biases to get a list of floats """
        if isinstance(biases, list):
            return biases
        elif isinstance(biases, dict):
            return [biases[class_name] for class_name in self.classes]
        else:
            return [0.0 for _ in self.classes]

    def get_w_norm(self):
        """ Get the norm of the hyperplane to normalize the distance """
        self._w_norm = np.linalg.norm(self.coef_)

    def fit(self, X, y, sample_weight=None):

        # Fit the label Encoder (to change labels to indices)
        self._le.fit(y)

        # Fit the SVM using the mother class (LinearSVC) fit method
        super().fit(X, y, sample_weight)

        # Record the norm for all the hyperplanes (useful during inference)
        self.get_w_norm()

    def predict(self, X):
        """ Performa a prediction with the biased hyerplane """

        # Get the decision output (distance to the the hyperplanes separating the different classes)
        decision_y = self.decision_function(X)

        # Add the bias to each hyperplane (normalized for each class)
        dist = decision_y / self._w_norm + self._biases

        # Return the corresponding class
        return self._le.inverse_transform(np.argmax(dist, axis=1))

注意：您不要在训练期间使用偏差，仅在预测期间使用，因为 SVC 会翻译超平面以补偿您在训练期间的偏差。

【讨论】：

你直接训练 BiasedSVC(biases=[1.0,0.0,1.0])

它的抛出错误@Benjamin Breton ``` BiasedSVC(random_state=0, tol=1e-5, max_iter=max_iteration, verbose=1, biases=[1.0,0.0,1.0]) ```抛出的错误是 ``` TypeError: super() 需要至少 1 个参数（给定 0）```

@Benjamen Breton ，这看起来像是类实现中的一些小问题

@Newbee 与我合作：biased_svc = BiasedSVC(biases=[1.0,0.0,1.0]) 与 sklearn 版本 0.20.2跨度>

@Newbee 似乎是一个 python2 兼容性问题。我在python3上测试过