如何使用多输出分类器实现网格搜索 cv？

Posted 2023-03-12

【中文标题】如何使用多输出分类器实现网格搜索 cv？

【英文标题】：How to implement Grid search cv with multi output classifier?

【发布时间】：2020-12-01 10:32:38

【问题描述】：

我正在处理一个数据集，必须做出两个预测，即 y 的 2 列，每列也是多类的。 因此，我正在使用 XGBoost 和 MultiOutput Classfier 并对其进行调整，我想使用 Grid Search CV。

xgb_clf = xgb.XGBClassifier(learning_rate=0.1,
                n_estimators=3000,
                max_depth=3,
                min_child_weight=1,
                subsample=0.8,
                colsample_bytree=0.8,
                objective='multi:softmax',
                nthread=4,
                num_class=9,
                seed=27
                )
model = MultiOutputClassifier(estimator=xgb_clf)
    param_test1 =  'estimator__max_depth':[3],'estimator__min_child_weight':[4]
gsearch1 = GridSearchCV(estimator =model, 
 param_grid = param_test1, scoring='roc_auc',n_jobs=4,iid=False, cv=5)
gsearch1.fit(X_train_split,y_train_split)
gsearch1.grid_scores_, gsearch1.best_params_, gsearch1.best_score_

但是当我这样做时，我得到一个错误

_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/joblib/externals/loky/process_executor.py", line 431, in _process_worker
    r = call_item()
  File "/usr/local/lib/python3.6/dist-packages/joblib/externals/loky/process_executor.py", line 285, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.6/dist-packages/joblib/_parallel_backends.py", line 595, in __call__
    return self.func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/joblib/parallel.py", line 253, in __call__
    for func, args, kwargs in self.items]
  File "/usr/local/lib/python3.6/dist-packages/joblib/parallel.py", line 253, in <listcomp>
    for func, args, kwargs in self.items]
  File "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_validation.py", line 544, in _fit_and_score
    test_scores = _score(estimator, X_test, y_test, scorer)
  File "/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_validation.py", line 591, in _score
    scores = scorer(estimator, X_test, y_test)
  File "/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_scorer.py", line 87, in __call__
    *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_scorer.py", line 300, in _score
    raise ValueError("0 format is not supported".format(y_type))
ValueError: multiclass-multioutput format is not supported
"""

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
<ipython-input-42-e53fdaaedf6b> in <module>()
      5 gsearch1 = GridSearchCV(estimator =model, 
      6  param_grid = param_test1, scoring='roc_auc',n_jobs=4,iid=False, cv=5)
----> 7 gsearch1.fit(X_train_split,y_train_split)
      8 gsearch1.grid_scores_, gsearch1.best_params_, gsearch1.best_score_

7 frames
/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_search.py in fit(self, X, y, groups, **fit_params)
    708                 return results
    709 
--> 710             self._run_search(evaluate_candidates)
    711 
    712         # For multi-metric evaluation, store the best_index_, best_params_ and

/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_search.py in _run_search(self, evaluate_candidates)
   1149     def _run_search(self, evaluate_candidates):
   1150         """Search all candidates in param_grid"""
-> 1151         evaluate_candidates(ParameterGrid(self.param_grid))
   1152 
   1153 

/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_search.py in evaluate_candidates(candidate_params)
    687                                for parameters, (train, test)
    688                                in product(candidate_params,
--> 689                                           cv.split(X, y, groups)))
    690 
    691                 if len(out) < 1:

/usr/local/lib/python3.6/dist-packages/joblib/parallel.py in __call__(self, iterable)
   1040 
   1041             with self._backend.retrieval_context():
-> 1042                 self.retrieve()
   1043             # Make sure that we get a last message telling us we are done
   1044             elapsed_time = time.time() - self._start_time

/usr/local/lib/python3.6/dist-packages/joblib/parallel.py in retrieve(self)
    919             try:
    920                 if getattr(self._backend, 'supports_timeout', False):
--> 921                     self._output.extend(job.get(timeout=self.timeout))
    922                 else:
    923                     self._output.extend(job.get())

/usr/local/lib/python3.6/dist-packages/joblib/_parallel_backends.py in wrap_future_result(future, timeout)
    540         AsyncResults.get from multiprocessing."""
    541         try:
--> 542             return future.result(timeout=timeout)
    543         except CfTimeoutError as e:
    544             raise TimeoutError from e

/usr/lib/python3.6/concurrent/futures/_base.py in result(self, timeout)
    430                 raise CancelledError()
    431             elif self._state == FINISHED:
--> 432                 return self.__get_result()
    433             else:
    434                 raise TimeoutError()

/usr/lib/python3.6/concurrent/futures/_base.py in __get_result(self)
    382     def __get_result(self):
    383         if self._exception:
--> 384             raise self._exception
    385         else:
    386             return self._result

ValueError: multiclass-multioutput format is not supported

我认为错误发生在我使用 roc_auc 作为我的评分方法但我不知道如何解决它。我应该使用其他评分方法吗？

【问题讨论】：

请编辑并为您的问题添加更多上下文：您正在使用哪种技术、平台和运行时环境，您正在解决什么问题。另外，请说明您要达到的目标。

【参考方案1】：

是的，你想的没错。问题在于 ROC AUC 分数对二元分类情况有效。相反，您可以使用所有班级的 ROC AUC 分数的平均值。

# from https://***.com/questions/39685740/calculate-sklearn-roc-auc-score-for-multi-class
from sklearn.metrics import roc_auc_score
import numpy as np

def roc_auc_score_multiclass(actual_class, pred_class, average = "macro"):

  #creating a set of all the unique classes using the actual class list
  unique_class = set(actual_class)
  roc_auc_dict = 
  for per_class in unique_class:
    #creating a list of all the classes except the current class 
    other_class = [x for x in unique_class if x != per_class]

    #marking the current class as 1 and all other classes as 0
    new_actual_class = [0 if x in other_class else 1 for x in actual_class]
    new_pred_class = [0 if x in other_class else 1 for x in pred_class]

    #using the sklearn metrics method to calculate the roc_auc_score
    roc_auc = roc_auc_score(new_actual_class, new_pred_class, average = average)
    roc_auc_dict[per_class] = roc_auc

  return np.mean([x for x in roc_auc_dict.values()])

使用此函数，您可以获得每个班级相对于所有其他班级的 ROC AUC 分数。然后，您可以取该值的平均值并将其用作得分手。您可能需要使用 make_scorer 函数 (https://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html) 将函数转换为记分器对象。

【讨论】：

我的问题不是多类。我的问题是我有多个输出，即我的模型预测了两列 y。所以，我需要评估它们。

是的，所以提供的函数可以满足您的需求，它接收 2 个数组并计算 ROC AUC 并计算所有 ROC AUC 的平均值/