如何在多标签设置中最大化召回率?
Posted
技术标签:
【中文标题】如何在多标签设置中最大化召回率?【英文标题】:How to maximize recall in multilabel setting? 【发布时间】:2018-08-27 09:22:09 【问题描述】:我有一个文本分类问题,我想将三个标签(-1、0、1)之一分配给文本文档。最重要的指标是召回率:我关心所有应该标记为“-1”的文本确实都标记为“-1”。标记为“-1”的所有内容确实标记为“-1”的精度不太重要。
到目前为止,我在 scikit-learn 中使用了一个带有逻辑回归的管道。超参数在 GridSearchCV 中进行了调整,但到目前为止,准确性已最大化。
steps = [('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', LogisticRegression())]
parameters = 'vect__ngram_range': [(1, 1), (1, 2), (1, 3), (1, 4)],
'tfidf__use_idf': (True, False),
'clf__C': [0.001, 0.01, 0.1, 1, 10],
pipeline = Pipeline(steps)
text_clf = GridSearchCV(pipeline, parameters, cv = 5)
text_clf.fit(X_train, y_train)
y_pred = text_clf.predict(X_test)
scores = cross_val_score(text_clf, X_test, y_test, cv = 5)
变化
text_clf = GridSearchCV(pipeline, parameters, scoring = 'recall', cv = 5)
不起作用,因为它是一个多类设置。有人知道我如何重新制定这个以最大限度地提高召回率吗?
【问题讨论】:
这个问题是 multiclass 问题,而不是 multilabel 问题。多标签是每个示例可能有多个与之相关的类的情况。 【参考方案1】:如果度量标准仅提供一个数字作为 GridSearchCV 将用于对结果进行排序的分数,则网格搜索可以工作。
在多标签设置的情况下,您需要决定要为不同的标签使用哪种类型的平均。您可以使用以下替代方法:
scoring = 'recall_micro'
scoring = 'recall_macro'
scoring = 'recall_weighted'
scoring = 'recall_samples'
关于这些的描述,请参考documentation of recall_score:
average : string, [None, ‘binary’ (default), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data: 'binary': Only report results for the class specified by pos_label. This is applicable only if targets (y_true,pred) are binary. 'micro': Calculate metrics globally by counting the total true positives, false negatives and false positives. 'macro': Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account. 'weighted': Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall. 'samples': Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
【讨论】:
以上是关于如何在多标签设置中最大化召回率?的主要内容,如果未能解决你的问题,请参考以下文章
R语言使用yardstick包的pr_curve函数评估多分类(Multiclass)模型的性能查看模型在多分类每个分类上的ROC曲线(precision(精准率),R代表的是recall(召回率)
R语言使用yardstick包的pr_curve函数评估多分类(Multiclass)模型的性能查看模型在多分类每个分类上的ROC曲线(precision(精准率),R代表的是recall(召回率)
Spark ML - MulticlassClassificationEvaluator - 我们可以通过每个类标签获得精度/召回率吗?