XGBoost 提前停止给出 KeyError: 'best_msg'
Posted
技术标签:
【中文标题】XGBoost 提前停止给出 KeyError: \'best_msg\'【英文标题】:XGBoost early stopping gives KeyError: 'best_msg'XGBoost 提前停止给出 KeyError: 'best_msg' 【发布时间】:2018-09-13 22:29:10 【问题描述】:我正在尝试使用 XGBoost scikit 包装器在回归问题中提前停止。奇怪的是,提前停止eval_metric
(在我的情况下为rmse
)的计算在每个early stopping
轮次都失败。这很奇怪,因为没有early stopping
的eval_set
确实可以使用相同的估算器。
代码如下:
eval_train_indices=y.dropna()[:-n_splits].index
eval_test_indices=y.dropna()[-n_splits:].index
X_train, X_test=X.loc[eval_train_indices,:], X.loc[eval_test_indices,:]
y_train, y_test = y.loc[eval_train_indices], y.loc[eval_test_indices]
eval_set = [(X_train, y_train), (X_test, y_test)]
predictor=XGBRegressor(n_estimators = 50000, subsample=0.8, **params)
predictor.fit(X, y,
eval_metric=["rmse"],
eval_set=eval_set,
early_stopping_rounds=40,
verbose=True)
以及它产生的错误消息:
<ipython-input-65-358402bfa21c> in fit(self, T)
147 early_stopping_rounds=40,
148 verbose=True)
150
151 n_estimators=int(self.predictor.best_iteration*1.0)
/Users/Nicolas/anaconda2/lib/python2.7/site-packages/xgboost-0.7-py2.7.egg/xgboost/sklearn.pyc in fit(self, X, y, sample_weight, eval_set, eval_metric, early_stopping_rounds, verbose, xgb_model)
291 early_stopping_rounds=early_stopping_rounds,
292 evals_result=evals_result, obj=obj, feval=feval,
--> 293 verbose_eval=verbose, xgb_model=xgb_model)
294
295 if evals_result:
/Users/Nicolas/anaconda2/lib/python2.7/site-packages/xgboost-0.7-py2.7.egg/xgboost/training.pyc in train(params, dtrain, num_boost_round, evals, obj, feval, maximize, early_stopping_rounds, evals_result, verbose_eval, xgb_model, callbacks, learning_rates)
202 evals=evals,
203 obj=obj, feval=feval,
--> 204 xgb_model=xgb_model, callbacks=callbacks)
205
206
/Users/Nicolas/anaconda2/lib/python2.7/site-packages/xgboost-0.7-py2.7.egg/xgboost/training.pyc in _train_internal(params, dtrain, num_boost_round, evals, obj, feval, xgb_model, callbacks)
97 end_iteration=num_boost_round,
98 rank=rank,
---> 99 evaluation_result_list=evaluation_result_list))
100 except EarlyStopException:
101 break
/Users/Nicolas/anaconda2/lib/python2.7/site-packages/xgboost-0.7-py2.7.egg/xgboost/callback.pyc in callback(env)
245 best_msg=state['best_msg'])
246 elif env.iteration - best_iteration >= stopping_rounds:
--> 247 best_msg = state['best_msg']
248 if verbose and env.rank == 0:
249 msg = "Stopping. Best iteration:\n\n\n"
KeyError: 'best_msg'
由于某种原因,XGB 似乎无法在早期停止轮次中计算 RMSE,尽管在没有 early stopping
的 eval 火车和测试集上进行测试时它确实有效。当verbose=True
时,显示如下:
[0] validation_0-rmse:nan validation_1-rmse:nan
Multiple eval metrics have been passed: 'validation_1-rmse' will be used for early stopping.
Will train until validation_1-rmse hasn't improved in 40 rounds.
[1] validation_0-rmse:nan validation_1-rmse:nan
[2] validation_0-rmse:nan validation_1-rmse:nan
[3] validation_0-rmse:nan validation_1-rmse:nan
[4] validation_0-rmse:nan validation_1-rmse:nan
[5] validation_0-rmse:nan validation_1-rmse:nan
[6] validation_0-rmse:nan validation_1-rmse:nan
[7] validation_0-rmse:nan validation_1-rmse:nan
[8] validation_0-rmse:nan validation_1-rmse:nan
[9] validation_0-rmse:nan validation_1-rmse:nan
[10] validation_0-rmse:nan validation_1-rmse:nan
[11] validation_0-rmse:nan validation_1-rmse:nan
[12] validation_0-rmse:nan validation_1-rmse:nan
[13] validation_0-rmse:nan validation_1-rmse:nan
[14] validation_0-rmse:nan validation_1-rmse:nan
[15] validation_0-rmse:nan validation_1-rmse:nan
[16] validation_0-rmse:nan validation_1-rmse:nan
[17] validation_0-rmse:nan validation_1-rmse:nan
[18] validation_0-rmse:nan validation_1-rmse:nan
[19] validation_0-rmse:nan validation_1-rmse:nan
[20] validation_0-rmse:nan validation_1-rmse:nan
[21] validation_0-rmse:nan validation_1-rmse:nan
[22] validation_0-rmse:nan validation_1-rmse:nan
[23] validation_0-rmse:nan validation_1-rmse:nan
[24] validation_0-rmse:nan validation_1-rmse:nan
[25] validation_0-rmse:nan validation_1-rmse:nan
[26] validation_0-rmse:nan validation_1-rmse:nan
[27] validation_0-rmse:nan validation_1-rmse:nan
[28] validation_0-rmse:nan validation_1-rmse:nan
[29] validation_0-rmse:nan validation_1-rmse:nan
[30] validation_0-rmse:nan validation_1-rmse:nan
[31] validation_0-rmse:nan validation_1-rmse:nan
[32] validation_0-rmse:nan validation_1-rmse:nan
[33] validation_0-rmse:nan validation_1-rmse:nan
[34] validation_0-rmse:nan validation_1-rmse:nan
[35] validation_0-rmse:nan validation_1-rmse:nan
[36] validation_0-rmse:nan validation_1-rmse:nan
[37] validation_0-rmse:nan validation_1-rmse:nan
[38] validation_0-rmse:nan validation_1-rmse:nan
[39] validation_0-rmse:nan validation_1-rmse:nan
[40] validation_0-rmse:nan validation_1-rmse:nan
我什至不明白是什么导致计算 RMSE 失败。这可能是由于缺少值,但是当我打印predictor.predict(X_test)
时没有值
【问题讨论】:
该错误是由 X 中的缺失值引起的。XGB 可以处理 model.fit(X, y) 中的缺失值,但不能在早期停止轮次中处理,原因不明。 【参考方案1】:这是由于 Nan 值;尝试删除或替换它们并检查它是否有效。
【讨论】:
【参考方案2】:我只有在升级到 xgboost=0.80 以使用 SHAP 模块后才会遇到此问题。 xgboost=0.6a1 的早期版本运行良好。
【讨论】:
以上是关于XGBoost 提前停止给出 KeyError: 'best_msg'的主要内容,如果未能解决你的问题,请参考以下文章
在 SciKit-Learn 中使用 XGBoost 交叉验证进行网格搜索和提前停止