如何使用 XGboost 进行微调

Posted 2023-03-28

【中文标题】如何使用 XGboost 进行微调

【英文标题】：How to fine tune using XGboost

【发布时间】：2021-03-27 11:07:31

【问题描述】：

我正在尝试获取 Mercedes Greener Manufacturing 数据集的通用模型。所以，我正在尝试使用 XGBoost Regressor 来达到同样的效果。我使用了一个 1-100 的循环作为训练测试集的种子，以便获得更好的采样。我使用 PCA 将尺寸减小到 8 。

如何使用 xgboost 进行微调，以免得到过度拟合的模型？

X_train,X_test,y_train,y_test = train_test_split(X_pc,
                                                     y,
                                                     test_size=0.2,
                                                   random_state = i)
    model = XGBRegressor()
    model.fit(X_train,y_train)
    
    train=model.score(X_train,y_train)
    test=model.score(X_test,y_test)

输出

TEST: 0.28278595203767265 TRAIN: 0.9041892366322192 RS: 0
TEST: 0.3803514386218507 TRAIN: 0.9099759411069458 RS: 1
TEST: 0.3357132066270113 TRAIN: 0.9113739827130357 RS: 2
TEST: 0.3003256802391573 TRAIN: 0.901560899846001 RS: 3
TEST: 0.3769044561739856 TRAIN: 0.9034886060173257 RS: 4
TEST: 0.3449160536081909 TRAIN: 0.9092295020552124 RS: 5
TEST: 0.43083817087609166 TRAIN: 0.8957931397175393 RS: 6
TEST: 0.27375366705147564 TRAIN: 0.912349291318306 RS: 7
TEST: 0.39315883169376264 TRAIN: 0.9090768492254802 RS: 8
TEST: 0.38714220182913905 TRAIN: 0.9089864030990132 RS: 9
TEST: 0.37089065589124093 TRAIN: 0.9099379400411342 RS: 10
TEST: 0.3785854487827084 TRAIN: 0.9080405667805768 RS: 11
TEST: 0.29249852154319345 TRAIN: 0.9057747080596891 RS: 12
TEST: 0.34881642748048425 TRAIN: 0.9077565004654295 RS: 13

【参考方案1】：

random_state 参数用于确保拆分的可重复性，以便运行您的实验的其他人可以重新创建您的结果。

有多种方法可以有效地训练模型并减少过度拟合的机会。一种这样的策略是使用Cross Fold Validation 和Grid Search 来确定模型的最佳参数。这是您的模型的外观。

import xgboost as xgb
from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold


X_train,X_test,y_train,y_test = train_test_split(X_pc, y, test_size=0.2)
xgb_model = xgb.XGBRegressor()

# Define the parameter options for the Grid Search
paramters = 

# Define the GridSearch class, including cross-validation, using your XGBoost model
clf = GridSearchCV(xgb_model, parameters, cv=StratifiedKFold(n_folds=5, shuffle=True).split(X_train, y_train))

# Fit the model
clf.fit(X_train, y_train)

要检索得到的最佳估计器，例如在训练期间表现最佳的 XGBoost 版本，您可以执行以下操作：

best_estimator = clf.best_estimator_