PicklingError: Can't pickle <function mean_root_squared_error_func at 0x7f2efa224e18>: it's no

Posted 2023-03-12

【中文标题】PicklingError: Can\'t pickle <function mean_root_squared_error_func at 0x7f2efa224e18>: it\'s not found as __main__.mean_root_squared_error_func

【英文标题】：PicklingError: Can't pickle <function mean_root_squared_error_func at 0x7f2efa224e18>: it's not found as __main__.mean_root_squared_error_funcPicklingError: Can't pickle <function mean_root_squared_error_func at 0x7f2efa224e18>: it's not found as __main__.mean_root_squared_error_func

【发布时间】：2020-08-18 17:09:02

【问题描述】：

我已经为 GridSearchCV 函数创建了一个自定义记分器：

def mean_root_squared_error_func(y_true, y_pred):

    return np.sqrt(mean_squared_error(y_true, y_pred))

这就是我在代码中调用函数的方式

scoring_grid='r_squared': "r2",
              'neg_mean_sqr_error': make_scorer(mean_root_squared_error_func, greater_is_better=False)
             

虽然在运行 GridSearchCV 函数并尝试腌制最佳估算器对象后，我得到了这个错误（使用 pickle 或 joblib 函数）：

with open(os.path.join(os.getcwd(),filename), 'wb') as file:
    joblib.dump(best_model_dictionary[list[0]], file)

PicklingError: Can't pickle <function mean_root_squared_error_func at 0x7f2efa224e18>: it's not found as __main__.mean_root_squared_error_func

我在 *** 上阅读了 pickle 文档和一些类似问题，尽管我没有设法处理此错误。

编辑 1 请注意，scikit-learn docs 中的“neg_root_mean_squared_error”不受支持。它返回以下错误：

ValueError: 'neg_root_mean_squared_error' is not a valid scoring value. Use sorted(sklearn.metrics.SCORERS.keys()) to get valid options.

编辑 2 在请求发布更多代码后： 返回 GridSearchCV 使用的 pipeline_object、parameters_grid、评分网格的函数

def regressor_component_initialization(model_name):
    """
    Create the components of:
        - Pipeline object
        - Hyperparameter grid,
        - Scorint metrics

    For the regression model specified
    """

    if model_name=="RandomForest":

        pipeline_object=Pipeline([
                    ('random_forest_regressor', RandomForestRegressor(random_state=123))
                    ])

        pipe_params=
                    'random_forest_regressor__max_depth':[20, None],
                    'random_forest_regressor__max_features':['log2', None]
                    

        scoring_grid='r_squared': "r2",
                      'neg_mean_sqr_error': make_scorer(mean_root_squared_error_func, greater_is_better=False)
                    

    elif model_name=="DecisionTree":

        pipeline_object=Pipeline([
                    ('decision_tree_regressor', DecisionTreeRegressor(random_state=123))
                    ])

        pipe_params=
                    'decision_tree_regressor__max_depth':[20, None],
                    'decision_tree_regressor__max_features':['log2', None]
                    

        scoring_grid='r_squared': "r2",
                      'neg_mean_sqr_error': make_scorer(mean_root_squared_error_func, greater_is_better=False)
                    

    return pipeline_object, pipe_params, scoring_grid

接下来，GridSearchCV的功能：

def hyperparameter_tuning(method, model_object, param_grid, cv, scoring, refit, n_iter=1000, split_ratiο=0.2):

    if method=="gridsearch":

        search_object=GridSearchCV(model_object, param_grid=param_grid, cv = cv, verbose=1, n_jobs = -1, scoring=scoring, refit=refit, return_train_score=True)
    return search_object, refit

然后将 search_object 拟合到 X_train, y_train

【问题讨论】：

发布完整代码。 PicklingError 错误确实很奇怪

@seralouk 我添加了更多代码，显示了 gridsearchCV 的调用方式

【参考方案1】：

Dill 是我用来序列化所有内容的工具。它应该能够处理奇怪的作用域函数和 lambda 函数。

import dill
with open(os.path.join(os.getcwd(),filename), 'wb') as file:
    dill.dump(best_model_dictionary[list[0]], file)

【讨论】：

感谢您的回答。我会试试 dill 模块，我会通知你结果。