获取 PermissionError：[WinError 32] 进程无法访问该文件，因为它在 scikitlearn GridSearchCV 期间正被另一个进程使用

Posted 2023-03-12

【中文标题】获取 PermissionError：[WinError 32] 进程无法访问该文件，因为它在 scikitlearn GridSearchCV 期间正被另一个进程使用

【英文标题】：Getting PermissionError: [WinError 32] The process cannot access the file because it is being used by another process during scikitlearn GridSearchCV

【发布时间】：2020-03-21 07:02:00

【问题描述】：

在GridSearchCV 的scikit-learn 功能期间获取PermissionError: [WinError 32] The process cannot access the file because it is being used by another process。

这是我的脚本：

df = pd.read_csv(CSV_FILE_PATH, sep=',', header=0)

df_train, df_test = train_test_split(df, test_size=0.2, random_state=random_state)
X_train, y_train = df_train.iloc[:, 1:-1], df_train.iloc[:, -1]
X_test, y_test = df_test.iloc[:, 1:-1], df_test.iloc[:, -1]

X_train = np.reshape(X_train.values, (X_train.shape[0], 1, X_train.shape[1]))
X_test = np.reshape(X_test.values, (X_test.shape[0], 1, X_test.shape[1]))

model = KerasClassifier(build_fn=create_model_optimizer, epochs=epochs, batch_size=batch_size, verbose=0)

param_grid = dict(optimizer=['adadelta', 'sgd', 'adam'])

grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, cv=3)
grid_result = grid.fit(X_test, to_categorical(y_test))
print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))
means = grid_result.cv_results_['mean_test_score']
stds = grid_result.cv_results_['std_test_score']
params = grid_result.cv_results_['params']
for mean, stdev, param in zip(means, stds, params):
    print("%f (%f) with: %r" % (mean, stdev, param))

错误堆栈跟踪：

D:\.virtualenvs\signal_processing_keras-0Sqytsnt\lib\site-packages\joblib\disk.py:122: UserWarning: Unable to delete folder C:\Users\talha\AppData\Local\Temp\joblib_memmapping_folder_9640_7658187734 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))
Traceback (most recent call last):
  File "D:\.virtualenvs\signal_processing_keras-0Sqytsnt\lib\site-packages\joblib\disk.py", line 115, in delete_folder
    shutil.rmtree(folder_path, False, None)
  File "D:\.virtualenvs\signal_processing_keras-0Sqytsnt\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "D:\.virtualenvs\signal_processing_keras-0Sqytsnt\lib\shutil.py", line 400, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "D:\.virtualenvs\signal_processing_keras-0Sqytsnt\lib\shutil.py", line 398, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\talha\\AppData\\Local\\Temp\\joblib_memmapping_folder_9640_7658187734\\9640-1516123463368-0106d247f5d8461b9e06e246788ccb51.pkl'
D:\.virtualenvs\signal_processing_keras-0Sqytsnt\lib\site-packages\joblib\_memmapping_reducer.py:409: UserWarning: Failed to clean temporary folder: C:\Users\talha\AppData\Local\Temp\joblib_memmapping_folder_9640_7658187734
  .format(pool_folder))

软件栈：

System:
    python: 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
executable: D:\.virtualenvs\signal_processing_keras-0Sqytsnt\Scripts\python.exe
   machine: Windows-10-10.0.18362-SP0

Python deps:
       pip: 19.3.1
setuptools: 41.6.0
   sklearn: 0.21.3
     numpy: 1.17.4
     scipy: 1.3.2
    Cython: None
    pandas: 0.25.3
Windows-10-10.0.18362-SP0
Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
NumPy 1.17.4
SciPy 1.3.2
Scikit-Learn 0.21.3

【问题讨论】：

可能有用：github.com/scikit-learn/scikit-learn/issues/12546

感谢您的反馈，已经查看了那里发布的 cmets。 @BramVanroy

【参考方案1】：

我在尝试使用 scikit-learn 执行逻辑回归时遇到了类似的问题。更新包“joblib”和“scikit-learn”为我修复了它：

pip install --upgrade joblib scikit-learn