ValueError: need at most 63 handles, got a sequence of length 65

Posted 2021-08-29 Data+Science+Insight

ValueError: need at most 63 handles, got a sequence of length 65

使用如下randomsearch程序：

# 设置n_jobs= -1就会发生错误

# 而使用n_jobs= 10 或者其他的具体数值则不会而且程序可以正常运行；

randomSearch = RandomizedSearchCV(cls,
                                      param_distributions=param_dict,
                                      cv=num_cv, scoring=grid_scoring,
                                      n_iter=numIters, refit=True, verbose=0,
                                      n_jobs= -1)

Some research suggested this might be a python issue in windows: https://bugs.python.org/issue26903. 

Solution would be to limit the number of workers to 61.

Sounds like you're running a lot of cores there! Set n_jobs=60 and this should work. See more details here https://github.com/psf/black/issues/564

完整错误日志：

Exception in thread Thread-8:
Traceback (most recent call last):
  File "D:\\anaconda\\lib\\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "D:\\anaconda\\lib\\site-packages\\joblib\\externals\\loky\\process_executor.py", line 555, in run
    result_item, is_broken, bpe = self.wait_result_broken_or_wakeup()
  File "D:\\anaconda\\lib\\site-packages\\joblib\\externals\\loky\\process_executor.py", line 609, in wait_result_broken_or_wakeup
    ready = wait(readers + worker_sentinels)
  File "D:\\anaconda\\lib\\multiprocessing\\connection.py", line 868, in wait
    ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout)
  File "D:\\anaconda\\lib\\multiprocessing\\connection.py", line 800, in _exhaustive_wait
    res = _winapi.WaitForMultipleObjects(L, False, timeout)
ValueError: need at most 63 handles, got a sequence of length 65

参考：Exception in thread QueueManagerThread - scikit-learn

参考：ValueError: need at most 63 handles, got a sequence of length 65 #564

参考：ValueError: need at most 63 handles, got a sequence of length 65