[Ray.Tune] [已解决] Queue objects should only be shared between processes through inheritance

Posted 2022-12-02 m0_38052500

问题错误代码

2021-12-24 17:48:01,170	WARNING experiment.py:256 -- No name detected on trainable. Using DEFAULT.
2021-12-24 17:48:01,171	INFO registry.py:70 -- Detected unknown callable for trainable. Converting to class.
Traceback (most recent call last):
  File "/home/bullet/PycharmProjects/LowlightRecognition/STTR/code/main_downsampled.py", line 773, in <module>
    local_dir="/home/bullet/PycharmProjects/LowlightRecognition/coldnight/n3_storage/ray_results",
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/site-packages/ray/tune/tune.py", line 444, in run
    restore=restore)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/site-packages/ray/tune/experiment.py", line 113, in __init__
    self._run_identifier = Experiment.register_if_needed(run)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/site-packages/ray/tune/experiment.py", line 258, in register_if_needed
    register_trainable(name, run_object)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/site-packages/ray/tune/registry.py", line 76, in register_trainable
    _global_registry.register(TRAINABLE_CLASS, name, trainable)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/site-packages/ray/tune/registry.py", line 150, in register
    self._to_flush[(category, key)] = pickle.dumps_debug(value)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/site-packages/ray/cloudpickle/__init__.py", line 39, in dumps_debug
    return dumps(obj, *args, **kwargs)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
    cp.dump(obj)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 620, in dump
    return Pickler.dump(self, obj)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/multiprocessing/queues.py", line 58, in __getstate__
    context.assert_spawning(self)
  File "/home/bullet/anaconda3/envs/yolox/lib/python3.7/multiprocessing/context.py", line 356, in assert_spawning
    ' through inheritance' % type(obj).__name__
RuntimeError: Queue objects should only be shared between processes through inheritance

解决方法

梳理trainable函数，保证trainable中的对象要么通过trainable传参得到，要么自己内部定义——这是wrapper的要义。
而对于我的问题是因为tensorflow的writer对象被定义在了trainable之外，因此出现了Queue objects should only be shared between processes through inheritance(序列中的对象只能通过继承产生，而不能从外获取。)

原代码的抽象框架：

writer = SummaryWriter('./' + name_exp)  # 问题代码的位置

class Processor():
	def __init__(arg):
		 self.writer = SummaryWriter('./' + name_exp)  # 问题代码更换后的位置。
	def start():
		# 使用writer的语句。

def trainable(arg,config,...):
	process = Processor(arg)
	process.start()

if __name__ == '__main__':
	analysis = tune.run(...)