An attempt has been made to start a new process before the current process has finished its bootstra
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了An attempt has been made to start a new process before the current process has finished its bootstra相关的知识，希望对你有一定的参考价值。
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\spawn.py", line 125, in _main
prepare(preparation_data)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\spawn.py", line 236, in prepare
_fixup_main_from_path(data[\'init_main_from_path\'])
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "E:\\Eprogramfiles\\Anaconda3\\lib\\runpy.py", line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File "E:\\Eprogramfiles\\Anaconda3\\lib\\runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "E:\\Eprogramfiles\\Anaconda3\\lib\\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "E:\\other\\lightvit\\bert_test_demo01.py", line 100, in <module>
for i, batch in enumerate(train):
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\torch\\utils\\data\\dataloader.py", line 359, in __iter__
return self._get_iterator()
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\torch\\utils\\data\\dataloader.py", line 305, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\torch\\utils\\data\\dataloader.py", line 918, in __init__
w.start()
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\process.py", line 121, in start
self._popen = self._Popen(self)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\context.py", line 327, in _Popen
return Popen(process_obj)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\popen_spawn_win32.py", line 45, in __init__
prep_data = spawn.get_preparation_data(process_obj._name)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\spawn.py", line 154, in get_preparation_data
_check_not_importing_main()
File "E:\\Eprogramfiles\\Anaconda3\\lib\\multiprocessing\\spawn.py", line 134, in _check_not_importing_main
raise RuntimeError(\'\'\'
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == \'__main__\':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
报错的原因 ：
linux系统中可以使用多个子进程加载数据，windows系统里是不可以的， num_workers的设置，设置为0或者采用默认为0的设置。
===========================================================
E:\\Eprogramfiles\\Anaconda3\\python.exe E:/other/lightvit/bert_test_demo01.py
\'HTTPSConnectionPool(host=\'huggingface.co\', port=443): Max retries exceeded with url: /bert-base-chinese/resolve/main/tokenizer_config.json (Caused by ProxyError(\'Cannot connect to proxy.\', NewConnectionError(\'<urllib3.connection.HTTPSConnection object at 0x0000024FA9894370>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it\')))\' thrown while requesting HEAD https://huggingface.co/bert-base-chinese/resolve/main/tokenizer_config.json
Traceback (most recent call last):
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\util\\connection.py", line 84, in create_connection
raise err
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\util\\connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\connectionpool.py", line 667, in urlopen
self._prepare_proxy(conn)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\connectionpool.py", line 932, in _prepare_proxy
conn.connect()
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\connection.py", line 309, in connect
conn = self._new_conn()
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\connection.py", line 171, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x0000024FA9894370>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\requests\\adapters.py", line 439, in send
resp = conn.urlopen(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\connectionpool.py", line 726, in urlopen
retries = retries.increment(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\urllib3\\util\\retry.py", line 446, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=\'huggingface.co\', port=443): Max retries exceeded with url: /bert-base-chinese/resolve/main/tokenizer_config.json (Caused by ProxyError(\'Cannot connect to proxy.\', NewConnectionError(\'<urllib3.connection.HTTPSConnection object at 0x0000024FA9894370>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it\')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:/other/lightvit/bert_test_demo01.py", line 72, in <module>
train = Data.DataLoader(dataset=MyDataset(sentences, labels), batch_size=batch_size, shuffle=True, num_workers=1)
File "E:/other/lightvit/bert_test_demo01.py", line 44, in __init__
self.tokenizer = AutoTokenizer.from_pretrained(model)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\transformers\\models\\auto\\tokenization_auto.py", line 643, in from_pretrained
tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\transformers\\models\\auto\\tokenization_auto.py", line 487, in get_tokenizer_config
resolved_config_file = cached_file(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\transformers\\utils\\hub.py", line 417, in cached_file
resolved_file = hf_hub_download(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\huggingface_hub\\utils\\_validators.py", line 120, in _inner_fn
return fn(*args, **kwargs)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\huggingface_hub\\file_download.py", line 1195, in hf_hub_download
metadata = get_hf_file_metadata(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\huggingface_hub\\utils\\_validators.py", line 120, in _inner_fn
return fn(*args, **kwargs)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\huggingface_hub\\file_download.py", line 1532, in get_hf_file_metadata
r = _request_wrapper(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\huggingface_hub\\file_download.py", line 407, in _request_wrapper
response = _request_wrapper(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\huggingface_hub\\file_download.py", line 442, in _request_wrapper
return http_backoff(
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\huggingface_hub\\utils\\_http.py", line 228, in http_backoff
raise err
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\huggingface_hub\\utils\\_http.py", line 212, in http_backoff
response = session.request(method=method, url=url, **kwargs)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\requests\\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\requests\\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "E:\\Eprogramfiles\\Anaconda3\\lib\\site-packages\\requests\\adapters.py", line 510, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host=\'huggingface.co\', port=443): Max retries exceeded with url: /bert-base-chinese/resolve/main/tokenizer_config.json (Caused by ProxyError(\'Cannot connect to proxy.\', NewConnectionError(\'<urllib3.connection.HTTPSConnection object at 0x0000024FA9894370>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it\')))
Process finished with exit code 1
解决方法：FQ
=============================================
=============================================
=============================================
An attempt to set a process's DebugPort or ExceptionPort was made
Problem Description
=================
We tried to use ADPlus to capture dump file. But the size of dump files are all under 20K.
We tried it many times.
Trouble Shoot
=================
I tried to use the “PsExec.exe –s –i –d cmd.exe” to initialize ADPLUS. No luck.
I tried to use WinDBG attach to the process, I failed with information below.
Detail Message is as below.
---------------------------
Could not attach to process 1272, NTSTATUS 0xC0000048
已试图设置进程的 DebugPort 或 ExceptionPort，但该进程中已存在端口，或试图设置文件的 CompletionPort，但文件中已设置端口，或已试图设置 ALPC 端口的相关完成端口，但该端口已设置。
Did more research, we found the root cause and solution.
We saw DebugDiag, and we asked customer to open that. We see the dialog below.
There it is! 1272 is our SharePoint w3wp.exe process.
Root Cause
========================
Debug Diag already attached to the process.
Debug Diag has rules, which can attach to target process. Even if the rule is completed, it won’t let go of the process.
Another word to say is the debug port is still occurpied by DebugDiag, so other debuggers such as WinDBG or CDB.exe cannot attach and write dump file.
Solution
========================
1. Clear the Rules in DebugDiag.
2. Kill the following processes in task manager.
· DbgSvc.exe
· Dbghost.exe
Problem Resolved.
Dump can now be successfully written.
Lesson Learned
========================
Be careful with DebugDiag. When its rules are finished, it won’t let go of the process.
