连接被对方拒绝:10061:无法建立连接,因为目标机器主动拒绝它
Posted
技术标签:
【中文标题】连接被对方拒绝:10061:无法建立连接,因为目标机器主动拒绝它【英文标题】:Connection was refused by other side: 10061: No connection could be made because the target machine actively refused it 【发布时间】:2021-11-10 20:14:13 【问题描述】:我的步骤:
-
构建镜像
docker build . -t scrapy
运行容器docker run -it -p 8050:8050 --rm scrapy
在容器中运行scrapy项目:scrapy crawl foobar -o allobjects.json
这在本地工作,但在我的生产服务器上出现错误:
[scrapy.downloadermiddlewares.retry] 调试:重试
https://www.example.com via http://localhost:8050/execute>(失败 1 次):连接被对方拒绝:10061:无法建立连接,因为目标机器主动拒绝..
注意:我没有使用 Docker Desktop,我也不能在这台服务器上使用。
Dockerfile
FROM mcr.microsoft.com/windows/servercore:ltsc2019
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
RUN setx /M PATH $('C:\Users\ContainerAdministrator\miniconda3\Library\bin;C:\Users\ContainerAdministrator\miniconda3\Scripts;C:\Users\ContainerAdministrator\miniconda3;' + $Env:PATH)
RUN Invoke-WebRequest "https://repo.anaconda.com/miniconda/Miniconda3-py38_4.10.3-Windows-x86_64.exe" -OutFile miniconda3.exe -UseBasicParsing; \
Start-Process -FilePath 'miniconda3.exe' -Wait -ArgumentList '/S', '/D=C:\Users\ContainerAdministrator\miniconda3'; \
Remove-Item .\miniconda3.exe; \
conda install -y -c conda-forge scrapy;
RUN pip install scrapy-splash
RUN pip install scrapy-user-agents
#creates root directory if not exists, then enters it
WORKDIR /root/scrapy
COPY scrapy /root/scrapy
settings.py
SPLASH_URL = 'http://localhost:8050/'
输出与命令scrapy crawl foobar -o allobjects.json
2021-09-15 20:12:16 [scrapy.core.engine] INFO: Spider opened
2021-09-15 20:12:16 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min
)
2021-09-15 20:12:16 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2021-09-15 20:12:16 [py.warnings] WARNING: C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\scrapy_splash\re
quest.py:41: ScrapyDeprecationWarning: Call to deprecated function to_native_str. Use to_unicode instead.
url = to_native_str(url)
2021-09-15 20:12:16 [scrapy_user_agents.middlewares] DEBUG: Assigned User-Agent Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (Khtml, like Gecko) Chrome/66.0.3359.117 Safari/537.36
2021-09-15 20:12:16 [scrapy_user_agents.middlewares] DEBUG: Assigned User-Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36
2021-09-15 20:12:17 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.example.com via http://localhost:8050/execute> (failed 1 times): Connection was refused by other side: 10061: No connection could be made because the target machine actively refused it..
2021-09-15 20:12:17 [scrapy_user_agents.middlewares] DEBUG: Assigned User-Agent Mozilla/5.0 (Windows NT 10.0; WOW64) App
leWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36
2021-09-15 20:12:18 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.example.com via http://localhost:8050/execute> (failed 2 times): Connection was refused by other side: 10061: No connection
could be made because the target machine actively refused it..
2021-09-15 20:12:18 [scrapy_user_agents.middlewares] DEBUG: Assigned User-Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64
) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36
2021-09-15 20:12:19 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying <GET https://www.example.com via http://localhost:8050/execute> (failed 3 times): Connection was refused by other side: 10061: No con
nection could be made because the target machine actively refused it..
2021-09-15 20:12:19 [scrapy.core.scraper] ERROR: Error downloading <GET https://www.example.com via http://localhost:8050/execute>
Traceback (most recent call last):
File "C:\Users\ContainerAdministrator\miniconda3\lib\site-packages\scrapy\core\downloader\middleware.py", line 45, in
process_request
return (yield download_func(request=request, spider=spider))
twisted.internet.error.ConnectionRefusedError: Connection was refused by other side: 10061: No connection could be made
because the target machine actively refused it..
2021-09-15 20:12:19 [scrapy.core.engine] INFO: Closing spider (finished)
2021-09-15 20:12:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
'downloader/exception_count': 3,
'downloader/exception_type_count/twisted.internet.error.ConnectionRefusedError': 3,
'downloader/request_bytes': 4632,
'downloader/request_count': 3,
'downloader/request_method_count/POST': 3,
'elapsed_time_seconds': 3.310168,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2021, 9, 15, 18, 12, 19, 605641),
'log_count/DEBUG': 6,
'log_count/ERROR': 2,
'log_count/INFO': 10,
'log_count/WARNING': 46,
'retry/count': 2,
'retry/max_reached': 1,
'retry/reason_count/twisted.internet.error.ConnectionRefusedError': 2,
'scheduler/dequeued': 4,
'scheduler/dequeued/memory': 4,
'scheduler/enqueued': 4,
'scheduler/enqueued/memory': 4,
'splash/execute/request_count': 1,
'start_time': datetime.datetime(2021, 9, 15, 18, 12, 16, 295473)
2021-09-15 20:12:19 [scrapy.core.engine] INFO: Spider closed (finished)
我错过了什么?
我已经在这里检查过:
Scrapy, Splash and Connection was refused by other side: 10061 Scrapy + Splash = Connection Refused How to run splash using docker toolbox更新 1
我在我的 Dockerfile 中包含了EXPOSE 8050
,但得到了同样的错误。我在 docker 容器中尝试了netstat -a
,但里面好像没有 8050?
C:\root\scrapy>netstat -a
Active Connections
Proto Local Address Foreign Address State
TCP 0.0.0.0:135 c60d48724046:0 LISTENING
TCP 0.0.0.0:5985 c60d48724046:0 LISTENING
TCP 0.0.0.0:47001 c60d48724046:0 LISTENING
TCP 0.0.0.0:49152 c60d48724046:0 LISTENING
TCP 0.0.0.0:49153 c60d48724046:0 LISTENING
TCP 0.0.0.0:49154 c60d48724046:0 LISTENING
TCP 0.0.0.0:49155 c60d48724046:0 LISTENING
TCP 0.0.0.0:49159 c60d48724046:0 LISTENING
TCP [::]:135 c60d48724046:0 LISTENING
TCP [::]:5985 c60d48724046:0 LISTENING
TCP [::]:47001 c60d48724046:0 LISTENING
TCP [::]:49152 c60d48724046:0 LISTENING
TCP [::]:49153 c60d48724046:0 LISTENING
TCP [::]:49154 c60d48724046:0 LISTENING
TCP [::]:49155 c60d48724046:0 LISTENING
TCP [::]:49159 c60d48724046:0 LISTENING
UDP 0.0.0.0:5353 *:*
UDP 0.0.0.0:5355 *:*
UDP 127.0.0.1:51352 *:*
UDP [::]:5353 *:*
UDP [::]:5355 *:*
更新 2
我在主机操作系统上运行的命令:
docker ps
输出:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
bf615a00b74a scrapy "c:\\windows\\system32…" 52 seconds ago Up 49 seconds 0.0.0.0:8050->8050/tcp blissful_brahmagupta
netstat -a
输出(为了匿名,我更改了 ip/服务器名称):
Active Connections
Proto Local Address Foreign Address State
TCP 0.0.0.0:21 exampleserver:0 LISTENING
TCP 0.0.0.0:25 exampleserver:0 LISTENING
TCP 0.0.0.0:80 exampleserver:0 LISTENING
TCP 0.0.0.0:110 exampleserver:0 LISTENING
TCP 0.0.0.0:135 exampleserver:0 LISTENING
TCP 0.0.0.0:143 exampleserver:0 LISTENING
TCP 0.0.0.0:443 exampleserver:0 LISTENING
TCP 0.0.0.0:445 exampleserver:0 LISTENING
TCP 0.0.0.0:587 exampleserver:0 LISTENING
TCP 0.0.0.0:995 exampleserver:0 LISTENING
TCP 0.0.0.0:1433 exampleserver:0 LISTENING
TCP 0.0.0.0:2179 exampleserver:0 LISTENING
TCP 0.0.0.0:3306 exampleserver:0 LISTENING
TCP 0.0.0.0:3389 exampleserver:0 LISTENING
TCP 0.0.0.0:5985 exampleserver:0 LISTENING
TCP 0.0.0.0:8983 exampleserver:0 LISTENING
TCP 0.0.0.0:33060 exampleserver:0 LISTENING
TCP 0.0.0.0:47001 exampleserver:0 LISTENING
TCP 0.0.0.0:49231 exampleserver:0 LISTENING
TCP 0.0.0.0:49664 exampleserver:0 LISTENING
TCP 0.0.0.0:49665 exampleserver:0 LISTENING
TCP 0.0.0.0:49666 exampleserver:0 LISTENING
TCP 0.0.0.0:49667 exampleserver:0 LISTENING
TCP 0.0.0.0:49668 exampleserver:0 LISTENING
TCP 0.0.0.0:49673 exampleserver:0 LISTENING
TCP 0.0.0.0:49881 exampleserver:0 LISTENING
TCP 12.12.12.12:21 103.144.31.100:ftp SYN_RECEIVED
TCP 12.12.12.12:25 ip245:1256 TIME_WAIT
TCP 12.12.12.12:25 ip245:12756 TIME_WAIT
TCP 12.12.12.12:25 ip245:25324 TIME_WAIT
TCP 12.12.12.12:25 ip245:30624 TIME_WAIT
TCP 12.12.12.12:25 ip245:48206 TIME_WAIT
TCP 12.12.12.12:25 ip245:59510 TIME_WAIT
TCP 12.12.12.12:80 ec2-52-31-126-154:1440 ESTABLISHED
TCP 12.12.12.12:80 ec2-52-31-157-215:31240 ESTABLISHED
TCP 12.12.12.12:80 ec2-52-31-205-57:65197 ESTABLISHED
TCP 12.12.12.12:80 ninja-crawler92:36060 ESTABLISHED
TCP 12.12.12.12:80 13:62786 TIME_WAIT
TCP 12.12.12.12:80 16:22362 TIME_WAIT
TCP 12.12.12.12:80 19:4130 TIME_WAIT
TCP 12.12.12.12:80 22:30072 TIME_WAIT
TCP 12.12.12.12:80 22:51362 TIME_WAIT
TCP 12.12.12.12:80 34:9586 TIME_WAIT
TCP 12.12.12.12:80 35:40210 TIME_WAIT
TCP 12.12.12.12:80 35:65164 TIME_WAIT
TCP 12.12.12.12:80 38:17882 TIME_WAIT
TCP 12.12.12.12:80 39:17918 TIME_WAIT
TCP 12.12.12.12:80 40:51642 TIME_WAIT
TCP 12.12.12.12:80 40:57586 TIME_WAIT
TCP 12.12.12.12:80 45:45800 TIME_WAIT
TCP 12.12.12.12:139 exampleserver:0 LISTENING
TCP 12.12.12.12:443 static:3610 TIME_WAIT
TCP 12.12.12.12:443 static:5823 TIME_WAIT
TCP 12.12.12.12:443 static:38855 TIME_WAIT
TCP 12.12.12.12:443 static:53579 TIME_WAIT
TCP 12.12.12.12:443 static:54816 TIME_WAIT
TCP 12.12.12.12:443 static:26725 TIME_WAIT
TCP 12.12.12.12:443 static:14749 TIME_WAIT
TCP 12.12.12.12:443 static:8533 TIME_WAIT
TCP 12.12.12.12:443 static:9136 TIME_WAIT
TCP 12.12.12.12:443 static:35494 TIME_WAIT
TCP 12.12.12.12:443 193:48688 TIME_WAIT
TCP 12.12.12.12:443 static:3161 TIME_WAIT
TCP 12.12.12.12:443 static:31667 TIME_WAIT
TCP 12.12.12.12:443 ec2-52-31-126-154:25042 ESTABLISHED
TCP 12.12.12.12:443 ec2-52-31-157-215:61630 ESTABLISHED
TCP 12.12.12.12:443 ec2-52-31-205-57:20864 ESTABLISHED
TCP 12.12.12.12:443 crawl-66-249-76-28:46983 ESTABLISHED
TCP 12.12.12.12:443 crawl-66-249-76-30:47250 ESTABLISHED
TCP 12.12.12.12:443 crawl-66-249-76-102:45115 ESTABLISHED
TCP 12.12.12.12:443 crawl-66-249-76-104:62362 ESTABLISHED
TCP 12.12.12.12:443 crawl-66-249-76-106:52575 ESTABLISHED
TCP 12.12.12.12:443 crawl-66-249-76-192:51273 ESTABLISHED
TCP 12.12.12.12:443 google-proxy-66-249-81-16:37717 ESTABLISHED
TCP 12.12.12.12:443 rate-limited-proxy-66-249-89-97:42078 ESTABLISHED
TCP 12.12.12.12:443 77-162-6-126:60721 ESTABLISHED
TCP 12.12.12.12:443 77-162-6-126:60728 ESTABLISHED
TCP 12.12.12.12:443 81-207-120-215:53600 ESTABLISHED
TCP 12.12.12.12:443 ip-83-134-52-36:51127 ESTABLISHED
TCP 12.12.12.12:443 host-83-232-56-99:2747 ESTABLISHED
TCP 12.12.12.12:443 84-29-102-40:57144 ESTABLISHED
TCP 12.12.12.12:443 84-104-10-105:57252 ESTABLISHED
TCP 12.12.12.12:443 exampleserver:54209 ESTABLISHED
TCP 12.12.12.12:443 static:37222 TIME_WAIT
TCP 12.12.12.12:443 static:net-device TIME_WAIT
TCP 12.12.12.12:443 static:7874 TIME_WAIT
TCP 12.12.12.12:443 static:33373 TIME_WAIT
TCP 12.12.12.12:443 static:60446 TIME_WAIT
TCP 12.12.12.12:443 92-111-50-210:54795 ESTABLISHED
TCP 12.12.12.12:443 static:2841 TIME_WAIT
TCP 12.12.12.12:443 ip-95-223-56-232:51129 ESTABLISHED
TCP 12.12.12.12:443 petalbot-114-119-135-120:32530 TIME_WAIT
TCP 12.12.12.12:443 petalbot-114-119-148-37:39746 TIME_WAIT
TCP 12.12.12.12:443 petalbot-114-119-148-47:39066 TIME_WAIT
TCP 12.12.12.12:443 petalbot-114-119-148-60:51178 SYN_RECEIVED
TCP 12.12.12.12:443 petalbot-114-119-148-160:11516 TIME_WAIT
TCP 12.12.12.12:443 petalbot-114-119-148-169:52484 TIME_WAIT
TCP 12.12.12.12:443 petalbot-114-119-148-191:41470 TIME_WAIT
TCP 12.12.12.12:443 petalbot-114-119-149-1:64570 TIME_WAIT
TCP 12.12.12.12:443 petalbot-114-119-149-168:1456 TIME_WAIT
TCP 12.12.12.12:443 petalbot-114-119-149-169:61436 TIME_WAIT
TCP 12.12.12.12:443 static:47402 TIME_WAIT
TCP 12.12.12.12:443 static:7710 TIME_WAIT
TCP 12.12.12.12:443 static:15334 TIME_WAIT
TCP 12.12.12.12:443 static:50492 TIME_WAIT
TCP 12.12.12.12:443 static:3896 TIME_WAIT
TCP 12.12.12.12:443 static:32136 TIME_WAIT
TCP 12.12.12.12:443 ninja-crawler97:19950 ESTABLISHED
TCP 12.12.12.12:443 static:9737 TIME_WAIT
TCP 12.12.12.12:443 1:14850 TIME_WAIT
TCP 12.12.12.12:443 2:9212 TIME_WAIT
TCP 12.12.12.12:443 2:38644 TIME_WAIT
TCP 12.12.12.12:443 2:40354 TIME_WAIT
TCP 12.12.12.12:443 2:61144 TIME_WAIT
TCP 12.12.12.12:443 9:4920 TIME_WAIT
TCP 12.12.12.12:443 9:10744 TIME_WAIT
TCP 12.12.12.12:443 9:41246 TIME_WAIT
TCP 12.12.12.12:443 10:55160 TIME_WAIT
TCP 12.12.12.12:443 12:28250 TIME_WAIT
TCP 12.12.12.12:443 12:48182 TIME_WAIT
TCP 12.12.12.12:443 13:6848 TIME_WAIT
TCP 12.12.12.12:443 13:41174 TIME_WAIT
TCP 12.12.12.12:443 14:11724 TIME_WAIT
TCP 12.12.12.12:443 14:23780 TIME_WAIT
TCP 12.12.12.12:443 14:35272 TIME_WAIT
TCP 12.12.12.12:443 14:42876 TIME_WAIT
TCP 12.12.12.12:443 15:50642 TIME_WAIT
TCP 12.12.12.12:443 16:11382 TIME_WAIT
TCP 12.12.12.12:443 16:43780 TIME_WAIT
TCP 12.12.12.12:443 17:18676 TIME_WAIT
TCP 12.12.12.12:443 18:40086 TIME_WAIT
TCP 12.12.12.12:443 20:14698 TIME_WAIT
TCP 12.12.12.12:443 21:8742 TIME_WAIT
TCP 12.12.12.12:443 21:9222 TIME_WAIT
TCP 12.12.12.12:443 21:10050 TIME_WAIT
TCP 12.12.12.12:443 21:22212 TIME_WAIT
TCP 12.12.12.12:443 23:20186 TIME_WAIT
TCP 12.12.12.12:443 24:9702 TIME_WAIT
TCP 12.12.12.12:443 24:29658 TIME_WAIT
TCP 12.12.12.12:443 24:54316 TIME_WAIT
TCP 12.12.12.12:443 24:54740 TIME_WAIT
TCP 12.12.12.12:443 26:63912 TIME_WAIT
TCP 12.12.12.12:443 34:38802 TIME_WAIT
TCP 12.12.12.12:443 34:48344 TIME_WAIT
TCP 12.12.12.12:443 35:19314 TIME_WAIT
TCP 12.12.12.12:443 35:56518 TIME_WAIT
TCP 12.12.12.12:443 36:26848 TIME_WAIT
TCP 12.12.12.12:443 36:29840 TIME_WAIT
TCP 12.12.12.12:443 37:22090 TIME_WAIT
TCP 12.12.12.12:443 37:41662 TIME_WAIT
TCP 12.12.12.12:443 37:62462 TIME_WAIT
TCP 12.12.12.12:443 37:65246 TIME_WAIT
TCP 12.12.12.12:443 38:3746 TIME_WAIT
TCP 12.12.12.12:443 38:13518 TIME_WAIT
TCP 12.12.12.12:443 38:19626 TIME_WAIT
TCP 12.12.12.12:443 38:46588 TIME_WAIT
TCP 12.12.12.12:443 38:55504 TIME_WAIT
TCP 12.12.12.12:443 39:13096 TIME_WAIT
TCP 12.12.12.12:443 40:14808 TIME_WAIT
TCP 12.12.12.12:443 40:18046 TIME_WAIT
TCP 12.12.12.12:443 40:19968 TIME_WAIT
TCP 12.12.12.12:443 40:37858 TIME_WAIT
TCP 12.12.12.12:443 40:47914 TIME_WAIT
TCP 12.12.12.12:443 40:54890 TIME_WAIT
TCP 12.12.12.12:443 40:58958 TIME_WAIT
TCP 12.12.12.12:443 40:61998 TIME_WAIT
TCP 12.12.12.12:443 41:5752 TIME_WAIT
TCP 12.12.12.12:443 41:6420 ESTABLISHED
TCP 12.12.12.12:443 41:6424 TIME_WAIT
TCP 12.12.12.12:443 41:8224 TIME_WAIT
TCP 12.12.12.12:443 41:23838 TIME_WAIT
TCP 12.12.12.12:443 41:56540 TIME_WAIT
TCP 12.12.12.12:443 42:44002 TIME_WAIT
TCP 12.12.12.12:443 42:48300 TIME_WAIT
TCP 12.12.12.12:443 45:16840 TIME_WAIT
TCP 12.12.12.12:443 45:44966 TIME_WAIT
TCP 12.12.12.12:443 45:45542 TIME_WAIT
TCP 12.12.12.12:443 static:6008 TIME_WAIT
TCP 12.12.12.12:443 static:50129 TIME_WAIT
TCP 12.12.12.12:443 static:11337 TIME_WAIT
TCP 12.12.12.12:443 static:57596 TIME_WAIT
TCP 12.12.12.12:443 188:13459 ESTABLISHED
TCP 12.12.12.12:443 193.32.169.146:40330 ESTABLISHED
TCP 12.12.12.12:443 dD5765B1A:53034 ESTABLISHED
TCP 12.12.12.12:443 ip-213-127-45-40:59736 ESTABLISHED
TCP 12.12.12.12:443 ip-213-127-45-40:59772 ESTABLISHED
TCP 12.12.12.12:3389 31.184.218.129:37280 CLOSE_WAIT
TCP 12.12.12.12:3389 219:63027 ESTABLISHED
TCP 12.12.12.12:3389 89.205.133.110:2255 ESTABLISHED
TCP 12.12.12.12:3389 101.204.228.50:55310 ESTABLISHED
TCP 12.12.12.12:3389 static:59832 ESTABLISHED
TCP 12.12.12.12:54209 exampleserver:https ESTABLISHED
TCP 12.12.12.12:54217 ams17s10-in-f10:https ESTABLISHED
TCP 12.12.12.12:54242 lhr26s05-in-f10:https ESTABLISHED
TCP 12.12.12.12:54288 server-52-222-139-23:https ESTABLISHED
TCP 12.12.12.12:54533 40.126.31.135:https TIME_WAIT
TCP 12.12.12.12:54534 20.73.194.208:https TIME_WAIT
TCP 12.12.12.12:54535 20.67.183.221:https TIME_WAIT
TCP 12.12.12.12:54544 40.125.122.176:https TIME_WAIT
TCP 12.12.12.12:54620 ec2-54-77-149-211:https CLOSE_WAIT
TCP 12.12.12.12:54623 a23-79-157-152:http TIME_WAIT
TCP 12.12.12.12:54625 a23-79-157-152:http TIME_WAIT
TCP 12.12.12.12:54626 a23-79-157-152:http TIME_WAIT
TCP 12.12.12.12:54627 a23-79-157-152:http TIME_WAIT
TCP 12.12.12.12:54628 a23-79-157-152:http TIME_WAIT
TCP 12.12.12.12:54629 a23-79-157-152:http TIME_WAIT
TCP 12.12.12.12:54633 80-69-93-62:https CLOSE_WAIT
TCP 12.12.12.12:54637 exampleserver:ms-sql-s TIME_WAIT
TCP 127.0.0.1:3306 exampleserver:54634 ESTABLISHED
TCP 127.0.0.1:49674 exampleserver:49675 ESTABLISHED
TCP 127.0.0.1:49675 exampleserver:49674 ESTABLISHED
TCP 127.0.0.1:49676 exampleserver:49677 ESTABLISHED
TCP 127.0.0.1:49677 exampleserver:49676 ESTABLISHED
TCP 127.0.0.1:54634 exampleserver:3306 ESTABLISHED
TCP 172.25.64.1:53 exampleserver:0 LISTENING
TCP 172.25.64.1:139 exampleserver:0 LISTENING
TCP 172.25.64.1:54638 exampleserver:ms-sql-s TIME_WAIT
TCP [::]:21 exampleserver:0 LISTENING
TCP [::]:25 exampleserver:0 LISTENING
TCP [::]:80 exampleserver:0 LISTENING
TCP [::]:135 exampleserver:0 LISTENING
TCP [::]:443 exampleserver:0 LISTENING
TCP [::]:445 exampleserver:0 LISTENING
TCP [::]:587 exampleserver:0 LISTENING
TCP [::]:1433 exampleserver:0 LISTENING
TCP [::]:2179 exampleserver:0 LISTENING
TCP [::]:3306 exampleserver:0 LISTENING
TCP [::]:3389 exampleserver:0 LISTENING
TCP [::]:5985 exampleserver:0 LISTENING
TCP [::]:8983 exampleserver:0 LISTENING
TCP [::]:33060 exampleserver:0 LISTENING
TCP [::]:47001 exampleserver:0 LISTENING
TCP [::]:49231 exampleserver:0 LISTENING
TCP [::]:49664 exampleserver:0 LISTENING
TCP [::]:49665 exampleserver:0 LISTENING
TCP [::]:49666 exampleserver:0 LISTENING
TCP [::]:49667 exampleserver:0 LISTENING
TCP [::]:49668 exampleserver:0 LISTENING
TCP [::]:49673 exampleserver:0 LISTENING
TCP [::]:49881 exampleserver:0 LISTENING
TCP [::1]:8983 exampleserver:53194 ESTABLISHED
TCP [::1]:8983 exampleserver:54283 ESTABLISHED
TCP [::1]:8983 exampleserver:54476 TIME_WAIT
TCP [::1]:8983 exampleserver:54622 ESTABLISHED
TCP [::1]:8983 exampleserver:54632 FIN_WAIT_2
TCP [::1]:8983 exampleserver:62954 ESTABLISHED
TCP [::1]:8983 exampleserver:62971 ESTABLISHED
TCP [::1]:8983 exampleserver:63272 ESTABLISHED
TCP [::1]:53194 exampleserver:8983 ESTABLISHED
TCP [::1]:54283 exampleserver:8983 ESTABLISHED
TCP [::1]:54622 exampleserver:8983 ESTABLISHED
TCP [::1]:54632 exampleserver:8983 CLOSE_WAIT
TCP [::1]:62954 exampleserver:8983 ESTABLISHED
TCP [::1]:62971 exampleserver:8983 ESTABLISHED
TCP [::1]:63272 exampleserver:8983 ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:1433 exampleserver:53967 ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:1433 exampleserver:53981 ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:1433 exampleserver:54262 ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:1433 exampleserver:54276 ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:1433 exampleserver:54635 ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:53967 exampleserver:ms-sql-s ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:53981 exampleserver:ms-sql-s ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:54262 exampleserver:ms-sql-s ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:54276 exampleserver:ms-sql-s ESTABLISHED
TCP [fe80::67ff:cd37:2e9:65fe%14]:54635 exampleserver:ms-sql-s ESTABLISHED
TCP [fe80::bc0d:78b4:381:b364%18]:54636 exampleserver:ms-sql-s TIME_WAIT
UDP 0.0.0.0:123 *:*
UDP 0.0.0.0:500 *:*
UDP 0.0.0.0:3389 *:*
UDP 0.0.0.0:4500 *:*
UDP 0.0.0.0:5353 *:*
UDP 0.0.0.0:5355 *:*
UDP 12.12.12.12:137 *:*
UDP 12.12.12.12:138 *:*
UDP 127.0.0.1:61804 *:*
UDP 172.25.64.1:53 *:*
UDP 172.25.64.1:137 *:*
UDP 172.25.64.1:138 *:*
UDP [::]:123 *:*
UDP [::]:500 *:*
UDP [::]:3389 *:*
UDP [::]:4500 *:*
UDP [::]:5353 *:*
UDP [::]:5355 *:*
更新 3
在 Windows 防火墙中,我为入站和出站创建了新的 TCP 端口规则,允许端口“80,443,8050”(用于域/私有/公共)
然后在我的容器中重试运行C:\root\scrapy>scrapy foobar -o allobjects.json
,但仍然收到“连接被拒绝”错误。
我运行了这些命令:
在我的容器中:C:\root\scrapy>netstat -a
在我的主机操作系统上:D:\Progams\netstat -a
在这两个输出中都没有显示端口 8050...但我预计它会在那里?
更新 4
基于@LSerni/@Pankaj 对 Splash 的建议:
我不知道在您的示例中我需要输入什么而不是 scrapinghub/splash
。
我试过了:
PS D:\Programs\image_addons> docker run -p 8050:8050 -p 5023:5023 scrapinghub/splash
Unable to find image 'scrapinghub/splash:latest' locally
latest: Pulling from scrapinghub/splash
docker: image operating system "linux" cannot be used on this platform.
还有:
PS D:\Programs\image_addons> docker run -p 8050:8050 -p 5023:5023 scrapy/splash
Unable to find image 'scrapy/splash:latest' locally
docker: Error response from daemon: pull access denied for scrapy/splash, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
从我的帖子中可以看出,Splash 是通过我的 Docker 文件安装到我的 scrapy 映像中的,那么我需要如何启动它?
更新 5
基于@Thiago Curvelo 对 Splash 的评论:
所以在本地 Windows 10 和 Docker 桌面上,上述设置适用于我,我没有(至少没有明确地)启动一个单独的 Splash 实例,但也许这是构建到 Docker 桌面的逻辑?在我的服务器上,我无法使用 Docker Desktop,因为它不支持 Hyper-V。
如果您查看我的 Dockerfile,您会看到我将 scrapy-splash
安装到我的映像中。
现在说到你的意思:显然我需要单独启动一个Splash
实例?
该容器是否必须与我的scrapy
容器分开并行运行?或者可以(并且应该?)我将Splash
图像添加到我现有的scrapy
图像并基于它运行一个容器?我是新手,所以不确定这是否是我应该搜索的内容。
(我也只是在尝试scrapy/splash
,因为我认为scrapinghub
部分可能指的是图像名称,在我的情况下是scrapy
,我想我也错了;-))
【问题讨论】:
你试过在 dockerfile 中暴露 8050 端口然后进行构建吗?基本上是 EXPOSE 8050 @Flo 你可能把Splash 误认为scrapy-splash。第一个是无头浏览器引擎,而后者是集成 Splash 和 Scrapy 的包装器。无论如何,您都需要启动一个 Splash 实例,该实例默认在 Linux 容器中运行。如果您无法在系统中启动 Linux 容器,则可能需要构建自己的映像或为此使用 VM。另外,您弄错了图像的名称。正确的是scrapinghub/splash
(没有可用的scrapy/splash
)
恕我直言,我认为在 Windows 中安装 Splash 不值得麻烦(或者即使有可能)。我的建议是在 Linux VM 中运行 Splash(例如使用 Virtualbox)或在云提供商中启动它。另一种选择是使用Zyte (formely Scrapinghub) platform,它在他们的应用程序中提供 Splash 实例。
你可以坚持使用你的scrapy容器并为Splash拥有一个专用的VM,只要它们可以通过网络相互访问。
我不是 Windows 专家,但我认为在 WSL 上运行 docker 应该可以。
【参考方案1】:
你需要先启动一个Splash instance,让它监听8050端口。例如:
docker run -dit -p 8050:8050 --name my_splash scrapinghub/splash
然后,将启动 URL 设置为指向该正在运行的容器:
settings.py:
SPLASH_URL = 'http://my_splash:8050/'
最后,启动 Scrapy 容器,将其链接到 Splash 容器:
docker run -it --link my_splash --rm scrapy
这样你就可以将 Scrapy 的请求发送到 Splash。
【讨论】:
@RajVerma 它显然不是防火墙:没有 8050 监听端口(netstat
这么说)。现在,可能需要打开防火墙以允许访问 8050 端口,但如果由于没有人在监听而导致端口不存在,则允许访问不存在的对象将始终没有任何结果。
嗯,有一个更新 3,发布防火墙更新,它也不起作用,所以防火墙无法显示。它也可能是 AD 级别的一些组策略,但肯定需要进一步挖掘。我的假设完全基于初始数据,结果证明是错误的。
@Thiago Curvelo:谢谢!你能看看我的更新5吗?这是到达我想去的地方的最后一步(我认为)。【参考方案2】:
你说你签了this question。
您是否启动了 Splash 容器?我从Pankaj's answer 引用上述问题:
在 docker 上运行启动画面
# Start the container:
$ sudo docker run -p 8050:8050 -p 5023:5023 scrapinghub/splash
# Splash is now available at [...] ports 8050 (http) and 5023 (telnet).
【讨论】:
谢谢!你能看看我的更新5吗?这是到达我想要的地方的最后一步(我认为)。 @Flo 是的,您必须“与 scrapy 容器分开并并行地”运行 Splash 容器。当你看到端口 8050 上升时,你就知道你已经准备好了。以上是关于连接被对方拒绝:10061:无法建立连接,因为目标机器主动拒绝它的主要内容,如果未能解决你的问题,请参考以下文章
TCP错误代码10061:无法建立连接,因为目标机器主动拒绝它 127.0.0.1:8732
DJANGO EMAIL CONFIRMATION: [WinError 10061] 由于目标机器主动拒绝,无法建立连接
python socket报错[WinError 10061] 由于目标计算机积极拒绝,无法连接。 是啥意思?