无法在 AWS Elastic Beanstalk 上安装 pycurl==7.43.0.5

Posted 2023-02-24

【中文标题】无法在 AWS Elastic Beanstalk 上安装 pycurl==7.43.0.5

【英文标题】：Unable to install pycurl==7.43.0.5 on AWS Elastic Beanstalk

【发布时间】：2020-10-13 06:13:21

【问题描述】：

大家好，我一直在尝试在运行 python 3.7 的 EB 上添加 pycurl，但是当我尝试启动我的 celery 应用程序时，它返回以下错误：

“curl客户端需要pycurl库”

正在执行的命令是：

celery -A sportspot worker -B

[2020-06-23 07:36:14,815: CRITICAL/MainProcess] Unrecoverable error: ImportError('The curl client requires the pycurl library.')
2020-06-23 07:36:16,508 P8649 [INFO]    Traceback (most recent call last):
2020-06-23 07:36:16,508 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 20, in get_client
2020-06-23 07:36:16,508 P8649 [INFO]        return hub._current_http_client
2020-06-23 07:36:16,508 P8649 [INFO]    AttributeError: 'Hub' object has no attribute '_current_http_client'
2020-06-23 07:36:16,508 P8649 [INFO]    
2020-06-23 07:36:16,508 P8649 [INFO]    During handling of the above exception, another exception occurred:
2020-06-23 07:36:16,508 P8649 [INFO]    
2020-06-23 07:36:16,508 P8649 [INFO]    Traceback (most recent call last):
2020-06-23 07:36:16,508 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/worker/worker.py", line 208, in start
2020-06-23 07:36:16,508 P8649 [INFO]        self.blueprint.start(self)
2020-06-23 07:36:16,508 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
2020-06-23 07:36:16,508 P8649 [INFO]        step.start(parent)
2020-06-23 07:36:16,508 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/bootsteps.py", line 369, in start
2020-06-23 07:36:16,508 P8649 [INFO]        return self.obj.start()
2020-06-23 07:36:16,508 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
2020-06-23 07:36:16,509 P8649 [INFO]        blueprint.start(self)
2020-06-23 07:36:16,509 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
2020-06-23 07:36:16,509 P8649 [INFO]        step.start(parent)
2020-06-23 07:36:16,509 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 599, in start
2020-06-23 07:36:16,509 P8649 [INFO]        c.loop(*c.loop_args())
2020-06-23 07:36:16,509 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/worker/loops.py", line 83, in asynloop
2020-06-23 07:36:16,509 P8649 [INFO]        next(loop)
2020-06-23 07:36:16,509 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 301, in create_loop
2020-06-23 07:36:16,509 P8649 [INFO]        item()
2020-06-23 07:36:16,509 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/vine/promises.py", line 170, in __call__
2020-06-23 07:36:16,509 P8649 [INFO]        return self.throw()
2020-06-23 07:36:16,509 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/vine/promises.py", line 167, in __call__
2020-06-23 07:36:16,509 P8649 [INFO]        retval = fun(*final_args, **final_kwargs)
2020-06-23 07:36:16,509 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/transport/SQS.py", line 391, in _schedule_queue
2020-06-23 07:36:16,509 P8649 [INFO]        queue, callback=promise(self._loop1, (queue,)),
2020-06-23 07:36:16,509 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/transport/SQS.py", line 407, in _get_bulk_async
2020-06-23 07:36:16,510 P8649 [INFO]        return self._get_async(queue, maxcount, callback=callback)
2020-06-23 07:36:16,510 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/transport/SQS.py", line 417, in _get_async
2020-06-23 07:36:16,510 P8649 [INFO]        qname, count=count, connection=self.asynsqs(queue=qname),
2020-06-23 07:36:16,510 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/transport/SQS.py", line 571, in asynsqs
2020-06-23 07:36:16,510 P8649 [INFO]        region=self.region
2020-06-23 07:36:16,510 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/aws/sqs/connection.py", line 27, in __init__
2020-06-23 07:36:16,510 P8649 [INFO]        **kwargs
2020-06-23 07:36:16,510 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/aws/connection.py", line 198, in __init__
2020-06-23 07:36:16,510 P8649 [INFO]        **http_client_params)
2020-06-23 07:36:16,510 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/aws/connection.py", line 151, in __init__
2020-06-23 07:36:16,510 P8649 [INFO]        self._httpclient = http_client or get_client()
2020-06-23 07:36:16,510 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 22, in get_client
2020-06-23 07:36:16,510 P8649 [INFO]        client = hub._current_http_client = Client(hub, **kwargs)
2020-06-23 07:36:16,510 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 13, in Client
2020-06-23 07:36:16,510 P8649 [INFO]        return CurlClient(hub, **kwargs)
2020-06-23 07:36:16,511 P8649 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/http/curl.py", line 43, in __init__
2020-06-23 07:36:16,511 P8649 [INFO]        raise ImportError('The curl client requires the pycurl library.')
2020-06-23 07:36:16,511 P8649 [INFO]    ImportError: The curl client requires the pycurl library.
2020-06-23 07:36:16,511 P8649 [INFO]     
2020-06-23 07:36:16,511 P8649 [INFO]     -------------- [e-mail] v4.4.5 (cliffs)
2020-06-23 07:36:16,511 P8649 [INFO]    --- ***** ----- 
2020-06-23 07:36:16,511 P8649 [INFO]    -- ******* ---- Linux-4.14.177-139.254.amzn2.x86_64-x86_64-with-glibc2.2.5 2020-06-23 07:36:14
2020-06-23 07:36:16,511 P8649 [INFO]    - *** --- * --- 
2020-06-23 07:36:16,511 P8649 [INFO]    - ** ---------- [config]
2020-06-23 07:36:16,511 P8649 [INFO]    - ** ---------- .> app:         sportspot:0x7f1916c1c290
2020-06-23 07:36:16,511 P8649 [INFO]    - ** ---------- .> transport:   sqs://localhost//
2020-06-23 07:36:16,511 P8649 [INFO]    - ** ---------- .> results:     disabled://
2020-06-23 07:36:16,511 P8649 [INFO]    - *** --- * --- .> concurrency: 1 (prefork)
2020-06-23 07:36:16,511 P8649 [INFO]    -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
2020-06-23 07:36:16,511 P8649 [INFO]    --- ***** ----- 
2020-06-23 07:36:16,511 P8649 [INFO]     -------------- [queues]
2020-06-23 07:36:16,512 P8649 [INFO]                    .> celery           exchange=celery(direct) key=celery

但是我尝试了不同的解决方法来解决这个问题，但没有运气，其中一些喜欢在我的 requierements.txt 中指定 pycurl 的不同变体，如下所示（当然我一次只使用一个选项进行部署，而不是全部他们）：

pycurl==7.43.0.5 #no luck here
pycurl==7.43.0.5 --global-option="--with-nss" #no luck here
pycurl==7.43.0.5 --global-option="--with-openssl" #no luck here
pycurl==7.19.3 #no luck dowgrading

我也尝试从 eb packages 命令安装包，如下所示：

packages:
    yum:
        libcurl-devel: []
        libcurl4-openssl-dev: []
        libssl-dev: []
        python-pycurl: [7.19.3]

如果我 ssh 进入 eb 实例然后转到 venv 并导入库，我会得到以下内容

[ec2-user@ip-172-31-17-183 ~]$ source /var/app/venv/staging-LQM1lest/bin/activate
(staging) [ec2-user@ip-172-31-17-183 ~]$ python3 -c 'import pycurl'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
(staging) [ec2-user@ip-172-31-17-183 ~]$ python -c 'import pycurl'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
(staging) [ec2-user@ip-172-31-17-183 ~]$ 

另外由于某种原因，eb 没有将虚拟环境存储在经典的 opt/python 路径上，所以它从一开始就让事情变得更加复杂，因为实际上只有 0 个关于它的文档。每个路径都不同，例如迁移或收集静态文件，我必须更改路径才能激活虚拟环境。

即使在亚马逊的文档中，所有教程都使用旧的“选择”路径...

在这一点上，我真的不知道还能尝试什么，因为当我在 localhost 上执行时，它可以完美运行。

我已经设法在没有 celery 的情况下部署它，但执行该部分对我们来说也非常重要。

如果您阅读了整个问题，我非常感谢您花时间分析这个问题！ :)

【参考方案1】：

我通过执行以下操作为在 python 3.7 上运行的 EB 实例解决了这个问题：

首先我通过 ssh 进入实例，然后运行以下命令：

sudo su
cd /var/app/
source /venv/staging-LQM1lest/bin/activate
(staging): pip uninstall pycurl
(staging): export CPPFLAGS=-I/usr/local/opt/openssl/include
(staging): export LDFLAGS=-L/usr/local/opt/openssl/lib
(staging): pip install pycurl --global-option="--with-openssl"

由于这些命令运行良好，我将它们转换为 ebconfig 文件中的单个命令，如下所示：

首先从requirements.txt中删除pycurl 然后将以下内容添加到您的 eb 扩展文件中：

container_commands:
  .
  . some other commands...
  .
  04_installpycurl:
    command: "source /var/app/venv/staging-LQM1lest/bin/activate && export CPPFLAGS=-I/usr/local/opt/openssl/include && export LDFLAGS=-L/usr/local/opt/openssl/lib && pip install pycurl --global-option='--with-openssl'"

请注意，我尝试从 requirements.txt 文件安装 pycurl --global-option='--with-openssl' 没有结果，似乎导出这两个变量对安装有影响，我没有关于这些事情为什么或做什么的想法，但解决了问题:)

最后注意到我的命令首先进入 venv 并激活它，所以如果你的 venv 路径不同，你必须改变它！