无法让 Celery 使用 prod 设置处理数字海洋液滴

Posted 2023-02-23

【中文标题】无法让 Celery 使用 prod 设置处理数字海洋液滴

【英文标题】：Can't get Celery to work on digital ocean droplet with prod settings

【发布时间】：2021-07-31 02:11:04

【问题描述】：

它在本地服务器上运行良好，但是当我尝试在 ssh 后启动工作器时出现错误。

/var/www/bin/celery -A stock worker -l info

我知道 DJANGO_SETTINGS_MODULE 设置正确，因为我有一个打印语句显示它已设置（并且服务器的其余部分是实时的，使用生产设置）。我也试过使用这个命令，它给出了同样的错误。

DJANGO_SETTINGS_MODULE="stock.settings.pro" /var/www/bin/celery -A stock worker -l info

我的主目录中有一个 celery 目录（在我的设置目录旁边）。它包含一个 init.py 文件和一个 conf.py（设置结果后端）。这是 init 文件：

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from celery.schedules import crontab

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'stock.settings.pro')
DJANGO_SETTINGS_MODULE = os.environ.get('DJANGO_SETTINGS_MODULE')
print("celery - ", DJANGO_SETTINGS_MODULE)
BASE_REDIS_URL = os.environ.get('REDIS_URL', 'redis://localhost:6379')

app = Celery('stock')
app.config_from_object('django.conf:settings', namespace='CELERY')

app.autodiscover_tasks()

app.conf.broker_url = BASE_REDIS_URL

app.conf.beat_scheduler = 'django_celery_beat.schedulers:DatabaseScheduler'

这是回溯错误：

Traceback (most recent call last):
File "/var/www/lib/python3.8/site-packages/kombu/utils/objects.py", line 42, in __get__
return obj.__dict__[self.__name__]
KeyError: 'data'

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/www/bin/celery", line 8, in <module>
sys.exit(main())
File "/var/www/lib/python3.8/site-packages/celery/__main__.py", line 16, in main
_main()
File "/var/www/lib/python3.8/site-packages/celery/bin/celery.py", line 322, in main
cmd.execute_from_commandline(argv)
File "/var/www/lib/python3.8/site-packages/celery/bin/celery.py", line 499, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/var/www/lib/python3.8/site-packages/celery/bin/base.py", line 305, in execute_from_commandline
return self.handle_argv(self.prog_name, argv[1:])
File "/var/www/lib/python3.8/site-packages/celery/bin/celery.py", line 491, in handle_argv
return self.execute(command, argv)
File "/var/www/lib/python3.8/site-packages/celery/bin/celery.py", line 415, in execute
return cls(
File "/var/www/lib/python3.8/site-packages/celery/bin/worker.py", line 221, in run_from_argv
*self.parse_options(prog_name, argv, command))
File "/var/www/lib/python3.8/site-packages/celery/bin/base.py", line 428, in parse_options
self.parser = self.create_parser(prog_name, command)
File "/var/www/lib/python3.8/site-packages/celery/bin/base.py", line 444, in create_parser
self.add_arguments(parser)
File "/var/www/lib/python3.8/site-packages/celery/bin/worker.py", line 278, in add_arguments
default=conf.worker_state_db,
File "/var/www/lib/python3.8/site-packages/celery/utils/collections.py", line 134, in __getattr__
return self[k]
File "/var/www/lib/python3.8/site-packages/celery/utils/collections.py", line 444, in __getitem__
return getitem(k)
File "/var/www/lib/python3.8/site-packages/celery/utils/collections.py", line 287, in __getitem__
return mapping[_key]
File "/usr/lib/python3.8/collections/__init__.py", line 1006, in __getitem__
if key in self.data:
File "/var/www/lib/python3.8/site-packages/kombu/utils/objects.py", line 44, in __get__
value = obj.__dict__[self.__name__] = self.__get(obj)
File "/var/www/lib/python3.8/site-packages/celery/app/base.py", line 141, in data
return self.callback()
File "/var/www/lib/python3.8/site-packages/celery/app/base.py", line 920, in _finalize_pending_conf
conf = self._conf = self._load_config()
File "/var/www/lib/python3.8/site-packages/celery/app/base.py", line 930, in _load_config
self.loader.config_from_object(self._config_source)
File "/var/www/lib/python3.8/site-packages/celery/loaders/base.py", line 131, in config_from_object
self._conf = force_mapping(obj)
File "/var/www/lib/python3.8/site-packages/celery/utils/collections.py", line 54, in force_mapping
if isinstance(m, (LazyObject, LazySettings)):
File "/var/www/lib/python3.8/site-packages/django/utils/functional.py", line 224, in inner
self._setup()
File "/var/www/lib/python3.8/site-packages/django/conf/__init__.py", line 63, in _setup
self._wrapped = Settings(settings_module)
File "/var/www/lib/python3.8/site-packages/django/conf/__init__.py", line 161, in __init__
raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.

非常感谢您的帮助，我已经尝试解决这个问题几天了，但还没有真正得到任何帮助！

【参考方案1】：

我在寻找其他内容时偶然发现了您的问题。根据您的日志，问题显然是：“SECRET_KEY 设置不能为空。”

在 celery 运行时，您的密钥似乎还没有“存在”在您的环境中。你有同样的发展吗？为了进行测试，请尝试在您的 settings.py 文件和/或 celery 初始化文件的顶部设置一个随机密钥，如上所述。如果这个问题消失了，那么你就知道问题所在了。你可以从那里修复它。

通常 SECRET_KEY 应该是您环境的一部分，并在 celery 运行之前填充。在这种情况下，您的运行顺序可能是错误的，或者环境变量设置不正确（或在 celery 初始化之后）。

我希望这是有道理的。

在我自己的开发（非生产）环境中，我的所有变量包括 .env 文件中的 SECRET_KEY 并且会遇到同样的问题。基本上 celery init.py 文件将在 SECRET_KEY 初始化之前加载。为了规避这个问题，我只是在我的 settings.py 文件顶部放置了一个标志：

if development:
   import dotenv
   dotenv.load_dotenv()

根据环境，我将“开发”设置为 True 或 False。再次重申：如果在生产中发生此错误，您的初始化顺序很可能有问题。