Celery Beat 调度程序不会在 Django 中执行我的任务
Posted
技术标签:
【中文标题】Celery Beat 调度程序不会在 Django 中执行我的任务【英文标题】：Celery Beat scheduler won't execute my task in Django 【发布时间】：2020-06-05 05:47:56 【问题描述】：我在 Django 中设置 celery 模块时遇到了一些麻烦。我不仅对设置要求感到非常困惑，而且我也很难理解我的错误。
我想在每天的特定时间运行一个函数，这是我以前用 crontab 做的事情。 Celery 显然更适合这种类型的任务，这就是我试图转向这种解决方案的原因。
设置似乎没问题，我看到 celery 调度程序在我调用它时运行，但在间隔开始时我的任务没有得到处理。
我看到很多 (!!!) celery 的不同设置，所以我的代码可能包含冗余和不必要的行。
我要触发的函数名为 team_accountabillity()，存在于我的应用“myteam.task.py”中
我有 Celery 4.4.0，我使用 rabbitMQ 作为队列管理器（我还安装了 redis，因为很多教程都使用这个）。
这是我的 proj/settings.py
from __future__ import absolute_import
import os
from datetime import timedelta
from celery import Celery
from django.conf import settings
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')
CELERY_BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_BEAT_SCHEDULER =
'team accountability':
'task':'team_accountability', 'schedule': timedelta(seconds=10),
,
CELERY_TIMEZONE = 'UTC'
CELERY_ENABLE_UTC = True
INSTALLED_APPS = [
'celerybeat_status',
'django_celery_beat',
'myteam.apps.MyteamConfig',
]
proj/celery.py 文件：
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from celery.schedules import crontab
from django.apps import apps
from django.conf import settings
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')
app = Celery('proj')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks(lambda: [n.name for n in apps.get_app_configs()])
##documentation regarding the next line:
##https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html#beat-entries
##I find it very weird that this command is very similar to the CELERY_BEAT_SCHEDULER in settings.py
##I honestly don't know what this does
@app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
sender.add_periodic_task(10.0, team_accountability, name='team accountability')
@app.task(bind=True)
def debug_task(self):
print('Request: 0!r'.format(self.request))
proj/init.py 文件：
from __future__ import absolute_import, unicode_literals
from .celery import app as celery_app
__all__ = ('celery_app')
最后，这是我的 app/task.py (myteam/task.py)
from __future__ import absolute_import, unicode_literals
from celery import shared_task
from myteam.models import Team
##decorator should be @shared_task or @task???
@shared_task
def team_accountability():
t_all = Team.objects.all()
for t in t_all:
total = 0
s_all = t.stocks.all()
for s in s_all:
symbol = s.symbol
queryset = Stock.objects.filter(symbol__contains=symbol).order_by("-id").first()
total = total + queryset.change_percent
t.total = round(total, 3)
t.save()
说了这么多，下面是我运行命令时得到的结果：
celery -A proj worker -l debug
我得到的是这样的：
[2020-02-21 12:39:03,420: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2020-02-21 12:39:03,420: DEBUG/MainProcess] | Worker: Building graph...
[2020-02-21 12:39:03,421: DEBUG/MainProcess] | Worker: New boot order: StateDB, Beat, Timer, Hub, Pool, Autoscaler, Consumer
[2020-02-21 12:39:03,429: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2020-02-21 12:39:03,429: DEBUG/MainProcess] | Consumer: Building graph...
[2020-02-21 12:39:03,447: DEBUG/MainProcess] | Consumer: New boot order: Connection, Events, Mingle, Gossip, Tasks, Control, Agent, Heart, event loop
-------------- celery@matador-B450M-DS3H v4.4.0 (cliffs)
--- ***** -----
-- ******* ---- Linux-4.15.0-70-generic-x86_64-with-debian-buster-sid 2020-02-21 12:39:03
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: TheRodeoProject:0x7f00682ecc10
- ** ---------- .> transport: redis://localhost:6379//
- ** ---------- .> results: redis://localhost:6379/
- *** --- * --- .> concurrency: 12 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. TheRodeoProject.celery.debug_task
. celery.accumulate
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap
[2020-02-21 12:39:03,455: DEBUG/MainProcess] | Worker: Starting Hub
[2020-02-21 12:39:03,455: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:03,455: DEBUG/MainProcess] | Worker: Starting Pool
[2020-02-21 12:39:03,891: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:03,891: DEBUG/MainProcess] | Worker: Starting Consumer
[2020-02-21 12:39:03,891: DEBUG/MainProcess] | Consumer: Starting Connection
[2020-02-21 12:39:03,900: INFO/MainProcess] Connected to redis://localhost:6379//
[2020-02-21 12:39:03,901: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:03,901: DEBUG/MainProcess] | Consumer: Starting Events
[2020-02-21 12:39:03,908: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:03,908: DEBUG/MainProcess] | Consumer: Starting Mingle
[2020-02-21 12:39:03,908: INFO/MainProcess] mingle: searching for neighbors
[2020-02-21 12:39:04,924: INFO/MainProcess] mingle: all alone
[2020-02-21 12:39:04,924: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:04,925: DEBUG/MainProcess] | Consumer: Starting Gossip
[2020-02-21 12:39:04,930: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:04,930: DEBUG/MainProcess] | Consumer: Starting Tasks
[2020-02-21 12:39:04,934: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:04,934: DEBUG/MainProcess] | Consumer: Starting Control
[2020-02-21 12:39:04,935: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:04,935: DEBUG/MainProcess] | Consumer: Starting Heart
[2020-02-21 12:39:04,936: DEBUG/MainProcess] ^-- substep ok
[2020-02-21 12:39:04,936: DEBUG/MainProcess] | Consumer: Starting event loop
[2020-02-21 12:39:04,937: DEBUG/MainProcess] | Worker: Hub.register Pool...
[2020-02-21 12:39:04,937: WARNING/MainProcess] /home/matador/anaconda3/envs/venv1/lib/python3.7/site-packages/celery/fixups/django.py:203: UserWarning: Using settings.DEBUG leads to a memory
leak, never use this setting in production environments!
leak, never use this setting in production environments!''')
[2020-02-21 12:39:04,937: INFO/MainProcess] celery@matador-B450M-DS3H ready.
正如您所见，当我运行此命令时，我想添加到队列中的任务没有出现。其余的似乎还不错。
编辑：
看来解决这个问题的方法是用 celery beat 代替普通的 celery。
【问题讨论】：
你能不能尝试另一种方式来添加任务到调度器： app.conf.beat_schedule = '清理 Celery 后端的过期结果': 'task': 'celery.backend_cleanup', 'schedule' : crontab(小时=4, 分钟=0), 【参考方案1】：您可能需要运行一个工作 celery 进程。
celery -A proj worker -l debug
【讨论】：
我以前试过这个，但我不确定如何处理结果。我获得了基本上看起来像 --help 函数的命令列表。不过我的任务没有完成。 什么意思？ 我得到一长串命令和选项，如下所示：用法：celery以上是关于Celery Beat 调度程序不会在 Django 中执行我的任务的主要内容，如果未能解决你的问题，请参考以下文章