Celery学习--- Celery 最佳实践之与django结合实现异步任务

Posted 2021-01-02 ftl1012

django 可以轻松跟celery结合实现异步任务，只需简单配置即可

同步执行和异步执行

注意：即使Celery的任务没有执行完成，但是已经创建了任务ID。可以利用前台的定时任务发送Ajax异步请求根据ID查询结果

项目整合

项目的目录结构：

项目前提： 安装并启动Redis

CeleryTest/settings.py

INSTALLED_APPS = [
   ...
 ‘app01‘,   # 注册app
]
MIDDLEWARE = [
...
# ‘django.middleware.csrf.CsrfViewMiddleware‘,
      ...
]

STATICFILES_DIRS = (os.path.join(BASE_DIR, "statics"),)  # 现添加的配置,这里是元组，注意逗号
TEMPLATES = [
   ...
   ‘DIRS‘: [os.path.join(BASE_DIR, ‘templates‘)],
]
# for celery
CELERY_BROKER_URL = ‘redis://192.168.2.105‘,
CELERY_BACKEND_URL = ‘redis://192.168.2.105‘,  # 用于Celery的返回结果的接收

CeleryTest/urls.py

from django.contrib import admin
from django.urls import path
from django.conf.urls import url, include
from app01 import views
urlpatterns = [
   url(r‘index/‘, views.Index),
   url(r‘task_res/‘, views.task_res),
]

app01/views.py

from django.shortcuts import render,HttpResponse
from app01 import tasks
# Create your views here.
# 视图触发Celery的用户请求

def Index(request):
    print(‘进入Index...‘)
    res1 = tasks.add(5, 999)
    res = tasks.add.delay(5, 1000)
    print("res:", res)
    return HttpResponse(res)

# 前台通过ID获取Celery的结果
from celery.result import AsyncResult
def task_res(request):
    result = AsyncResult(id="5cf8ad07-8770-450e-9ccd-8244e8eeed19")
    # return HttpResponse(result.get())
    return HttpResponse(result.status)  # 状态有Pending, Success, Failure等结果

app01/tasks.py   文件名必须为tasks.py

# 文件名必须为tasks.py,Djaogo才能发现Celery
from __future__ import absolute_import, unicode_literals
from celery import shared_task

# Django starts so that the @shared_task decorator (mentioned later) will use it:　　
@shared_task    # Django的各个App里都可以导入这个任务，否则只能在app01这个Django的App内使用
def add(x, y):
    return x + y

@shared_task
def mul(x, y):
    return x * y

@shared_task
def xsum(numbers):
    return sum(numbers)

CeleryTest/celery.py 文件名必须为celery.py

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

# set the default Django settings module for the ‘celery‘ program.
os.environ.setdefault(‘DJANGO_SETTINGS_MODULE‘, ‘CeleryTest.settings‘)

app = Celery(‘CeleryTest‘)

# Using a string here means the worker don‘t have to serialize
# the configuration object to child processes.
# - namespace=‘CELERY‘ means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object(‘django.conf:settings‘, namespace=‘CELERY‘)

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()
@app.task(bind=True)
def debug_task(self):
    print(‘Request: {0!r}‘.format(self.request))

CeleryTest/__init__.py

from __future__ import absolute_import, unicode_literals

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app

__all__ = [‘celery_app‘]

Django前台运行结果[获取到了任务ID]：此时只有Django操作

Django后台运行结果：此时只有Django操作

上传文件到Linux服务器【Linux服务器需安装好Django服务】

需要Ubuntu下安装Django

pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple django

Linux下启动Celery的worker： 【此时Win7下的Django配合Linux下的Celery执行】

[email protected]:~/Celery/CeleryTest$  celery -A CeleryTest worker -l info

前台浏览器触发Django的views请求到Linux的Celery的worker进行结果处理

梳理一下整个的流程：

   Win7下的Django配置好了和Celery的整合，连接上Redis。当有请求从浏览器发送过来的时候，从URL映射到Django的views里面，在views里面调用了tasks.add.delay(5, 1000)此时将任务请求存储在了Redis里面。后台Linux下同样也放了一份根Win7下同样的工程[实际上只需要Celery的部分即可]，进入Linux下的目录后启动Celery的worker，worker从Redis中取出任务去执行。而与此同时Django已经将任务的ID返回给了前台，前台可以根据任务ID返回的状态判断任务是否完成，完成后从Redis中获取任务的结果进行页面渲染即可，从而达到了异步的效果，就是前台不用直接等待结果的返回，而是根据结果的状态来获取最后的结果。

Linux下安装Django

pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple django

问题解决

【20180430】一个由于Django版本和Celery版本差异引起的错误，导致一下午很失落，最后误打误撞解决,很失望

报错信息：

KeyError: ‘backend‘

During handling of the above exception, another exception occurred:

settings.py    ---Celery的配置错误：

  # for celery
CELERY_BROKER_URL = ‘redis://192.168.2.105‘,
CELERY_RESULT_BACKEND = ‘redis://192.168.2.105‘,  # 用于Celery的返回结果的接收

问题解决：

settings.py

# for celery
CELERY_BROKER_URL = ‘redis://192.168.2.105‘,
CELERY_BACKEND_URL = ‘redis://192.168.2.105‘,  # 用于Celery的返回结果的接收