python3 异步任务之----celery

Posted 2020-11-20 vglede

celery是一个“自带电池”的任务队列。

运行环境：

• Django==1.11.4
• PyMySQL==0.8.1
• configparser==3.5.0
• django-crontab==0.7.1
• celery==3.1.25
• redis==3.2.8

工程列表：

 

1. 在工程下的settings.py文件中添加如下内容：

BROKER_URL = \'redis://127.0.0.1:6379\'
CELERY_RESULT_BACKEND = \'redis://127.0.0.1:6379\'
CELERY_ACCEPT_CONTENT = [\'application/json\']
CELERY_TASK_SERIALIZER = \'json\'
CELERY_RESULT_SERIALIZER = \'json\'
CELERY_TIMEZONE = \'Asia/Shanghai

　　2.在工程目录下添加celery.py文件，增加如下内容：

# coding=utf8
# Autor : Vglede
# Time  : 2018/6/26 16:53
# File  : celery.py

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

# set the default Django settings module for the \'Ops\' program.
os.environ.setdefault(\'DJANGO_SETTINGS_MODULE\', \'Ops.settings\')
app = Celery(\'release\')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object(\'django.conf:settings\')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


@app.task(bind=True)
def debug_task(self):
    print(\'Request: {0!r}\'.format(self.request))

　　3.在工程下面__init__.py添加如下代码

from .celery import app as celery_app

　　4.在注册的应用（release）下添加任务：

# coding=utf8
# Autor : Vglede
# Time  : 2018/6/27 10:46
# File  : release_api.py
from celery import task

@task
def release_async_bash(src_info):
   print("job[release_async_bash] include %s is running"%(str(src_info)))
    result = True
    return result

　　5.通过任务函数名，使用delay()方法去启动任务。

#somecode
exec_order="哈哈"
release_async_bash.delay(exec_order)

　　6.创建pid以及log存放目录：

mkdir  -p /var/run/celery/
mkdir  -p /var/log/celery/

　　7.启动celery（附上编者写的脚本）：

# coding=utf8
# Autor : Vglede
# Time  : 2018/6/28 14:53
# File  : contorl_celery.sh

#!/bin/sh
#

case "$1" in
    start)
        celery multi start w1 -A Ops -l info --pidfile=/var/run/celery/%n.pid \\
                                        --logfile=/var/log/celery/%n.log
        ;;
    stop)
        if [ ! -f $PIDFILE ]
        then
                echo "$PIDFILE does not exist, process is not running"
        else
                PIDFILE=`ls /var/run/celery/*.pid`
                PID=$(cat $PIDFILE)
                echo "Stopping ..."
                celery multi stop w1 -A Ops -l info --pidfile=/var/run/celery/%n.pid \\
                                        --logfile=/var/log/celery/%n.log
                while [ -x /proc/${PID} ]
                do
                    echo "Waiting for Celery to shutdown ..."
                    sleep 1
                done
                echo "Celery stopped"
        fi
        ;;
    restart)
        if [ ! -f $PIDFILE ]
        then
                echo "$PIDFILE does not exist, process is not running"
        else
                PIDFILE=`ls /var/run/celery/*.pid`
                PID=$(cat $PIDFILE)
                echo "Stopping ..."
                celery multi stop w1 -A Ops -l info --pidfile=/var/run/celery/%n.pid \\
                                        --logfile=/var/log/celery/%n.log
                while [ -x /proc/${PID} ]
                do
                    echo "Waiting for Celery to shutdown ..."
                    sleep 1
                done
                echo "Celery stopped"
        fi
        celery multi start w1 -A Ops -l info --pidfile=/var/run/celery/%n.pid \\
                                        --logfile=/var/log/celery/%n.log
        ;;

    *)
        echo "Please use start or stop as first argument"
        ;;
esac 

 了解更多，参考celery3.1.7文档（http://docs.jinkan.org/docs/celery/getting-started/first-steps-with-celery.html）