Celery / Heroku - 使用 Heroku run python 在后台运行时,延迟()啥也不做
Posted
技术标签:
【中文标题】Celery / Heroku - 使用 Heroku run python 在后台运行时,延迟()啥也不做【英文标题】:Celery / Heroku - delay() does nothing when ran in the background using Heroku run pythonCelery / Heroku - 使用 Heroku run python 在后台运行时,延迟()什么也不做 【发布时间】:2018-06-13 07:13:34 【问题描述】:我正在尝试使用 Celery 添加后台工作人员,以便每天自动运行脚本以更新我的网站提供的信息。我已经将 Celery 与 Django 和 Heroku 集成,我可以导入模块并使用该功能,但是当我使用 add.delay()
command 时它会冻结,直到我按 Ctrl+C 取消该命令。我正在使用 celery 4.1 以下是我运行命令的方式:
heroku ps:scale worker=1
heroku run python
>>>from Buylist.tasks import *
>>>add(2,3)
>>>5
>>>add.delay(2,3)
#-- Freezes until I press Control+C
如果您能帮我找出我的设置在哪里配置错误,那就太好了。我正在测试自动取款机。 tasks.py
是获取工作示例的示例代码,然后我将继续了解 CELERY_BEAT
设置
项目/celery.py
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
app = Celery('project')
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')
app.conf.update(BROKER_URL=os.environ['REDIS_URL'],
CELERY_RESULT_BACKEND=os.environ['REDIS_URL'])
# Load task modules from all registered Django app configs.
app.autodiscover_tasks()
@app.task(bind=True)
def debug_task(self):
print('Request: 0!r'.format(self.request))
Buylist/tasks # Buylist 是我根目录中的单个应用程序
# Create your tasks here
from __future__ import absolute_import, unicode_literals
from celery import shared_task
from celery import Celery
from celery.schedules import crontab
@shared_task
def test(arg):
print(arg)
@shared_task
def add(x, y):
return x + y
@shared_task
def mul(x, y):
return x * y
@shared_task
def xsum(numbers):
return sum(numbers)
项目/settings.py
from __future__ import absolute_import, unicode_literals
import dj_database_url
"""
Django settings for project project.
Generated by 'django-admin startproject' using Django 2.0.
For more information on this file, see
https://docs.djangoproject.com/en/2.0/topics/settings/
For the full list of settings and their values, see
https://docs.djangoproject.com/en/2.0/ref/settings/
"""
import os
import django_heroku
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
#Read secret key from a file
SECRET_KEY = 'KEY'
# SECURITY WARNING: don't run with debug turned on in production!
#DEBUG = True
DEBUG = bool( os.environ.get('DJANGO_DEBUG', True) )
ALLOWED_HOSTS = [
'shrouded-ocean-19461.herokuapp.com', 'localhost', '127.0.0.1',
]
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django_celery_results',
'Buylist',
#'django_q',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'project.urls'
PROJECT_DIR = os.path.dirname(__file__)
TEMPLATES = [
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [
os.path.join(PROJECT_DIR, "templates"),
],
'APP_DIRS': True,
'OPTIONS':
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
,
,
]
WSGI_APPLICATION = 'project.wsgi.application'
# Database
# https://docs.djangoproject.com/en/2.0/ref/settings/#databases
DATABASES =
'default':
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
# Password validation
# https://docs.djangoproject.com/en/2.0/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
,
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
,
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
,
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
,
]
# Internationalization
# https://docs.djangoproject.com/en/2.0/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
# Static files (CSS, javascript, Images)
# https://docs.djangoproject.com/en/2.0/howto/static-files/
# Heroku: Update database configuration from $DATABASE_URL.
db_from_env = dj_database_url.config(conn_max_age=500)
DATABASES['default'].update(db_from_env)
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.10/howto/static-files/
# The absolute path to the directory where collectstatic will collect static files for deployment.
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
# The URL tos use when referring to static files (where they will be served from)
STATIC_URL = '/static/'
# Simplified static file serving.
# https://warehouse.python.org/project/whitenoise/
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
#CELERY_RESULT_BACKEND = 'django-db'
项目/init.py
from __future__ import absolute_import, unicode_literals
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app
__all__ = ['celery_app']
这是我的 procfile 我在 Heroku 端安装了 Heroku Redis 应用程序
web: gunicorn project.wsgi --log-file -
worker: celery worker --app=tasks.app
Procfile 位于主 Git 根目录中。如果您需要更多信息,请告诉我,我可以提供!非常感谢!
【问题讨论】:
那个 .env 文件在 git 中并部署到 Heroku 吗?不应该。 @DanielRoseman 是的,我将它放在 Procfile 所在的根目录中。应该在哪里? 位置正确,但它应该只在您的本地机器上,而不是致力于版本控制或部署。在 Heroku 上,您的应用将从环境中获取值,该环境不会由附加组件填充。 我明白了!谢谢。我会继续删除该设置。 文件已被删除!在生产版本中仍然有同样的问题。我更新了代码以反映 .env 更改 【参考方案1】:单个工人的 procfile 应该是这样的
worker: celery -A <folder containing celery.py> worker -l info
我的代理 url 也没有正确配置。在 heroku 上设置您的代理,并在将其添加到项目后从概览页面中单击它。就我而言,我使用了CloudAMQP - RabbitMQ manager
。单击它后,将显示有关您的经纪人网址的信息。密码、用户名、url 等。你想复制 url,在你的 Django 应用程序的settings.py
或你配置 Celery 配置文件的地方,你想设置你的代理 url。我的在settings.py
中看起来像这样@
CELERY_BROKER_URL = 'amqp://<USER>:<PASSWORD>@chimpanzee.rmq.cloudamqp.com/<USER>'
这是在将以下行放入 celery.py
告诉它在 settings.py
中查找配置之后。
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
app.config_from_object('django.conf:settings', namespace='CELERY')
把它推到heroku,看看能不能解决你的问题。如果您尝试在本地启动 Windows 上的工作程序,则必须在终端中使用 eventlet
选项。
希望这会有所帮助,如果您仍有问题,请告诉我。
【讨论】:
以上是关于Celery / Heroku - 使用 Heroku run python 在后台运行时,延迟()啥也不做的主要内容,如果未能解决你的问题,请参考以下文章
Django/Celery 和 CloudAMQP/Heroku 的连接错误