如何在aws elastic bean上安装redis和celery

Posted 2023-03-04

【中文标题】如何在aws elastic bean上安装redis和celery

【英文标题】：How to install redis and celery on aws elasticbean

【发布时间】：2016-02-27 06:53:45

【问题描述】：

我正在尝试在 aws elasticbean 上为我的 django 应用程序安装 redis 和 celery。以下是文件夹 .ebextensions/ 中的配置文件

 option_settings:
  "aws:elasticbeanstalk:application:environment":
    DJANGO_SETTINGS_MODULE: "lawCalender.settings"
    PYTHONPATH: "/opt/python/current/app/lawCalender:$PYTHONPATH"
  "aws:elasticbeanstalk:container:python":
    WSGIPath: "lawCalender/wsgi.py"


Resources:
  MyCacheSecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      GroupDescription: "Lock cache down to webserver access only"
      SecurityGroupIngress :
        - IpProtocol : "tcp"
          FromPort :
            Fn::GetOptionSetting:
              OptionName : "CachePort"
              DefaultValue: "6379"
          ToPort :
            Fn::GetOptionSetting:
              OptionName : "CachePort"
              DefaultValue: "6379"
          SourceSecurityGroupName:
            Ref: "AWSEBSecurityGroup"
  MyElastiCache:
    Type: "AWS::ElastiCache::CacheCluster"
    Properties:
      CacheNodeType:
        Fn::GetOptionSetting:
          OptionName : "CacheNodeType"
          DefaultValue : "cache.t1.micro"
      NumCacheNodes:
        Fn::GetOptionSetting:
          OptionName : "NumCacheNodes"
          DefaultValue : "1"
      Engine:
        Fn::GetOptionSetting:
          OptionName : "Engine"
          DefaultValue : "redis"
      VpcSecurityGroupIds:
        -
          Fn::GetAtt:
            - MyCacheSecurityGroup
            - GroupId

Outputs:
  ElastiCache:
    Description : "ID of ElastiCache Cache Cluster with Redis Engine"
    Value :
      Ref : "MyElastiCache"

files:
    "/opt/python/log/django.log":
        mode: "000666"
        owner: ec2-user
        group: ec2-user
        content: |
          # Log file  
          Test Data             
    "/opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
          #!/usr/bin/env bash
          # Get django environment variables
          celeryenv=`cat /opt/python/current/env | tr '\n' ',' | sed 's/%/%%/g' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g'`
            celeryenv=$celeryenv%?

          # Create celery configuraiton script
          celeryconf="[program:celeryd]
          ; Set full path to celery program if using virtualenv
          command=/opt/python/run/venv/bin/celery worker -A lawCalender --loglevel=INFO

          directory=/opt/python/current/app
          user=nobody
          numprocs=1
          stdout_logfile=/var/log/celery-worker.log
          stderr_logfile=/var/log/celery-worker.log
          autostart=true
          autorestart=true
          startsecs=10

          ; Need to wait for currently executing tasks to finish at shutdown.
          ; Increase this if you have very long running tasks.
          stopwaitsecs = 600

          ; When resorting to send SIGKILL to the program to terminate it
          ; send SIGKILL to its whole process group instead,
          ; taking care of its children as well.
          killasgroup=true

          ; if rabbitmq is supervised, set its priority higher
          ; so it starts first
          priority=998

          environment=$celeryenv"

          # Create celerybeat configuraiton script
          celerybeatconf="[program:celerybeat]
          ; Set full path to celery program if using virtualenv
          command=/opt/python/run/venv/bin/celery beat -A lawCalender --loglevel=INFO

          ; remove the -A avtotest argument if you are not using an app instance

          directory=/opt/python/current/app
          user=nobody
          numprocs=1
          stdout_logfile=/var/log/celerybeat.log
          stderr_logfile=/var/log/celerybeat.log
          autostart=true
          autorestart=true
          startsecs=10

          ; Need to wait for currently executing tasks to finish at shutdown.
          ; Increase this if you have very long running tasks.
          stopwaitsecs = 600

          ; When resorting to send SIGKILL to the program to terminate it
          ; send SIGKILL to its whole process group instead,
          ; taking care of its children as well.
          killasgroup=true

          ; if rabbitmq is supervised, set its priority higher
          ; so it starts first
          priority=999

          environment=$celeryenv"

          # Create the celery and beat supervisord conf script
          echo "$celeryconf" | tee /opt/python/etc/celery.conf
          echo "$celerybeatconf" | tee /opt/python/etc/celerybeat.conf

          # Add configuration script to supervisord conf (if not there already)
          if ! grep -Fxq "[include]" /opt/python/etc/supervisord.conf
            then
              echo "[include]" | tee -a /opt/python/etc/supervisord.conf
              echo "files: celery.conf celerybeat.conf" | tee -a /opt/python/etc/supervisord.conf

          fi

          # Reread the supervisord config
          supervisorctl -c /opt/python/etc/supervisord.conf reread

          # Update supervisord in cache without restarting all services
          supervisorctl -c /opt/python/etc/supervisord.conf update

          # Start/Restart celeryd through supervisord
          supervisorctl -c /opt/python/etc/supervisord.conf restart celeryd

由于某种原因，这不起作用。我可以看到所有文件都已创建并位于不同的文件夹中，但我看不到在服务器上安装或运行 redis。 我究竟做错了什么 ？如果您需要任何其他信息，请发表评论。 （我对 aws 和 beanstalk 很陌生。）

【参考方案1】：

Elastic Beanstalk 可以很好地与 Web 服务器 (django) 配合使用，并且很难为其他任何东西（例如 redis）进行配置。我可以建议一种不同的方法吗？使用 ElastiCache 作为您的 redis 主机，使用 SQS 作为您的 celery 代理。 一般来说 - 您可以避开内部设置，将部署简化到极致，让 aws 完成所有工作。这就是它的设计方式。

【讨论】：

sqs 是一种排队服务。我可以把它和芹菜一起用。但它无法取代它。我需要一些东西来处理后台任务。