Gunicorn + Django + Virtualenv + init.d 服务(CentOS)

Posted

技术标签:

【中文标题】Gunicorn + Django + Virtualenv + init.d 服务(CentOS)【英文标题】:Gunicorn + Django + Virtualenv + init.d service (CentOS) 【发布时间】:2015-06-21 15:10:15 【问题描述】:

详情:

OS = CentOS 6.6
python = 2.7.9
virtualenv = 12.1.1
gunicorn = 19.3.0
django-admin.py = 1.6.10
geonode = 2.4

我正在尝试使用 Gunicorn 在 CentOS 6.6 上添加一个 django 应用程序(geonode,http://geonode.org/)作为 init.d 服务。任何帮助将不胜感激。

CentOS 6.6 的默认 python 是 2.6 版本,所以我安装了 python 2.7.9 作为备用版本:

sudo wget https://www.python.org/ftp/python/2.7.9/Python-2.7.9.tgz
sudo tar -xzf Python-2.7.9.tgz
cd Python-2.7.9
sudo ./configure --prefix=/usr/local --enable-unicode=ucs4 --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"
sudo make
sudo make altinstall

已安装 setuptools、pip 和 virtualenv:

su -
wget https://pypi.python.org/packages/source/s/setuptools/setuptools-15.0.tar.gz
tar -xvf setuptools-15.0.tar.gz
cd setuptools-15.0
python2.7 setup.py install
cd ..
rm -fr setuptools-15.0*
curl https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py | python2.7 -
pip2.7 install virtualenv

将运行 django 应用程序的已配置服务帐户:

sudo mkdir -p /webapps/geonode_django/
sudo groupadd --system webapps
sudo useradd --system --gid webapps --shell /bin/bash --home /webapps/geonode_django geonode
sudo chown geonode /webapps/geonode_django/

设置虚拟环境并安装python依赖:

su -
su - geonode
virtualenv -p /usr/local/bin/python2.7 .
source bin/activate
export PYCURL_SSL_LIBRARY=nss
pip install pycurl
pip install urlgrabber
pip install numpy
pip install pillow
pip install pastescript
pip install psycopg2
pip install gunicorn
wget https://pypi.python.org/packages/source/G/GDAL/GDAL-1.11.2.tar.gz
tar -xvf GDAL-1.11.2.tar.gz
cd GDAL-1.11.2
python setup.py build_ext --include-dirs=/opt/gdal-1.11.2/include/
python setup.py install
cd ..
rm -fr GDAL-1.11.2*

克隆并安装了 django 应用程序:

git clone https://github.com/GeoNode/geonode.git
cd geonode
pip install -e . --log install-geonode.log

执行了所需的 geonode(django 应用程序)命令

su - geonode
source bin/activate
cd geonode
python manage.py createsuperuser
python manage.py collectstatic

创建了一个脚本来运行 gunicorn:

mkdir /webapps/geonode_django/scripts
vim /webapps/geonode_django/scripts/gunicorn-app.sh

gunicorn-app.sh 的开始

#!/bin/bash
set -e
cd /webapps/geonode_django/geonode
source /webapps/geonode_django/bin/activate
exec /webapps/geonode_django/bin/gunicorn geonode.wsgi:application \
--workers=2 \
--bind=0.0.0.0:8000 \
--user=geonode --group=webapps --log-level=debug \
--log-file=/webapps/geonode_django/logs/gunicorn.log 2>>/webapps/geonode_django/logs/gunicorn.log

gunicorn-app.sh 结束 注意:脚本基于此处找到的一个 - https://gist.githubusercontent.com/cspanring/4639342/raw/a44ff78aec9e1919a9e4c25886de331e787201d2/gunicorn-app.sh

mkdir /webapps/geonode_django/logs
chmod +x /webapps/geonode_django/scripts/gunicorn-app.sh

注意:我可以以 root 和 geonode 身份成功运行脚本并显示启动页面,但是当我将其添加到 init.d 脚本并作为“服务”运行时,结果如下:

[root@geonode_test ~]# curl localhost:8000
<html>
   <head>
     <title>Internal Server Error</title>
   </head>
   <body>
     <h1><p>Internal Server Error</p></h1>

   </body>
</html>

gunicorn.log 的输出

[2015-04-15 06:48:42 +0000] [6983] [DEBUG] Current configuration:
  proxy_protocol: False
  worker_connections: 1000
  statsd_host: None
  max_requests_jitter: 0
  post_fork: <function post_fork at 0x7f685ecd02a8>
  pythonpath: None
  enable_stdio_inheritance: False
  worker_class: sync
  ssl_version: 3
  suppress_ragged_eofs: True
  syslog: False
  syslog_facility: user
  when_ready: <function when_ready at 0x7f685ed46f50>
  pre_fork: <function pre_fork at 0x7f685ecd0140>
  cert_reqs: 0
  preload_app: False
  keepalive: 2
  accesslog: None
  group: 493
  graceful_timeout: 30
  do_handshake_on_connect: False
  spew: False
  workers: 2
  proc_name: None
  sendfile: True
  pidfile: None
  umask: 0
  on_reload: <function on_reload at 0x7f685ed46de8>
  pre_exec: <function pre_exec at 0x7f685ecd0848>
  worker_tmp_dir: None
  post_worker_init: <function post_worker_init at 0x7f685ecd0410>
  limit_request_fields: 100
  on_exit: <function on_exit at 0x7f685ecd0ed8>
  config: None
  secure_scheme_headers: 'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'
  proxy_allow_ips: ['127.0.0.1']
  pre_request: <function pre_request at 0x7f685ecd09b0>
  post_request: <function post_request at 0x7f685ecd0aa0>
  user: 496
  forwarded_allow_ips: ['127.0.0.1']
  worker_int: <function worker_int at 0x7f685ecd0578>
  threads: 1
  max_requests: 0
  limit_request_line: 4094
  access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
  certfile: None
  worker_exit: <function worker_exit at 0x7f685ecd0c08>
  chdir: /webapps/geonode_django/geonode
  paste: None
  default_proc_name: geonode.wsgi:application
  errorlog: /webapps/geonode_django/logs/gunicorn.log
  loglevel: debug
  logconfig: None
  syslog_addr: udp://localhost:514
  syslog_prefix: None
  daemon: False
  ciphers: TLSv1
  on_starting: <function on_starting at 0x7f685ed46c80>
  worker_abort: <function worker_abort at 0x7f685ecd06e0>
  bind: ['0.0.0.0:8000']
  raw_env: []
  reload: False
  check_config: False
  limit_request_field_size: 8190
  nworkers_changed: <function nworkers_changed at 0x7f685ecd0d70>
  timeout: 30
  ca_certs: None
  django_settings: None
  tmp_upload_dir: None
  keyfile: None
  backlog: 2048
  logger_class: gunicorn.glogging.Logger
  statsd_prefix:
[2015-04-15 06:48:42 +0000] [6983] [INFO] Starting gunicorn 19.3.0
[2015-04-15 06:48:42 +0000] [6983] [DEBUG] Arbiter booted
[2015-04-15 06:48:42 +0000] [6983] [INFO] Listening at: http://0.0.0.0:8000 (6983)
[2015-04-15 06:48:42 +0000] [6983] [INFO] Using worker: sync
[2015-04-15 06:48:42 +0000] [6992] [INFO] Booting worker with pid: 6992
[2015-04-15 06:48:42 +0000] [6993] [INFO] Booting worker with pid: 6993
[2015-04-15 06:48:42 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:43 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:44 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:45 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:46 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:47 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:48 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:49 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:50 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:51 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:51 +0000] [6983] [DEBUG] 2 workers
[2015-04-15 06:48:51 +0000] [6983] [INFO] Handling signal: int
[2015-04-15 06:48:51 +0000] [6992] [INFO] Worker exiting (pid: 6992)
[2015-04-15 06:48:51 +0000] [6993] [INFO] Worker exiting (pid: 6993)

ps aux 的输出 | grep gunicorn 作为服务运行时 gunicorn start

[root@geonode_test ~]# ps aux | grep gunicorn
root       7551  0.0  0.0 106364  1588 pts/1    S+   07:42   0:00 /bin/sh /sbin/service gunicorn start
root       7556  0.0  0.0 108468  1688 pts/1    S+   07:42   0:00 /bin/sh /etc/init.d/gunicorn start
root       7559  0.0  0.1 163384  1976 pts/1    S+   07:42   0:00 /bin/su   geonode /webapps/geonode_django/config/gunicorn-app.sh
geonode    7560  1.0  0.7 205892 13572 pts/1    S+   07:42   0:00 /webapps/geonode_django/bin/python2.7 /webapps/geonode_django/bin/gunicorn  geonode.wsgi:application --workers=2 --bind=0.0.0.0:8000 --user=geonode --group=webapps --log-level=debug --log-file=/webapps/geonode_django/logs/gunicorn.log
geonode    7569  0.3  0.9 219660 18732 pts/1    S+   07:42   0:00 /webapps/geonode_django/bin/python2.7 /webapps/geonode_django/bin/gunicorn geonode.wsgi:application --workers=2 --bind=0.0.0.0:8000 --user=geonode --group=webapps --log-level=debug --log-file=/webapps/geonode_django/logs/gunicorn.log
geonode    7570  2.1  2.3 450024 44512 pts/1    S+   07:42   0:00 /webapps/geonode_django/bin/python2.7 /webapps/geonode_django/bin/gunicorn geonode.wsgi:application --workers=2 --bind=0.0.0.0:8000 --user=geonode --group=webapps --log-level=debug --log-file=/webapps/geonode_django/logs/gunicorn.log
root       7576  0.0  0.0 103256   856 pts/3    S+   07:42   0:00 grep gunicorn

注意:这有效,只是不作为服务 ps aux 的输出 | grep gunicorn 作为 /etc/init.d/gunicorn start 运行时

[root@geonode_test ~]# ps aux | grep gunicorn
root       7647  0.0  0.0 106368  1596 pts/1    S+   07:47   0:00 /bin/sh /etc/init.d/gunicorn start
root       7650  0.0  0.1 163384  1980 pts/1    S+   07:47   0:00 /bin/su geonode /webapps/geonode_django/config/gunicorn-app.sh
geonode    7651  1.3  0.7 205960 13584 pts/1    S+   07:47   0:00 /webapps/geonode_django/bin/python2.7 /webapps/geonode_django/bin/gunicorn geonode.wsgi:application --workers=2 --bind=0.0.0.0:8000 --user=geonode --group=webapps --log-level=debug --log-file=/webapps/geonode_django/logs/gunicorn.log
geonode    7660  0.4  0.9 219664 18740 pts/1    S+   07:47   0:00 /webapps/geonode_django/bin/python2.7 /webapps/geonode_django/bin/gunicorn geonode.wsgi:application --workers=2 --bind=0.0.0.0:8000 --user=geonode --group=webapps --log-level=debug --log-file=/webapps/geonode_django/logs/gunicorn.log
geonode    7661 10.1  4.4 414248 84380 pts/1    S+   07:47   0:00 /webapps/geonode_django/bin/python2.7 /webapps/geonode_django/bin/gunicorn geonode.wsgi:application --workers=2 --bind=0.0.0.0:8000 --user=geonode --group=webapps --log-level=debug --log-file=/webapps/geonode_django/logs/gunicorn.log
root       7679  0.0  0.0 103256   856 pts/3    S+   07:47   0:00 grep gunicorn

日志输出:

[2015-04-15 07:47:27 +0000] [7651] [DEBUG] Current configuration:
  proxy_protocol: False
  worker_connections: 1000
  statsd_host: None
  max_requests_jitter: 0
  post_fork: <function post_fork at 0x7f390e0b72a8>
  pythonpath: None
  enable_stdio_inheritance: False
  worker_class: sync
  ssl_version: 3
  suppress_ragged_eofs: True
  syslog: False
  syslog_facility: user
  when_ready: <function when_ready at 0x7f390e12cf50>
  pre_fork: <function pre_fork at 0x7f390e0b7140>
  cert_reqs: 0
  preload_app: False
  keepalive: 2
  accesslog: None
  group: 493
  graceful_timeout: 30
  do_handshake_on_connect: False
  spew: False
  workers: 2
  proc_name: None
  sendfile: True
  pidfile: None
  umask: 0
  on_reload: <function on_reload at 0x7f390e12cde8>
  pre_exec: <function pre_exec at 0x7f390e0b7848>
  worker_tmp_dir: None
  post_worker_init: <function post_worker_init at 0x7f390e0b7410>
  limit_request_fields: 100
  on_exit: <function on_exit at 0x7f390e0b7ed8>
  config: None
  secure_scheme_headers: 'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'
  proxy_allow_ips: ['127.0.0.1']
  pre_request: <function pre_request at 0x7f390e0b79b0>
  post_request: <function post_request at 0x7f390e0b7aa0>
  user: 496
  forwarded_allow_ips: ['127.0.0.1']
  worker_int: <function worker_int at 0x7f390e0b7578>
  threads: 1
  max_requests: 0
  limit_request_line: 4094
  access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
  certfile: None
  worker_exit: <function worker_exit at 0x7f390e0b7c08>
  chdir: /webapps/geonode_django/geonode
  paste: None
  default_proc_name: geonode.wsgi:application
  errorlog: /webapps/geonode_django/logs/gunicorn.log
  loglevel: debug
  logconfig: None
  syslog_addr: udp://localhost:514
  syslog_prefix: None
  daemon: False
  ciphers: TLSv1
  on_starting: <function on_starting at 0x7f390e12cc80>
  worker_abort: <function worker_abort at 0x7f390e0b76e0>
  bind: ['0.0.0.0:8000']
  raw_env: []
  reload: False
  check_config: False
  limit_request_field_size: 8190
  nworkers_changed: <function nworkers_changed at 0x7f390e0b7d70>
  timeout: 30
  ca_certs: None
  django_settings: None
  tmp_upload_dir: None
  keyfile: None
  backlog: 2048
  logger_class: gunicorn.glogging.Logger
  statsd_prefix:
[2015-04-15 07:47:27 +0000] [7651] [INFO] Starting gunicorn 19.3.0
[2015-04-15 07:47:27 +0000] [7651] [DEBUG] Arbiter booted
[2015-04-15 07:47:27 +0000] [7651] [INFO] Listening at: http://0.0.0.0:8000 (7651)
[2015-04-15 07:47:27 +0000] [7651] [INFO] Using worker: sync
[2015-04-15 07:47:27 +0000] [7660] [INFO] Booting worker with pid: 7660
[2015-04-15 07:47:27 +0000] [7661] [INFO] Booting worker with pid: 7661
[2015-04-15 07:47:27 +0000] [7651] [DEBUG] 2 workers

我还在另一个 django 测试应用程序上对其进行了测试,并且 init 服务确实有效,所以我很茫然:(

/webapps/geonode_django/geonode/test.py

import os
import pprint
from wsgiref.validate import validator
import sys

from gunicorn import __version__
#@validator
def application(environ, start_response):
    """Simplest possible application object"""

    errors = environ['wsgi.errors']
#    pprint.pprint(('ENVIRON', environ), stream=errors)

    data = b'Hello, World!\n'
    status = '200 OK'

    response_headers = [
        ('Content-type', 'text/plain'),
        ('Content-Length', str(len(data))),
        ('X-Gunicorn-Version', __version__),
        ("Test", "test тест"),
    ]
    start_response(status, response_headers)
    return iter([data])

修改/webapps/geonode_django/scripts/gunicorn-app.sh

exec /webapps/geonode_django/bin/gunicorn geonode.wsgi:application \exec /webapps/geonode_django/bin/gunicorn test:application \

【问题讨论】:

注意:selinux没有阻塞,netstat显示在/etc/init.d/gunicorn start和service gunicorn start运行时正在监听8000端口 【参考方案1】:

Gunicorn 的日志不是最有帮助的,出于沮丧,我尝试了 uwsgi,它立即显示了问题:

Traceback (most recent call last):
  File "/webapps/geonode_django/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 187, in __call__
    self.load_middleware()
  File "/webapps/geonode_django/lib/python2.7/site-packages/django/core/handlers/base.py", line 47, in load_middleware
    mw_instance = mw_class()
  File "/webapps/geonode_django/lib/python2.7/site-packages/django/middleware/locale.py", line 24, in __init__
    for url_pattern in get_resolver(None).url_patterns:
  File "/webapps/geonode_django/lib/python2.7/site-packages/django/core/urlresolvers.py", line 365, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/webapps/geonode_django/lib/python2.7/site-packages/django/core/urlresolvers.py", line 360, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/webapps/geonode_django/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
    __import__(name)
  File "./geonode/urls.py", line 24, in <module>
    from geonode.sitemap import LayerSitemap, MapSitemap
  File "./geonode/sitemap.py", line 21, in <module>
    from geonode.maps.models import Layer, Map
  File "./geonode/maps/models.py", line 35, in <module>
    from geonode.layers.models import Layer
  File "./geonode/layers/models.py", line 33, in <module>
    from geonode.base.models import ResourceBase, ResourceBaseManager, resourcebase_post_save
  File "./geonode/base/models.py", line 28, in <module>
    from geonode.utils import bbox_to_wkt
  File "./geonode/utils.py", line 29, in <module>
    from osgeo import ogr
  File "/webapps/geonode_django/lib/python2.7/site-packages/GDAL-1.11.2-py2.7-linux-x86_64.egg/osgeo/__init__.py", line 21, in <module>
    _gdal = swig_import_helper()
  File "/webapps/geonode_django/lib/python2.7/site-packages/GDAL-1.11.2-py2.7-linux-x86_64.egg/osgeo/__init__.py", line 17, in swig_import_helper
    _mod = imp.load_module('_gdal', fp, pathname, description)
ImportError: /webapps/geonode_django/lib/python2.7/site-packages/GDAL-1.11.2-py2.7-linux-x86_64.egg/osgeo/_gdal.so: undefined symbol: GDALRasterBandGetVirtualMem

终于!!好的...所以 gunicorn-app.sh 需要一些变量,所以我添加了以下内容:

export GDAL_HOME=/opt/gdal-1.11.2
export GDAL_DATA=$GDAL_HOME/data
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$GDAL_HOME/lib
export PATH=$GDAL_HOME/bin:/usr/pgsql-9.3/bin:$PATH

使用 service gunicorn start 对其进行了测试...并且可以正常工作。好吧,希望这对其他人有所帮助。

【讨论】:

以上是关于Gunicorn + Django + Virtualenv + init.d 服务(CentOS)的主要内容,如果未能解决你的问题,请参考以下文章

使用 Gunicorn 和 nginx 部署 Django 项目

使用 Gunicorn 运行 Django - 最佳实践

django-gunicorn-nginx:502 网关错误

django + virtualenv + gunicorn - 没有名为 django.core.wsgi 的模块?

如何让 Django 使用 Gunicorn 提供静态文件?

Gunicorn + Django + Virtualenv + init.d 服务(CentOS)