Gunicorn源码分析

Posted 2022-12-02 程序员小熊

    首先gunicorn是基于“prefork worker”模型，gunicorn启动之后会有一个master进程和多个worker进程，这个master进程的主要功能就是管理worker进程，所有的请求和响应都是由worker进程进行处理，pre-fork和fork模式类似，都是通过一个单独的进程来处理请求，但是不同的是，pre-fork模式会提前开启大量的进程，等待处理收到的请求，所以这个模式能够更快的处理请求，等到所有的进程都在处理请求时，服务器会额外开启其他的新进程。

    首先master进程就是一个循环，用来不断侦听不同进程信号并作出不同的动作，仅此而已。它通过一些信号，诸如TTIN, TTOU, 和CHLD等等， 管理着那些正在运行的worker进程。在gunicorn的arbiter.py中：

class Arbiter(object):
    """
    Arbiter maintain the workers processes alive. It launches or
    kills them if needed. It also manages application reloading
    via SIGHUP/USR2.
    """

    # A flag indicating if a worker failed to
    # to boot. If a worker process exist with
    # this error code, the arbiter will terminate.
    WORKER_BOOT_ERROR = 3

    # A flag indicating if an application failed to be loaded
    APP_LOAD_ERROR = 4

    START_CTX = 

    LISTENERS = []
    WORKERS = 
    PIPE = []

    # I love dynamic languages
    SIG_QUEUE = []
    SIGNALS = [getattr(signal, "SIG%s" % x)
               for x in "HUP QUIT INT TERM TTIN TTOU USR1 USR2 WINCH".split()]
    SIG_NAMES = dict(
        (getattr(signal, name), name[3:].lower()) for name in dir(signal)
        if name[:3] == "SIG" and name[3] != "_"
    )

    def __init__(self, app):
        os.environ["SERVER_SOFTWARE"] = SERVER_SOFTWARE

        self._num_workers = None
        self._last_logged_active_worker_count = None
        self.log = None

        self.setup(app)

        self.pidfile = None
        self.systemd = False
        self.worker_age = 0
        self.reexec_pid = 0
        self.master_pid = 0
        self.master_name = "Master"

        cwd = util.getcwd()

        args = sys.argv[:]
        args.insert(0, sys.executable)

        # init start context
        self.START_CTX = 
            "args": args,
            "cwd": cwd,
            0: sys.executable
        

我们可以从代码中看到许多信号量：

HUP QUIT INT TERM TTIN TTOU USR1 USR2 WINCH

这些信号量的含义是：

• HUP，重启所有的配置和所有的worker进程
• QUIT，正常关闭，它会等待所有worker进程处理完各自的东西后关闭
• INT/TERM，立即关闭，强行中止所有的处理
• TTIN，增加一个worker进程
• TTOU，减少一个worker进程
• USR1，重新打开由master和worker所有的日志处理
• USR2，重新运行master和worker
• WINCH，正常关闭所有worker进程，保持主控master进程的运行

同时，对于不同的信号量来说，gunicorn做了不同的处理：

    def handle_chld(self, sig, frame):
        "SIGCHLD handling"
        self.reap_workers()
        self.wakeup()

    def handle_hup(self):
        """\\
        HUP handling.
        - Reload configuration
        - Start the new worker processes with a new configuration
        - Gracefully shutdown the old worker processes
        """
        self.log.info("Hang up: %s", self.master_name)
        self.reload()

    def handle_term(self):
        "SIGTERM handling"
        raise StopIteration

    def handle_int(self):
        "SIGINT handling"
        self.stop(False)
        raise StopIteration

    def handle_quit(self):
        "SIGQUIT handling"
        self.stop(False)
        raise StopIteration

    def handle_ttin(self):
        """\\
        SIGTTIN handling.
        Increases the number of workers by one.
        """
        self.num_workers += 1
        self.manage_workers()

    def handle_ttou(self):
        """\\
        SIGTTOU handling.
        Decreases the number of workers by one.
        """
        if self.num_workers <= 1:
            return
        self.num_workers -= 1
        self.manage_workers()

    def handle_usr1(self):
        """\\
        SIGUSR1 handling.
        Kill all workers by sending them a SIGUSR1
        """
        self.log.reopen_files()
        self.kill_workers(signal.SIGUSR1)

    def handle_usr2(self):
        """\\
        SIGUSR2 handling.
        Creates a new master/worker set as a slave of the current
        master without affecting old workers. Use this to do live
        deployment with the ability to backout a change.
        """
        self.reexec()

    def handle_winch(self):
        """SIGWINCH handling"""
        if self.cfg.daemon:
            self.log.info("graceful stop of workers")
            self.num_workers = 0
            self.kill_workers(signal.SIGTERM)
        else:
            self.log.debug("SIGWINCH ignored. Not daemonized")