在 Python Django 中启用调试时，如何仅关闭 SqlAlchemy 数据库池的日志记录？

Posted 2023-03-16

【中文标题】在 Python Django 中启用调试时，如何仅关闭 SqlAlchemy 数据库池的日志记录？

【英文标题】：How to turn off logging for SqlAlchemy db pool only, when debugging is enabled in Python Django?

【发布时间】：2021-06-20 23:09:15

【问题描述】：

上下文/环境

我有一个 Python Django 项目使用 SqlAlchemy django-postgrespool2 数据库池连接到 PostgreSQL 数据库。

我正在努力实现的目标

在settings.py 中，我设置了DEBUG = True，它将生成程序日志。但是，我只想禁用来自 SqlAlchemy 数据库池的日志记录，同时将 DEBUG 设置为 True。换句话说，在调试时，我想查看除来自 SqlAlchemy 数据库池的日志之外的所有日志。

我测试过的内容

下面的代码显示了我为禁用数据库池日志所做的不同尝试：

def turnOffLog(self):
        # --- Attempt 1, db pool logs still show ---
        logging.basicConfig()
        logging.getLogger('sqlalchemy').setLevel(logging.ERROR)

        # --- Attempt 2, db pool logs still show ---
        sqla_logger = logging.getLogger('sqlalchemy.engine.base.Engine')
        for hdlr in sqla_logger.handlers:
            sqla_logger.removeHandler(hdlr)

我还在创建 QueuePool 实例时将 echo=False 传递给它，它仍然不会删除日志：

dbPool = pool.QueuePool(dbConnection, max_overflow=dbConfig.poolMaxOverflow, pool_size=dbConfig.poolSize, recycle=dbConfig.poolRecycleTime, pre_ping=dbConfig.prePing, dialect=postgresql.dialect(), echo=False)

我的settings.py 文件（在 env-config 文件中将 DEBUG 设置为 True）：

"""
myapp
"""

import os
import logging
import socket
import sys
from decouple import Config, RepositoryEnv

import time

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# DECOUPLE
# env_config = Config(RepositoryEnv(BASE_DIR.replace('myapp-gui','')+'.env'))
env_config = Config(RepositoryEnv("c:\\Users\\myteam\\Documents\\django\\ATT49882.env"))

print('Using database:          \x1b[97m[\033[92m '+env_config.get('DB_NAME')+' \x1b[97m]\033[0m');
print('Using database host:     \x1b[97m[\033[92m '+env_config.get('DB_HOST')+' \x1b[97m]\033[0m');
print('Using database port:     \x1b[97m[\033[92m '+env_config.get('DB_PORT')+' \x1b[97m]\033[0m');

# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/2.1/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = env_config.get('SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
# OLD DEBUG = bool(env_config.get('DEBUG'))
try:
    DEBUG = bool(int(env_config.get('DEBUG')))
except Exception as err:
    DEBUG = True 

# LOGLEVEL
if DEBUG:
    LOGLEVEL = "DEBUG"
else:
    LOGLEVEL = "INFO"

ALLOWED_HOSTS = ['*']

# myapp version
if DEBUG:
    myapp_VERSION = time.time()
else:
    myapp_VERSION = 2.0

# myapp constants
myapp_CONSTANTS = 
    'myapp_VERSION': myapp_VERSION


# Application definition

INSTALLED_APPS = [
    ...
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'core.mrank-middleware.MrankMiddleware',
    'core.context_processor.LoginRequiredMiddleware',
]

LOGIN_URL = '/login/'

LOGIN_EXEMPT_URLS = []

REQUEST_WHITELIST_IP = ['127.0.0.1']

ROOT_URLCONF = 'myapp.urls'

CSRF_FAILURE_VIEW = 'core.views.csrf_failure'

# Setting for searching for html files in all templates folders, even in subfolders
TEMPLATE_LOADERS = (
    'django.template.loaders.filesystem.Loader',
    'django.template.loaders.app_directories.Loader',
)

TEMPLATES = [
    
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': 
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'core.context_processor.serverconfig',
                'core.context_processor.phraseconfig',
                'core.context_processor.myapp_constants'
            ],
        ,
    ,
]

WSGI_APPLICATION = 'myapp.wsgi.application'


# Database
# https://docs.djangoproject.com/en/2.1/ref/settings/#databases

# FOR DEVELOPMENT :-)
try:

    hostname = socket.gethostname()
except:
    hostname = ''


# bool(int(env_config.get('DEBUG')))

DATABASES = 
    'default': 
        'ENGINE': 'django.db.backends.postgresql_psycopg2', 
        'NAME': env_config.get('DB_NAME'),
        'OPTIONS': 
            'options': '-c search_path=myappdjango'
        ,
        'USER': env_config.get('DB_USER'),
        'PASSWORD': env_config.get('DB_PASSWORD'),
        'HOST': env_config.get('DB_HOST'),   # Or an IP Address that your DB is hosted on
        'PORT': env_config.get('DB_PORT')
    ,
    'pool': 
        'ENGINE': 'django_postgrespool2', 
        'NAME': env_config.get('DB_NAME'),
        'OPTIONS': 
            'options': '-c search_path=myappdjango'
        ,
        'USER': env_config.get('DB_USER'),
        'PASSWORD': env_config.get('DB_PASSWORD'),
        'HOST': env_config.get('DB_HOST'),
        'PORT': env_config.get('DB_PORT'),
        'POOL_SIZE': int(env_config.get('DB_POOL_SIZE', 100)),
        'POOL_MAX_OVERFLOW': int(env_config.get('DB_POOL_MAX_OVERFLOW', 100)),
        'POOL_RECYCLE_TIME': int(env_config.get('DB_POOL_RECYCLE_TIME', 3600)),
        "PRE_PING": bool(int(env_config.get('DB_POOL_PRE_PING', 0)))
    


# Password validation
# https://docs.djangoproject.com/en/2.1/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    ...
]


# Internationalization
# https://docs.djangoproject.com/en/2.1/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, javascript, Images)
# https://docs.djangoproject.com/en/2.1/howto/static-files/

STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

STATIC_URL = '/static/'

STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')

# Upload directory
MEDIA_URL = '/upload/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'upload')

# Custom settings
TIME = 60*60*60*60
SESSION_COOKIE_AGE = TIME
SESSION_IDLE_TIMEOUT = TIME
SESSION_EXPIRE_AT_BROWSER_CLOSE = True

LOGGING = 
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': 
        'default': 
            'format': '%(asctime)s:%(levelname)s:%(name)s: %(message)s',
            'datefmt': '%Y-%m-%d %H:%M:%S'
        
    ,
    'handlers': 
        'console': 
            'class': 'logging.StreamHandler',
            'formatter': 'default',
            'stream': sys.stdout
        ,
    ,
    'root': 
        'handlers': ['console'],
        'level': LOGLEVEL,
    ,

数据库池日志示例

只是我要删除的日志外观示例：

2021-03-24 09:38:00:DEBUG:z.pool: new connection
2021-03-24 09:38:00:DEBUG:z.pool: retrieved from pool
2021-03-24 09:38:00:DEBUG:z.pool: returned to pool

谢谢！

更新 #0

我也试过了，但它仍然没有删除数据库池日志：

logging.basicConfig()
logging.getLogger('sqlalchemy').disable = True

【问题讨论】：

您是否也尝试过sqlalchemy.pool 作为记录器名称？

【参考方案1】：

通过设置 engine.echo = False where engine = create_engine()

解决了这个问题

似乎 sqlalchemy 创建了自己的日志输出：

请务必注意，这两个标志独立于任何现有的日志记录配置工作，并且将无条件地使用 logging.basicConfig()。除了任何现有的记录器配置之外，这还具有配置的效果。因此，在显式配置日志记录时，请确保所有回显标志始终设置为 False，以避免出现重复的日志行。

您应该将 engine.echo 和/或 engine.echo_pool 设置为 False (engine = create_engine())。

在我的情况下（几乎与这个相同）engine.echo = False 解决了问题，并且未找到 engine.echo_pool。

这里有更多描述：https://docs.sqlalchemy.org/en/14/core/engines.html#more-on-the-echo-flag