Django 和只读数据库连接

Posted 2023-02-24

【中文标题】Django 和只读数据库连接

【英文标题】：Django and read-only database connections

【发布时间】：2016-12-09 18:08:26

【问题描述】：

假设一个 Django 应用程序应该使用两个 mysql 数据库：

default - 用于存储由模型 A 和 B 表示的数据（读写访问） support - 用于导入由模型 C 和 D 表示的数据（只读访问）

support 数据库是外部应用程序的一部分，不能修改。

由于 Django 应用程序对模型 A 和 B 使用内置 ORM，我认为它应该对模型 C 和 D 使用相同的 ORM，即使它们映射到外部表数据库 (support.)

为了实现这一点，我将模型 C 和 D 定义如下：

from django.db import models


class ExternalModel(models.Model):
    class Meta:
        managed = False
        abstract = True


class ModelC(ExternalModel):
    some_field = models.TextField(db_column='some_field')

    class Meta(ExternalModel.Meta):
        db_table = 'some_table_c'


class ModelD(ExternalModel):
    some_other_field = models.TextField(db_column='some_other_field')

    class Meta(ExternalModel.Meta):
        db_table = 'some_table_d'

然后我定义了一个数据库路由器：

from myapp.myapp.models import ExternalModel


class DatabaseRouter(object):
    def db_for_read(self, model, **hints):
        if issubclass(model, ExternalModel):
            return 'support'

        return 'default'

    def db_for_write(self, model, **hints):
        if issubclass(model, ExternalModel):
            return None

        return 'default'

    def allow_relation(self, obj1, obj2, **hints):
        return (isinstance(obj1, ExternalModel) == isinstance(obj2, ExternalModel))

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        return (db == 'default')

最后调整了settings.py：

# (...)

DATABASES = 
    'default': 
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': 
            'read_default_file': os.path.join(BASE_DIR, 'resources', 'default.cnf'),
        ,
    ,
    'support': 
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': 
            'read_default_file': os.path.join(BASE_DIR, 'resources', 'support.cnf'),
        ,
    ,


DATABASE_ROUTERS = ['myapp.database_router.DatabaseRouter']

# (...)

support.conf 中为 support 数据库指定的用户已被分配只读权限。

但是当我运行 python manage.py makemigrations 时，它会失败并显示以下输出：

    Traceback (most recent call last):
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/backends/mysql/base.py", line 112, in execute
    return self.cursor.execute(query, args)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
_mysql_exceptions.OperationalError: (1142, "CREATE command denied to user 'somedbuser'@'somehost' for table 'django_migrations'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/migrations/recorder.py", line 57, in ensure_schema
    editor.create_model(self.Migration)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/backends/base/schema.py", line 295, in create_model
    self.execute(sql, params or None)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/backends/base/schema.py", line 112, in execute
    cursor.execute(sql, params)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/backends/mysql/base.py", line 112, in execute
    return self.cursor.execute(query, args)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
django.db.utils.OperationalError: (1142, "CREATE command denied to user 'somedbuser'@'somehost' for table 'django_migrations'")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/core/management/base.py", line 305, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/core/management/base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/core/management/commands/makemigrations.py", line 100, in handle
    loader.check_consistent_history(connection)
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/migrations/loader.py", line 276, in check_consistent_history
    applied = recorder.applied_migrations()
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations
    self.ensure_schema()
  File "/Users/username/Development/stuff/myapp/lib/python3.5/site-packages/django/db/migrations/recorder.py", line 59, in ensure_schema
    raise MigrationSchemaMissing("Unable to create the django_migrations table (%s)" % exc)
django.db.migrations.exceptions.MigrationSchemaMissing: Unable to create the django_migrations table ((1142, "CREATE command denied to user 'somedbuser'@'somehost' for table 'django_migrations'"))

似乎 Django 尝试在只读数据库 support 中创建 django_migrations 表。

是否有任何干净的方法可以防止迁移机制尝试这样做？还是我必须使用另一个 ORM 库才能对 support 数据库进行只读访问？

【问题讨论】：

记录在案——我还没有找到任何解决方案。决定使用peewee 并在没有 Django 的情况下进行只读访问。

【参考方案1】：

我遇到了同样的问题（使用 Django 1.11），这个问题在我的谷歌搜索结果中排在首位。

您最初的解决方案只缺少一个关键部分。你需要告诉 Django 'C' 和 'D' 正在使用什么数据库模型。什么对我有用：

class ExternalModel(models.Model):
    class Meta:
        managed = False
        abstract = True    
        app_label = 'support'

然后在 allow_migrate() 部分告诉您的数据库路由器在遇到该 app_label 时的行为：

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if app_label == 'support':
            return False
        return (db == 'default')

我不确定这是 Django 团队眼中最正确的解决方案，但效果是 allow_migrate() 对于使用该 app_label 属性值定义的任何模型返回 False。

Django documentation on routers 没有明确提及这一点（或者，至少在模型代码示例中清楚地说明了 ORM 如何将 'db' 的值传递给 allow_migrate()），但在 'app_label' 和'托管'属性，你可以让它工作*。

* 在我的例子中，默认是 postgres，只读数据库是通过 cx_Oracle 的 Oracle 12。

【参考方案2】：

似乎在 Django 1.10.1 时间范围内，Tim Graham（主要的 Django 维护者）接受了一个抑制此特定异常的补丁，但后来撤回了该补丁，转而（大致）采用以下方法来解决此问题，并且使用 Django ORM 支持只读数据库。

定义一个数据库路由器，如Django documentation on routers 中所述，我在下面附加了一个示例路由器，该路由器路由到 基于模型元数据中的“app”标志的不同数据库。

在您的路由器 allow_migrations 方法中，为任何 db 参数返回 False 对应于只读数据库。这样可以防止迁移 模型表，无论它们将被路由到哪里。

接下来的部分有点奇怪，但是橡胶碰到了路面， 实际上回答了原来的问题。为了防止 makemigrations 试图在你的只读文件中创建 django_migrations 表 数据库，不应路由数据库流量。在示例中 路由器，这意味着“只读”在 DATABASE_APPS_MAPPING 中不是。

因此，只读数据库是通过“使用”显式访问的（例如 MyReadOnlyModel.objects.using('read_only').all()

Django database apps router

【讨论】：

这是一个非常方便的提示（迟来的——在我经历了所有测试代码的痛苦之后，找出了只读问题发生在哪里！）。我已经完成了第 1 步和第 2 步，但还没有想过要尝试第 3 步和第 4 步。但是，我修改了我的测试运行程序以使用一些覆盖的 util 函数，这些函数删除了在其测试设置中设置了 CREATE_DB=False 的数据库（这些只读数据库恰好是 Oracle）。到目前为止，一切都恢复正常了......

您能否将您的路由器代码作为粘贴箱或要点或博客链接分享？

【参考方案3】：

遇到了同样的问题。 Django 正在尝试在所有数据库中创建“django_migrations”表。 即使没有与只读数据库关联的模型也会发生这种情况 并且所有路由器都指向不同的数据库。

我也最终使用了 peewee。