如何使用 django-elasticsearch-dsl 将 django-modeltranslation 创建的列索引为 Elastic？

Posted 2023-02-23

【中文标题】如何使用 django-elasticsearch-dsl 将 django-modeltranslation 创建的列索引为 Elastic？

【英文标题】：How to index columns created by django-modeltranslation to Elastic, using django-elasticsearch-dsl?

【发布时间】：2020-07-22 05:44:10

【问题描述】：

我已经使用django-modeltranslation 翻译了我的模型字段，并使用django-elasticsearch-dsl 实现了搜索。

问题： django-modeltranslation 在数据库中创建翻译字段，而不是在我的模型中，并且搜索仅适用于模型创建的字段。由于 django-elasticsearch-dsl 正在检查模型以重建搜索索引。

当我尝试时：

python3 manage.py search_index --rebuild

我得到错误：

    File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/apps/registry.py", line 122, in populate
    app_config.ready()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/apps.py", line 14, in ready
    self.module.autodiscover()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/__init__.py", line 11, in autodiscover
    autodiscover_modules('documents')
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/utils/module_loading.py", line 47, in autodiscover_modules
    import_module('%s.%s' % (app_config.name, module_to_search))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/barbos/django-symbolsite/symbolgraph/search/documents.py", line 7, in <module>
    class SymbolDocument(Document):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/registries.py", line 65, in register_document
    field_instance = document.to_field(field_name, django_field)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/documents.py", line 142, in to_field
    "to an Elasticsearch field!".format(field_name)
django_elasticsearch_dsl.exceptions.ModelFieldNotMappedError: Cannot convert model field name_ru to an Elasticsearch field!
Oleh-MacSymbol-Pro:symbolgraph barbos$ python3 manage.py search_index --rebuild
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/documents.py", line 138, in to_field
    model_field.__class__](attr=field_name)
KeyError: <class 'modeltranslation.fields.field_factory.<locals>.TranslationFieldSpecific'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/core/management/__init__.py", line 377, in execute
    django.setup()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/apps/registry.py", line 122, in populate
    app_config.ready()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/apps.py", line 14, in ready
    self.module.autodiscover()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/__init__.py", line 11, in autodiscover
    autodiscover_modules('documents')
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/utils/module_loading.py", line 47, in autodiscover_modules
    import_module('%s.%s' % (app_config.name, module_to_search))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/barbos/django-symbolsite/symbolgraph/search/documents.py", line 7, in <module>
    class SymbolDocument(Document):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/registries.py", line 65, in register_document
    field_instance = document.to_field(field_name, django_field)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django_elasticsearch_dsl/documents.py", line 142, in to_field
    "to an Elasticsearch field!".format(field_name)
django_elasticsearch_dsl.exceptions.ModelFieldNotMappedError: Cannot convert model field name_ru to an Elasticsearch field!

Django 3.0.5 django-modeltranslation & django-elasticsearch-dsl - 最新版本 弹性搜索-7.6.2 db.sqlite3（我要迁移到PostgreSQL）

相关的django文件：

#settings.py:
INSTALLED_APPS = [
    'modeltranslation',
    'django_elasticsearch_dsl',
    'search',
    'symbol',
    'category',
    'homepage',
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

ELASTICSEARCH_DSL=
    'default': 
        'hosts': 'localhost:9200'
    ,


---
#translation.py
from modeltranslation.translator import translator, TranslationOptions
from .models import Symbol

class SymbolTranslationOptions(TranslationOptions):
    fields = ('name', 'content', 'meaning', 'tag',)

translator.register(Symbol, SymbolTranslationOptions)

---
#models.py
from django.db import models
from django.db.models.constraints import UniqueConstraint
class Symbol(models.Model): 
    id  = models.IntegerField(primary_key=True) 
    slug = models.SlugField(unique=True) #uniqe
    codepoint = models.CharField(max_length=255, unique=True) #this list should be in separete table
    symbol = models.CharField(max_length=50, unique=True)
    name = models.CharField(max_length=255, unique=True)
    category = models.ForeignKey('Category', on_delete=models.CASCADE)
    subcategory = models.ForeignKey('Subcategory', on_delete=models.CASCADE)
    show_in_category = models.BooleanField()
    popular = models.BooleanField(blank=True, unique=False, default=False)
    tag = models.CharField(max_length=255,blank=True)  #this list should be in separete table
    content = models.TextField(blank=True)
    meaning = models.TextField(blank=True)
    imgslug = models.CharField(unique=False, max_length=255, blank=True) #uniqe
    # name_en = models.CharField(max_length=255, unique=True, null=True)
    # name_ru = models.CharField(max_length=255, unique=True, null=True)
    # tag_en = models.CharField(max_length=255,blank=True, null=True)
    # tag_ru = models.CharField(max_length=255,blank=True, null=True)
    # content_en = models.TextField(blank=True, null=True)
    # content_ru = models.TextField(blank=True, null=True)
    # meaning_en = models.TextField(blank=True, null=True)
    # meaning_ru = models.TextField(blank=True, null=True)

    def __str__(self):
        return self.name

---
#documents.py
from django_elasticsearch_dsl import Document
from django_elasticsearch_dsl.registries import registry
from symbol.models import Symbol


@registry.register_document
class SymbolDocument(Document):
    class Index:
        # Name of the Elasticsearch index
        name = 'symbols'
        # See Elasticsearch Indices API reference for available settings
        settings = 'number_of_shards': 1,
                    'number_of_replicas': 0

    class Django:
        model = Symbol # The model associated with this Document

        # The fields of the model you want to be indexed in Elasticsearch
        fields = [
            'symbol',
            'name',
            'tag',
            'content',
            'meaning',
            'shortcode',
            'codepoint',
            'slug',
            'id',
            #
            # 'name_ru',
            # 'name_uk',
            # 'tag_ru',
            # 'tag_uk',
            # 'content_ru',
            # 'content_uk',
            # 'meaning_ru',
            # 'meaning_uk',
            # 'also_called_ru',
            # 'also_called_uk',
        ]

我发现了一种使用其他语言进行搜索的糟糕方法：

在 INSTALLED APPS 中禁用 django-modeltranslation。 在模型中添加 django-modeltranslation 在 DB 中创建的所有字段。 将这些字段添加到搜索应用程序中的documents.py 以创建Elastic 索引并重新启动django 服务器 检查“$ python3 manage.py makemigrations”是否返回：“未检测到更改”。 $ python3 manage.py search_index --rebuild 还原步骤 3、2、1 并重新启动 django 服务器

我需要将翻译包含到 Elasticsearch 索引中。 作为开发中的菜鸟，我不知道下一步该做什么。

我有两个想法（更像是一个方向），但不知道如何实现：

将 django-elasticsearch-dsl 更改为直接从 DB 索引。 将 django-modeltranslation 更改为能够将它在 DB 中创建的字段添加到模型中。

【问题讨论】：

在 Postgres trigram 搜索中遇到与 GinIndex + Django 模型翻译相同的问题

【参考方案1】：

您可以在Document 类中显式添加字段。喜欢关注

from django_elasticsearch_dsl import TextField

@registry.register_document
class SymbolDocument(Document):
    name_ru = TextField()
    ....

【讨论】：

对于在 trigram 搜索中使用普通 postgres 索引的 Django 的解决方案是什么样的，例如GinIndex 还是 GistIndex？

@Daniel 您找到解决问题的方法了吗？我面临同样的事情

@Yorbjörn 我只在manage.py makemigrations 上遇到了错误，所以我最终创建了一个带有所需索引的手动迁移。如果您愿意提出 SO 问题，我可以分享代码：) 将不胜感激！