到 AWS elasticsearch 的异步连接

Posted 2023-03-22

【中文标题】到 AWS elasticsearch 的异步连接

【英文标题】：Async connection to AWS elasticsearch

【发布时间】：2021-12-31 23:19:36

【问题描述】：

我正在努力使用异步连接连接到 AWS elasticsearch。

常规同步连接工作正常：

from requests_aws4auth import AWS4Auth
from elasticsearch import Elasticsearch, RequestsHttpConnection

AWS_ACCESS_KEY_ID = "<id>"
AWS_SECRET_ACCESS_KEY = "<key>"
AWS_REGION = "us-east-1"
awsauth = AWS4Auth(AWS_ACCESS_KEY_ID,
                   AWS_SECRET_ACCESS_KEY,
                   AWS_REGION, 'es')


es = Elasticsearch(
        ['https://url-to-elastic-in-aws/'],
        http_auth=awsauth,
        use_ssl=True,
        verify_certs=True,
        connection_class=RequestsHttpConnection
    )

print(await es.info())

但使用 connection_class=RequestsHttpConnection 或 connection_class=RequestsHttpConnection 的异步连接会引发异常

from requests_aws4auth import AWS4Auth
from elasticsearch import AsyncElasticsearch, AIOHttpConnection

AWS_ACCESS_KEY_ID = "<id>" 
AWS_SECRET_ACCESS_KEY = "<key>" 
AWS_REGION = "us-east-1" 
awsauth = AWS4Auth(AWS_ACCESS_KEY_ID,
                   AWS_SECRET_ACCESS_KEY,
                   AWS_REGION, 'es')


es = AsyncElasticsearch(
        ['https://url-to-elastic-in-aws/'],
        http_auth=awsauth,
        use_ssl=True,
        verify_certs=True,
        connection_class=AIOHttpConnection
    )

print(await es.info())

它是：TypeError: object tuple can't be used in 'await' expression it connection_class=RequestsHttpConnection

或

AttributeError: 'AWS4Auth' 对象没有属性 'encode'

我在互联网上搜索并找不到解决方案。有什么方法可以使用 AsyncElasticsearch 类连接到 AWS elasticsearch。

【参考方案1】：

我相信这是 RequestsHttpConnection 的一个已知限制，并且自去年以来一直存在：https://github.com/elastic/elasticsearch-py/issues/1333#issuecomment-676278567

建议在更新RequestsHttpsConnection 之前不要使用异步elasticsearch/opensearch 客户端。