python bigquery 库 DB-API 接口如何支持 WHERE IN 或 WHERE ANY 子句

Posted 2023-03-25

【中文标题】python bigquery 库 DB-API 接口如何支持 WHERE IN 或 WHERE ANY 子句

【英文标题】：How python bigquery library DB-API interface supports WHERE IN or WHERE ANY clause

【发布时间】：2019-09-05 09:33:09

【问题描述】：

我正在使用 python bigquery 库https://googleapis.dev/python/bigquery/latest/index.html 的 DB-API 接口。当我将参数传递给 WHERE IN 或 WHERE ANY 子句的 Cursor.execute() 时，它会引发如下错误

google-cloud-bigquery 版本：1.19.0

from google.cloud import bigquery

from google.cloud.bigquery import dbapi

client = bigquery.Client()
conn = dbapi.Connection(client)
curr = conn.cursor()

query = """
  SELECT name, state
  FROM `bigquery-public-data.usa_names.usa_1910_2013`
  WHERE state = %s
  LIMIT 2
"""
curr.execute(query, ('NY', ))
result = curr.fetchall()
print(result)

query = """
  SELECT name, state
  FROM `bigquery-public-data.usa_names.usa_1910_2013`
  WHERE state IN %s
  LIMIT 2
"""
curr.execute(query, (('NY', 'TX'), ))
result = curr.fetchall()
print(result)

输出

[Row(('Mildred', 'NY'), 'name': 0, 'state': 1), Row(('Irene', 'NY'), 'name': 0, 'state': 1)]
Traceback (most recent call last):
  File "hello_bq.py", line 25, in <module>
    curr.execute(query, (('NY', 'TX'), ))
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/cursor.py", line 159, in execute
    query_parameters = _helpers.to_query_parameters(parameters)
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 117, in to_query_parameters
    return to_query_parameters_list(parameters)
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 84, in to_query_parameters_list
    return [scalar_to_query_parameter(value) for value in parameters]
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 84, in <listcomp>
    return [scalar_to_query_parameter(value) for value in parameters]
  File "/home/haibin/.local/share/virtualenvs/python-6nCS1ipk/lib/python3.6/site-packages/google/cloud/bigquery/dbapi/_helpers.py", line 69, in scalar_to_query_parameter
    name, value
google.cloud.bigquery.dbapi.exceptions.ProgrammingError: encountered parameter None with value ('NY', 'TX') of unexpected type

感谢任何帮助。

【问题讨论】：

顺便说一句，按数组元素过滤应使用IN UNNEST(...) 构造（BigQuery docs）。话虽如此，BigQuery 1.19.0 确实缺乏对类数组参数的支持，但很可能会在下一个版本中添加（我刚刚为此开了一个 PR）。

【参考方案1】：

我在parameter conversion code 上看不到对数组类型的支持，但这种替代方法有效：

from google.cloud import bigquery

from google.cloud.bigquery import dbapi

client = bigquery.Client()
conn = dbapi.Connection(client)
curr = conn.cursor()

query = """
  SELECT name, state
  FROM `bigquery-public-data.usa_names.usa_1910_2013`
  WHERE state IN UNNEST(SPLIT(%s))
  LIMIT 2
"""
curr.execute(query, ('NY,TX', ))
result = curr.fetchall()
print(result)

有一个开放的 GitHub 问题来跟踪本机支持的进度：

https://github.com/googleapis/google-cloud-python/issues/9177

【讨论】：

FWIW，我打开了一个 PR 以添加对类似数组的参数的支持，而 BigQuery 1.20.0+ 中可能不再需要解决方法。