如何将 bigquery 返回的结果转换为 Json

Posted 2023-03-24

【中文标题】如何将 bigquery 返回的结果转换为 Json

【英文标题】：How to convert results returned from bigquery to Json

【发布时间】：2021-08-06 10:25:03

【问题描述】：

目前使用python从bigquery中拉取数据，得到结果后需要转成JSON格式。如何将结果转成JSON？

    Query = """
with data AS (
SELECT
end_time,
total_bytes_processed,
query
FROM
`project-id.region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT`
WHERE  date(end_time) = '2021-07-28'
GROUP BY
end_time,
job_id,
total_bytes_processed,
query
ORDER BY total_bytes_processed DESC
)
select as value
array_agg(struct( end_time,
regexp_extract(query, r'(?i)\sfrom\s+`?(?:[\w-]+\.)*([\w-]+\.[\w-]+)`?\s' ) as table,
(total_bytes_processed/1099511627776) * 5 as cost, query) 
order by (total_bytes_processed/1099511627776) * 5  desc limit 1)[offset(0)]
from data
group by timestamp_trunc(end_time, minute)
 """

我找到了这个To JSON STRING FUNCTION，但我还没有弄清楚如何在这个语句中实现它。非常感谢帮助或提示如何从 BigQuery 获得理想结果

【参考方案1】：

我可以使用 2 with 子句来做到这一点，如下所示

with data AS (
SELECT
end_time,
total_bytes_processed,
query
FROM
region-us.INFORMATION_SCHEMA.JOBS_BY_PROJECT
WHERE  date(end_time) = '2021-07-28'
GROUP BY
end_time,
job_id,
total_bytes_processed,
query
ORDER BY total_bytes_processed DESC
),
DATA1 AS (
select as value
array_agg(struct( end_time,
regexp_extract(query, r'(?i)\sfrom\s+`?(?:[\w-]+\.)*([\w-]+\.[\w-]+)`?\s' ) as table,
(total_bytes_processed/1099511627776) * 5 as cost, query) 
order by (total_bytes_processed/1099511627776) * 5  desc limit 1)[offset(0)]
from data as t
group by timestamp_trunc(end_time, minute)
)
select TO_JSON_STRING(t,true) as data
from DATA1 AS t

这是我得到的结果