BigQuery 如何获取 JSON 结构中的值的总和？

Posted 2023-03-25

【中文标题】BigQuery 如何获取 JSON 结构中的值的总和？

【英文标题】：BigQuery How to get the sum of values in a JSON struct?

【发布时间】：2017-11-30 16:36:56

【问题描述】：

我有以下查询

SELECT 
    JSON_EXTRACT(json, '$.Weights') as weight 
from 
(select '"Weights":"blue":1.0,"purple":0.0,"yellow":1.0,"green":1.0' as json)

返回：

"蓝色":1.0,"紫色":0.0,"黄色":1.0,"绿色":1.0

我想看看是否有办法总结所有颜色的值。返回的意思：

3.0

我一直在尝试使用 split 和 unnest 函数但没有成功，有什么建议吗？谢谢。

【参考方案1】：

这里是一个使用REGEXP_EXTRACT_ALL的例子：

WITH T AS (
  SELECT '"Weights":"blue":1.0,"purple":0.0,"yellow":1.0,"green":1.0' AS json
)
SELECT
  (
    SELECT SUM(CAST(val AS FLOAT64))
    FROM UNNEST(
      REGEXP_EXTRACT_ALL(
        JSON_EXTRACT(json, '$.Weights'),
        r':([^,]+)')
    ) AS val
  )
FROM T;

【讨论】：

完美工作。非常感谢！

【参考方案2】：

为了探索其他选择 -

以下是 BigQuery 标准 SQL

第一个例子是为每一行提取key:value对

#standardSQL
WITH `project.dataset.yourTbale` AS (
  SELECT 1 AS id, '"Weights":"blue":1.0,"purple":0.0,"yellow":1.0,"green":1.0' AS json 
  UNION ALL SELECT 2, '"Weights":"blue":1.0,"red":2.0,"yellow":1.0,"orange":3.0'
)
SELECT id,
  REPLACE(SPLIT(pair, ':')[OFFSET (0)], '"', '') color, 
  SAFE_CAST(SPLIT(pair, ':')[OFFSET (1)] AS FLOAT64) value
FROM `project.dataset.yourTbale`, 
UNNEST(SPLIT(REGEXP_REPLACE(JSON_EXTRACT(json, '$.Weights'), r'|', ''))) pair

这会给你下面的结果

id  color   value    
1   blue    1.0  
1   purple  0.0  
1   yellow  1.0  
1   green   1.0  
2   blue    1.0  
2   red     2.0  
2   yellow  1.0  
2   orange  3.0  

所以现在很容易将上面扩展到if there is a way to sum up all the values of the colors 的原始问题，甚至可以通过可能过滤特定颜色来进一步扩展它 - 请参见下面的示例

#standardSQL
WITH `project.dataset.yourTbale` AS (
  SELECT 1 AS id, '"Weights":"blue":1.0,"purple":0.0,"yellow":1.0,"green":1.0' AS json 
  UNION ALL SELECT 2, '"Weights":"blue":1.0,"red":2.0,"yellow":1.0,"orange":3.0'
)
SELECT id,
  SUM(SAFE_CAST(SPLIT(pair, ':')[OFFSET (1)] AS FLOAT64)) AS total
FROM `project.dataset.yourTbale`, 
UNNEST(SPLIT(REGEXP_REPLACE(JSON_EXTRACT(json, '$.Weights'), r'|', ''))) pair
WHERE REPLACE(SPLIT(pair, ':')[OFFSET (0)], '"', '') != 'blue'
GROUP BY id

结果如下（计算中不包括 color=blue）

id  total    
1   2.0  
2   6.0   

【讨论】：

这个也很好用，谢谢你的回答，米哈伊尔！