无法从具有数组和结构的 BigQuery 表中提取键值对

Posted 2023-03-25

【中文标题】无法从具有数组和结构的 BigQuery 表中提取键值对

【英文标题】：Unable to extract key-value pairs from BigQuery table which has arrays and structs

【发布时间】：2017-08-23 09:44:27

【问题描述】：

我已将 Firebase 分析数据与 google BigQuery 集成，并且每天都会创建一个带有 DATE 标记的新表。

一个示例表是“projectID.com_dev_sambhav_android.app_events_20170821”

一个示例表如下所示 sample table

我的要求是为 event_dim.name="notification_received" 获得以下所需格式的结果 desired output

为了获得所需的输出，我执行以下查询（标准 SQL）

SELECT event.name
 (SELECT param.value.string_value FROM UNNEST(event_dim.params) AS param WHERE param.key="notification_title") as notification_title,
 (SELECT param.value.string_value FROM UNNEST(event_dim.params) AS param WHERE param.key="item_id") as item_id 
FROM `projectID.com_dev_sambhav_ANDROID.app_events_20*`, UNNEST(event_dim) as event  
WHERE event.name = "notification_received"

但我得到了这个错误

错误：每个函数参数都是一个表达式，而不是一个查询。使用一个 查询作为表达式，查询必须用附加的 括号使其成为标量子查询表达式。

谁能帮我解决这个问题

【问题讨论】：

Debiprasad，请尽可能尝试在您的问题中使用文本而不是图像，因为谷歌可以索引文本，并且阅读您的问题的任何人都可以更轻松地了解信息。

【参考方案1】：

问题是逗号和别名。此查询有效：

#standardSQL
WITH `projectID.com_dev_sambhav_ANDROID.app_events_2017` AS(
  SELECT ARRAY< STRUCT<date STRING, name STRING, params ARRAY< STRUCT<key STRING, value STRUCT<string_value STRING> > > > > [STRUCT('20170814' AS date, 'notification_received' AS name, [STRUCT('notification_title' AS key, STRUCT('Amazing Offers two' AS string_value) AS value ), 
                                                                                                                STRUCT('firebase_screen_class' AS key, STRUCT('RetailerHomeActivity' AS string_value) AS value),
                                                                                                                STRUCT('notification_id' AS key, STRUCT('12345' AS string_value) AS value),
                                                                                                                STRUCT('firebase_screen_id' AS key, STRUCT('app' AS string_value) AS value),
                                                                                                                STRUCT('item_id' AS key, STRUCT('DEMO-02' AS string_value) AS value),
                                                                                                                STRUCT('firebase_screen' AS key, STRUCT('My Order' AS string_value) AS value)] AS params)] event_dim
)

SELECT
 event.name,
 (SELECT param.value.string_value FROM UNNEST(event.params) AS param WHERE param.key="notification_title") as notification_title,
 (SELECT param.value.string_value FROM UNNEST(event.params) AS param WHERE param.key="item_id") as item_id 
FROM `projectID.com_dev_sambhav_ANDROID.app_events_20*`, UNNEST(event_dim) as event  
WHERE event.name = "notification_received"

如果您 UNNEST 字段 event_dim 并将其命名为 event，那么您应该在查询中使用此别名作为参考。

作为补充，这也是解决您的问题的另一种方法（这只是另一种可能性，因此您在使用 BigQuery 时掌握了更多技术）：

#standardSQL
SELECT
  (SELECT date FROM UNNEST(event_dim)) date,
  (SELECT params.value.string_value FROM UNNEST(event_dim) event, UNNEST(event.params) params WHERE event.name = 'notification_received' AND params.key = 'notification_title') AS notification_title,
  (SELECT params.value.string_value FROM UNNEST(event_dim) event, UNNEST(event.params) params WHERE event.name = 'notification_received' AND params.key = 'item_id') AS item_id
FROM `projectID.com_dev_sambhav_ANDROID.app_events_2017`
WHERE EXISTS(SELECT 1 FROM UNNEST(event_dim) WHERE name = 'notification_received')

在处理高达 TB 的数据时，您可能会发现此查询仍然执行得很好。

【讨论】：

但是，我对另一个事件“notification_opened”使用了相同的查询，它抛出了错误。

` SELECT (SELECT date FROM UNNEST(event_dim)) date, (SELECT params.value.string_value FROM UNNEST(event_dim) event, UNNEST(event.params) params WHERE event.name = 'notification_opened' AND params.key = 'notification_title') AS notification_title, (SELECT params.value.string_value FROM UNNEST(event_dim) event, UNNEST(event.params) params WHERE event.name = 'notification_opened' AND params.key = 'item_id') AS item_id FROM shikhar-fa2b7.com_hul_sambhav_ANDROID.app_events_20* WHERE EXISTS(SELECT 1 FROM UNNEST(event_dim) WHERE name = 'notification_opened')`

错误：标量子查询产生了多个元素

当子查询找到多个具有相同param.value.string_value 的值时会发生这种情况。解决它的一种方法可能是将结果作为数组带来，例如ARRAY((select params.value.string_value...)，但这会改变数据的结构。让我知道这是否适合您。

我使用了第一个建议（ WITH table AS），它适用于 notification_opened。谢谢兄弟