如何在不破坏我的结构的情况下将特定单元格排除到 BigQuery 中的数组数组中？

Posted 2023-03-25

【中文标题】如何在不破坏我的结构的情况下将特定单元格排除到 BigQuery 中的数组数组中？

【英文标题】：How exclude specific cell into a array of array in BigQuery wihtout break my structure?

【发布时间】：2021-07-06 21:59:42

【问题描述】：

我尝试在 BigQuery 中选择我最终选择的所有值（在 "SELECT * from temp_ship as p" 下面的代码中），除了值 id 和 last_changed_at t1_ship_line（我的加入需要这些值，但之后，我不需要在最后一次选择中获取它们。在此过程中我也需要保留我的数组结构。

我尝试使用 except 但没有成功，有什么想法吗？

第 1 行是我实际拥有的，第 3 行是我想要获得的（我搜索以删除最终选择中的 2 橙色列，在没有数组的普通表中，我只写了我需要的列，但在这里我可以'例如，不要写 aaa.bbb.cccc :/)

这是我的代码

WITH
  t1_ship_line AS (
  SELECT
    id, last_changed_at, line_number as line_num, article_id as product_id, quantity as product_qty, order_item_id
  FROM
    `table1` ),
  temp_ship AS (
  SELECT
    CAST(o_id AS INT64) AS id,
    CAST(a.last_changed_at AS TIMESTAMP) AS last_changed_at,
    ARRAY_AGG( STRUCT (id AS id,
        comment AS comment,
        carrier_id AS carrier_code,
        carrier_label AS carrier_label,
        tracking AS tracking_code,
        b AS line )) AS shipment,
  FROM
    `table2` a
  LEFT JOIN t1_ship_line b
  ON a.o_id = b.id
  and a.last_changed_at = b.last_changed_at
  GROUP BY
    o_id,
    last_changed_at )

SELECT * from temp_ship as p

这里是两张表的数据示例：

table1

id,last_changed_at,line_num,product_id,quantity ,order_item_id

3000000000,2021-07-06 12:07:00 UTC,1,999999,6.0,0

3000000001,2021-07-06 12:07:00 UTC,1,999998,6.0,0

table2

id,o_id,last_changed_at,comment,carrier_id,carrier_label,tracking

100,3000000000,2021-07-06 12:07:00 UTC,COMMENT,nameofcarrierid,labelofcarrier,LOC

100,3000000001,2021-07-06 12:07:00 UTC,COMMENT,nameofcarrierid,labelofcarrier,LOC

提前感谢您的帮助；）

【问题讨论】：

提供输入数据样本和预期输出，以便我们为您提供帮助

@MikhailBerlyant 我提供了两张表的一些样本，如果还不够，请不要犹豫，回复，谢谢您的帮助

我无法重现您的案例，我不明白您为什么要执行 ARRAY_AGG。你能提供更多的上下文和一个真实的输入数据集（里面有货物）吗？

【参考方案1】：

花费一些精力后，这里的技巧是在 t1_ship_line 中执行 array_agg(struct ()) + 分区，如下所示：

t1_ship_line AS (
  SELECT
    id, last_changed_at, line_number as line_num, article_id as product_id, quantity as product_qty, order_item_id,
    array_agg(struct(line_number as line_num , article_id as product_id, cast(quantity as NUMERIC) as product_qty, cast(order_item_id as INT64) as order_item_id )) over (partition by orders_shipment_id,last_changed_at) as shipment
  FROM
    `table1` 
),
temp_shipment as (
SELECT
    CAST(orders_id AS INT64) AS id,
    CAST(a.last_changed_at AS TIMESTAMP) AS last_changed_at,
    ARRAY_AGG( STRUCT (id AS id,
        comment AS comment,
        carrier_id AS carrier_code,
        carrier_label AS carrier_label,
        tracking AS tracking_code,
        b.shipment as line )) as shipment
  FROM
    `table2` a
  LEFT JOIN t1_ship_line b
  ON a.orders_id = b.orders_shipment_id
  and a.last_changed_at = b.last_changed_at
  GROUP BY
    orders_id,
    last_changed_at
) select * from temp_shipment