LAST_VALUE with IF statement inside not backfill it's partition --> 在选择每个分区的第一行时丢失最后一个值（BigQuery/

Posted 2023-02-16

【中文标题】LAST_VALUE with IF statement inside not backfill it\'s partition --> 在选择每个分区的第一行时丢失最后一个值（BigQuery/SQL）

【英文标题】：LAST_VALUE with IF statement inside not backfilling it's partition --> losing last values when selecting first line of each partition (BigQuery/SQL)LAST_VALUE with IF statement inside not backfill it's partition --> 在选择每个分区的第一行时丢失最后一个值（BigQuery/SQL）

【发布时间】：2021-03-02 19:35:51

【问题描述】：

我遇到了窗口函数问题。对于包含与用户相关的事件的数据集，我想为某些人选择 FIRST_VALUE，为其他人选择 LAST_VALUE，然后将其压缩为每个用户的一行。

当使用 FIRST_VALUE/LAST_VALUE 方法、按用户分区并按日期/时间戳排序时，我使用 FIRST_VALUE 获得了令人满意的结果（= 我的第一个值中的行填充了整个列）。在 LAST_VALUE 子句中，我包含了一个 IF 语句，以创建一个说明帐户删除时间的列。它根本不起作用..有什么解决方法的建议吗？

包括下面的最小示例表，以及进一步向下的预期输出。

WITH dataset_table AS (
  SELECT DATE '2020-01-01' date , 1 user, 'german' user_language, 'created_account' event UNION ALL
  SELECT '2020-01-02', 1, 'german', 'successful_login' UNION ALL
  SELECT '2020-01-03', 1, 'english', 'screen_view' UNION ALL
  SELECT '2020-01-04', 1, 'english', 'deleted_account' UNION ALL
  SELECT '2020-01-01', 2, 'english', 'login' UNION ALL
  SELECT '2020-01-02', 2, 'english', 'settings' UNION ALL
  SELECT '2020-01-03', 2, 'english', 'NULL' UNION ALL
  SELECT '2020-01-04', 2, 'french', 'screen_view'
),

user_info AS (
    SELECT
        `date`,
        user,
        -- record first value for language = signup demographics
        FIRST_VALUE(user_language IGNORE NULLS) OVER time_order user_language,
        -- record last value for app removal - want to know if the user deleted their account and didn't return
        LAST_VALUE(IF(event = 'deleted_account', `date`, NULL)) OVER time_order deleted_account,
        ROW_NUMBER() OVER time_order row_idx
    FROM dataset_table
    WINDOW time_order AS (PARTITION BY user ORDER BY date)
)

SELECT
  *
FROM user_info
WHERE row_idx = 1 -- Here, I select the first row, but deleted_account hasn't been populated by the last value for user 1. The same test for FIRST_VALUE does populate the whole column with german, so if I'd use row_idx = 4 I'd get a correct answer to this example, but there are different amount of events for each user in reality, so I want to use row_idx 1 to pick out the ideal line. 

预期输出：

date         user  user_language  deleted_account row_idx 
2020-01-01   1     german         2020-01-04      1
2020-01-02   2     english        null            1

【问题讨论】：

GMB 的 MAX 工作时，您没有得到预期结果是有原因的：LAST_VALUE 的默认窗口是 rows unbounded preceding，而应该是 rows between unbounded preceding and unbounded following

@dnoeth: 或使用first_value() over(... order by date desc)

【参考方案1】：

我想你想要：

with dataset_table AS (...),
user_info AS (
    SELECT
        `date`,
        user,
        FIRST_VALUE(user_language IGNORE NULLS) OVER (PARTITION BY user ORDER BY date) user_language,
        MAX(IF(event = 'deleted_account', `date`, NULL)) OVER (PARTITION BY user) deleted_account,
        ROW_NUMBER() OVER (PARTITION BY user ORDER BY date) row_idx
    FROM dataset_table
)

SELECT *
FROM user_info
WHERE row_idx = 1 

【讨论】：

就是这样！当我看到解决方案时看起来很简单，但我花了很多时间自己没有找到它...... :) 谢谢！