如何从 Hiveql 中的 select over 语句中仅提取最近一周？

Posted 2023-04-12

【中文标题】如何从 Hiveql 中的 select over 语句中仅提取最近一周？

【英文标题】：How would I extract only the latest week from a select over statement in Hiveql?

【发布时间】：2020-11-13 19:14:51

【问题描述】：

我需要一些帮助，我创建了一个查询，该查询保留了元素是否针对特定度量返回 1 或 0 的运行总计，如果度量提供 0，则运行总计返回 0，示例如下：

year_week  element  measure  running_total
2020_40    A        1        1
2020_41    A        1        2
2020_42    A        1        3
2020_43    A        0        0
2020_44    A        1        1
2020_45    A        1        2
2020_40    B        1        1
2020_41    B        1        2
2020_42    B        1        3
2020_43    B        1        4
2020_44    B        1        5
2020_45    B        1        6

以上是使用这个查询实现的：

SELECT element,
       year_week,
       measure,
       SUM(measure) OVER (PARTITION BY element, flag_sum ORDER BY year_week ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS running_total
    FROM (
      SELECT *,
          SUM(measure_flag) OVER (PARTITION BY element ORDER BY year_week ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS flag_sum
      FROM (
        SELECT *,
            CASE WHEN  measure = 1 THEN 0 ELSE 1 END AS measure_flag
        FROM database.table ) x ) y

这很好并且有效 - 但我只想提供每个元素的最近几周的数据。所以在上面的例子中它会是：

year_week  element  measure  running_total
2020_45    A        1        2
2020_45    B        1        6

基本上我需要保持逻辑相同但限制返回的数据集。我已经尝试过了，但是它将结果从正确的运行总数更改为 1 或 0。

非常感谢任何帮助！

【参考方案1】：

您可以添加另一层嵌套，并使用row_number() 过滤每个element 的最新记录。

我建议：

select element, year_week, measure, running_total
from (
    select t.*,
        row_number() over(partition by element, grp order by year_week) - 1 as running_total
    from (
        select t.*,
            sum(1 - measure) over(partition by element order by year_week) as grp,
            row_number() over(partition by element order by year_week desc) as rn
        from mytable t
    ) t
) t
where rn = 1

我稍微简化了查询，考虑到 measure 的值只有 0 和 1，如您的示例数据所示。如果不是这样，那么：

select element, year_week, measure, running_total
from (
    select t.*,
        sum(measure) over(partition by element, grp order by year_week) as running_total
    from (
        select t.*,
            sum(case when measure = 0 then 1 else 0 end) over(partition by element order by year_week) as grp,
            row_number() over(partition by element order by year_week desc) as rn
        from mytable t
    ) t
) t
where rn = 1

【讨论】：

这看起来很棒 - 我会考虑实现它并让你知道它是如何进行的。感谢您的帮助！