运行查询时出错 查询执行期间超出资源

Posted 2023-03-24

【中文标题】运行查询时出错 查询执行期间超出资源

【英文标题】：Error running query Resources exceeded during query execution

【发布时间】：2021-06-30 13:17:25

【问题描述】：

当我在 GBQ 中运行该查询时，我收到此错误：

运行查询时出错

查询执行期间超出资源：没有足够的资源用于 查询计划 - 子查询太多或查询太复杂.. at [1:1]

下面的代码不是我开发的，所以我很难重写它。如果你能帮我让它重新运行，我会听到的。

execute immediate (select concat('create or replace table `ssmdw-208309.temp_piwik_and_ga.005_visitor_union` as ',query) from (select string_agg(CONCAT(' select substr(generate_uuid(),25,12) as rowid, ',columns,' from `piwik-272210.piwik.',table_name,'`'),'\nUnion all\n')  query FROM (
  SELECT
    table_column.table_name,
    STRING_AGG(CASE
        WHEN x.column_name IS NULL THEN CONCAT('null as ',table_column.column_name)
      ELSE
      table_column.column_name
    END
      ,', '
    ORDER BY
      table_column.column_name) columns
  FROM (
    SELECT
      tables.table_name,
      columns.column_name
    FROM (
      SELECT
        DISTINCT column_name
      FROM
        `piwik-272210.piwik.INFORMATION_SCHEMA.COLUMNS` a where table_name like 'visitor_%') columns
    FULL OUTER JOIN (
      SELECT
        DISTINCT table_name
      FROM
        `piwik-272210.piwik.INFORMATION_SCHEMA.COLUMNS` b where table_name like 'visitor_%') tables
    ON
      (1=1)) table_column
  LEFT JOIN
    `piwik-272210.piwik.INFORMATION_SCHEMA.COLUMNS` x
  ON
    (x.table_name = table_column.table_name
      AND x.column_name = table_column.column_name)
  GROUP BY
    1 )))

【参考方案1】：

尽可能避免使用“order by”子句。 “排序依据”尝试将结果拉到一个通常会导致此问题的节点中。

【讨论】：

感谢您的输入我现在已删除订单，但不幸的是我仍然收到此错误消息 --> 查询执行期间资源超出：没有足够的资源用于查询计划 - 子查询或查询太多复杂..在 [1:1]

尝试消除中间过多的子查询。您可以通过将中间结果存储在某个表中来做到这一点。这可以使用存储过程来实现。在某些表中存储以下两个查询的结果：SELECT DISTINCT column_name FROM piwik-272210.piwik.INFORMATION_SCHEMA.COLUMNS` a where table_name like 'visitor_%') columns SELECT DISTINCT table_name FROM piwik-272210.piwik.INFORMATION_SCHEMA.COLUMNS` b where table_name like 'visitor_%') 表然后执行其余操作。

感谢您的意见。我现在已经实施了你的解决方案，很遗憾我仍然收到相同的错误消息

【参考方案2】：

试试这个。然后调用该过程。

create or replace procedure `dataset_name'.'procedure_name'()

BEGIN

create temp table TABL1 as (
SELECT  DISTINCT column_name FROM  `piwik-272210.piwik.INFORMATION_SCHEMA.COLUMNS` a where table_name like 'visitor_%');
    

create temp table TABL2 as (
SELECT DISTINCT table_name FROM `piwik-272210.piwik.INFORMATION_SCHEMA.COLUMNS` b where table_name like 'visitor_%');

create temp table TABL3 as
SELECT tables.table_name, columns.column_name FROM (select column_name from TABL1 columns
                                                    FULL OUTER JOIN 
                                                    select table_name from TABL2 tables ON (1=1));

create or replace table dataset.TABL4 as
  SELECT   table_column.table_name, 
           STRING_AGG(CASE WHEN x.column_name IS NULL THEN CONCAT('null as ',table_column.column_name)  ELSE  table_column.column_name END) as columns
  FROM (
    SELECT table_name, column_name FROM TABL3
       ) table_column 
    left join `piwik-272210.piwik.INFORMATION_SCHEMA.COLUMNS` x
    ON    (x.table_name = table_column.table_name AND x.column_name = table_column.column_name)
))

drop table TABL1 ;
drop table TABL2 ;
drop table TABL3;

END;