如何一次刷新 Postgresql 9.3 中的所有物化视图？

Posted 2023-02-16

【中文标题】如何一次刷新 Postgresql 9.3 中的所有物化视图？

【英文标题】：How to refresh all materialized views in Postgresql 9.3 at once?

【发布时间】：2013-11-27 16:42:48

【问题描述】：

我正在将一堆数据加载到 PostgresQL 9.3 数据库中，然后我想刷新依赖于更新表的所有物化视图。有没有办法自动完成，而不是浏览每个视图并一个一个地刷新它们？我知道 Oracle 可以很容易地做到这一点，但在梳理 PostgreSQL 文档后我没有找到任何东西。

【问题讨论】：

检查 pg_matviews，选择您需要的所有视图并进行刷新。你可以为此编写一个函数。

看起来写我自己的函数是当前版本的唯一选择

【参考方案1】：

如果物化视图不相互依赖，上述答案可以正常工作。如果不是这种情况，那么刷新物化视图的顺序很重要（即，您需要先刷新不依赖于任何其他物化视图的物化视图，然后再刷新那些物化视图）。下面的代码将生成物化视图的有序列表，以便它们可以以正确的顺序更新。

CREATE OR REPLACE VIEW mat_view_dependencies AS
WITH RECURSIVE s(start_schemaname,start_mvname,schemaname,mvname,relkind,
               mvoid,depth) AS (
-- List of mat views -- with no dependencies
SELECT n.nspname AS start_schemaname, c.relname AS start_mvname,
n.nspname AS schemaname, c.relname AS mvname, c.relkind,
c.oid AS mvoid, 0 AS depth
FROM pg_class c JOIN pg_namespace n ON c.relnamespace=n.oid
WHERE c.relkind='m'
UNION
-- Recursively find all things depending on previous level
SELECT s.start_schemaname, s.start_mvname,
n.nspname AS schemaname, c.relname AS mvname,
c.relkind,
c.oid AS mvoid, depth+1 AS depth
FROM s
JOIN pg_depend d ON s.mvoid=d.refobjid
JOIN pg_rewrite r ON d.objid=r.oid
JOIN pg_class c ON r.ev_class=c.oid AND (c.relkind IN ('m','v'))
JOIN pg_namespace n ON n.oid=c.relnamespace
WHERE s.mvoid <> c.oid -- exclude the current MV which always depends on itself
)
SELECT * FROM s;

CREATE OR REPLACE VIEW mat_view_refresh_order AS
WITH b AS (
-- Select the highest depth of each mat view name
SELECT DISTINCT ON (schemaname,mvname) schemaname, mvname, depth
FROM mat_view_dependencies
WHERE relkind='m'
ORDER BY schemaname, mvname, depth DESC
)
-- Reorder appropriately
SELECT schemaname, mvname, depth AS refresh_order
FROM b
ORDER BY depth, schemaname, mvname
;

这可以在psql 中使用，以如下适当的顺序刷新所有视图：

WITH a AS (
SELECT 'REFRESH MATERIALIZED VIEW "' || schemaname || '"."' || mvname || '";' AS r
FROM mat_view_refresh_order
ORDER BY refresh_order
)
SELECT string_agg(r,E'\n') AS script FROM a \gset

\echo :script
:script

这最后一部分也可以像之前的解决方案一样转换成一个函数。

【讨论】：

以上是否意味着 frankhommers 的回答，我正在尝试将最后一部分变成一个函数..

【参考方案2】：

当该视图存在 UNIQUE 索引时，下面的 sn-p 使用 REFRESH MATERIALIZED VIEW CONCURRENTLY。

CREATE OR REPLACE FUNCTION public.refresh_materialized_views()
  RETURNS void
AS
$BODY$
  DECLARE
      refresh_sql text;
    BEGIN

    WITH matviews AS (
       SELECT t.oid,
              relname AS view_name,
              nspname AS schema_name
         FROM pg_class t
         JOIN pg_catalog.pg_namespace n ON n.oid = t.relnamespace
        WHERE t.relkind = 'm'
          AND nspname NOT LIKE 'pg-%'
    ), unique_indexes AS (
     SELECT m.oid,
            view_name,
            schema_name
       FROM pg_class i,
            pg_index ix,
            matviews m
      WHERE ix.indisunique = true
        AND ix.indexrelid = i.oid
        AND ix.indrelid = m.oid
    ), refresh_concurrently AS (
      SELECT 'REFRESH MATERIALIZED VIEW CONCURRENTLY ' || quote_ident(schema_name) || '.' || quote_ident(view_name) AS sql
        FROM unique_indexes
    ), refresh AS (
      SELECT 'REFRESH MATERIALIZED VIEW ' || quote_ident(schema_name) || '.' || quote_ident(view_name) AS sql
        FROM matviews
       WHERE oid != all (SELECT oid FROM unique_indexes)
    ), sql AS (
      SELECT sql FROM refresh_concurrently
      UNION ALL
      SELECT sql FROM refresh
    )

    SELECT string_agg(sql, E';\n') || E';\n' FROM sql INTO refresh_sql;

    EXECUTE refresh_sql;

    END;
$BODY$
LANGUAGE plpgsql VOLATILE;

这个 sn-p 接受一个模式名称来限制刷新的视图。

CREATE OR REPLACE FUNCTION public.refresh_materialized_views(_schema text)
  RETURNS void
AS
$BODY$
  DECLARE
      refresh_sql text;
    BEGIN

    WITH matviews AS (
       SELECT t.oid,
              relname AS view_name,
              nspname AS schema_name
         FROM pg_class t
         JOIN pg_catalog.pg_namespace n ON n.oid = t.relnamespace
        WHERE t.relkind = 'm'
          AND nspname NOT LIKE 'pg-%'
          AND nspname = _schema
    ), unique_indexes AS (
     SELECT m.oid,
            view_name,
            schema_name
       FROM pg_class i,
            pg_index ix,
            matviews m
      WHERE ix.indisunique = true
        AND ix.indexrelid = i.oid
        AND ix.indrelid = m.oid
    ), refresh_concurrently AS (
      SELECT 'REFRESH MATERIALIZED VIEW CONCURRENTLY ' || quote_ident(schema_name) || '.' || quote_ident(view_name) AS sql
        FROM unique_indexes
    ), refresh AS (
      SELECT 'REFRESH MATERIALIZED VIEW ' || quote_ident(schema_name) || '.' || quote_ident(view_name) AS sql
        FROM matviews
       WHERE oid != all (SELECT oid FROM unique_indexes)
    ), sql AS (
      SELECT sql FROM refresh_concurrently
      UNION ALL
      SELECT sql FROM refresh
    )

    SELECT string_agg(sql, E';\n') || E';\n' FROM sql INTO refresh_sql;

    EXECUTE refresh_sql;

    END;
$BODY$
LANGUAGE plpgsql VOLATILE;

【参考方案3】：

相同的方法，添加了在所有架构上运行它的选项，可选择同时运行。

CREATE OR REPLACE FUNCTION RefreshAllMaterializedViews(_schema TEXT DEFAULT '*', _concurrently BOOLEAN DEFAULT false)
RETURNS INT AS $$
  DECLARE
    r RECORD;
  BEGIN
    RAISE NOTICE 'Refreshing materialized view(s) in % %', CASE WHEN _schema = '*' THEN ' all schemas' ELSE 'schema "'|| _schema || '"' END, CASE WHEN _concurrently THEN 'concurrently' ELSE '' END;
    IF pg_is_in_recovery() THEN 
      RETURN 0;
    ELSE    
      FOR r IN SELECT schemaname, matviewname FROM pg_matviews WHERE schemaname = _schema OR _schema = '*' 
      LOOP
        RAISE NOTICE 'Refreshing %.%', r.schemaname, r.matviewname;
        EXECUTE 'REFRESH MATERIALIZED VIEW ' || CASE WHEN _concurrently THEN 'CONCURRENTLY ' ELSE '' END || '"' || r.schemaname || '"."' || r.matviewname || '"'; 
      END LOOP;
    END IF;
    RETURN 1;
  END 
$$ LANGUAGE plpgsql;

我也放到了GitHub上：https://github.com/frankhommers/RefreshAllMaterializedViews

【参考方案4】：

同样的方法，增加了备用检查

CREATE OR REPLACE FUNCTION RefreshAllMaterializedViews(schema_arg TEXT DEFAULT 'public')
RETURNS INT AS $$
DECLARE
    r RECORD;

BEGIN
    RAISE NOTICE 'Refreshing materialized view in schema %', schema_arg;
    if pg_is_in_recovery()  then 
    return 1;
    else
    FOR r IN SELECT matviewname FROM pg_matviews WHERE schemaname = schema_arg 
    LOOP
        RAISE NOTICE 'Refreshing %.%', schema_arg, r.matviewname;
        EXECUTE 'REFRESH MATERIALIZED VIEW ' || schema_arg || '.' || r.matviewname; 
    END LOOP;
    end if;
    RETURN 1;
END 
$$ LANGUAGE plpgsql;

【参考方案5】：

看起来当前版本的 PostgreSQL (9.3.1) 没有这样的功能，不得不编写我自己的函数：

CREATE OR REPLACE FUNCTION RefreshAllMaterializedViews(schema_arg TEXT DEFAULT 'public')
RETURNS INT AS $$
DECLARE
    r RECORD;
BEGIN
    RAISE NOTICE 'Refreshing materialized view in schema %', schema_arg;
    FOR r IN SELECT matviewname FROM pg_matviews WHERE schemaname = schema_arg 
    LOOP
        RAISE NOTICE 'Refreshing %.%', schema_arg, r.matviewname;
        EXECUTE 'REFRESH MATERIALIZED VIEW ' || schema_arg || '.' || r.matviewname; 
    END LOOP;

    RETURN 1;
END 
$$ LANGUAGE plpgsql;

（在github上：https://github.com/sorokine/RefreshAllMaterializedViews）

【讨论】：

现在在 9.4 中支持 "CONCURRENTLY" 关键字，您是否有兴趣使用它来防止表锁定？

我想在我开始安装 9.4 后立即进行调查。我认为这将是一个方便的补充。

今天我把代码拉到了github.com/sorokine/RefreshAllMaterializedViews的master分支中，它执行并发更新。

这是一个 SQL，用于查找无法同时刷新的物化视图（因为缺少 UNIQUE 索引）：dba.stackexchange.com/a/101938/8607我将这两个脚本作为一个团队使用。

请注意，如果某些物化视图依赖于其他物化视图，那么它们运行的​​顺序将很重要。该脚本不考虑这一点。如果你有这样的情况，你需要做一些更复杂的事情。