如何有效地维护传递闭包表？

Posted 2023-02-16

技术标签:

【中文标题】如何有效地维护传递闭包表？【英文标题】：How to maintain a transitive closure table efficiently? 【发布时间】：2013-09-30 08:26:20 【问题描述】：

我的关系数据库 (Firebird) 中有一个 DAG，其中包含两个表 edge 和 node（邻接列表模型）。我想递归地查询它们，但发现递归查询效率很低。所以我尝试实现触发器来维持 Dong et.al 之后的传递闭包。论文http://homepages.inf.ed.ac.uk/libkin/papers/tc-sql.pdf.

SELECTs 现在非常快，但DELETEs 非常慢，因为几乎整个图形都被复制一次删除。更糟糕的是，并发更新似乎是不可能的。

有没有更好的方法来实现这个？

编辑

我做了一些实验，并为 TC 表引入了一个引用计数器。这样，删除速度很快。我写了一些简单的测试用例，但我不确定我是否做得对。这是我到目前为止所拥有的：

CREATE GENERATOR graph_tc_seq;

CREATE TABLE EDGE (
    parent DECIMAL(10, 0) NOT NULL,
    child DECIMAL(10, 0) NOT NULL,
    PRIMARY KEY (parent, child)
);

CREATE TABLE GRAPH_TC (
    parent DECIMAL(10, 0) NOT NULL,
    child DECIMAL(10, 0) NOT NULL,
    refcount DECIMAL(9, 0),
    PRIMARY KEY (parent, child)
);

CREATE TABLE GRAPH_TC_TEMP (
    session_id DECIMAL(9, 0),
    parent DECIMAL(10, 0),
    child DECIMAL(10, 0)
);

CREATE PROCEDURE GRAPH_TC_CREATE (p_parent DECIMAL(10, 0), c_child DECIMAL(10, 0))
AS 
    declare variable tp_parent DECIMAL(10,0);
    declare variable tc_child DECIMAL(10,0);
    declare variable session_id DECIMAL(9,0);
    declare variable refs DECIMAL(9,0);
begin
    session_id = gen_id(graph_tc_seq,1);
    insert into graph_tc_temp (parent, child, session_id, refcount) values (:p_parent, :p_parent, :session_id, 1);
    insert into graph_tc_temp (parent, child, session_id, refcount) values (:c_child, :c_child, :session_id, 1);
    insert into graph_tc_temp (parent, child, session_id, refcount) values (:p_parent, :c_child, :session_id, 1);
    insert into graph_tc_temp (parent, child, session_id, refcount) select distinct :p_parent, child, :session_id, refcount from graph_tc where parent = :c_child and not parent = child;
    insert into graph_tc_temp (child, parent, session_id, refcount) select distinct :c_child, parent, :session_id, refcount from graph_tc where child = :p_parent and not parent = child;
    insert into graph_tc_temp (parent, child, session_id, refcount) select distinct a.parent, b.child, :session_id, a.refcount*b.refcount from graph_tc a, graph_tc b where a.child = :p_parent and b.parent = :c_child and not a.parent = a.child and not b.parent = b.child;
    for select parent, child, refcount from graph_tc_temp e where session_id= :session_id and exists (select * from graph_tc t where t.parent = e.parent and t.child = e.child ) into :tp_parent, :tc_child, :refs do begin
        update graph_tc set refcount=refcount+ :refs where parent = :tp_parent and child = :tc_child;
    end
    insert into graph_tc (parent, child, refcount) select parent, child, refcount from graph_tc_temp e where session_id = :session_id and not exists (select * from graph_tc t where t.parent = e.parent and t.child = e.child);
    delete from graph_tc_temp where session_id = :session_id;
end ^

CREATE PROCEDURE GRAPH_TC_DELETE (p_parent DECIMAL(10, 0), c_child DECIMAL(10, 0))
AS 
    declare variable tp_parent DECIMAL(10,0);
    declare variable tc_child DECIMAL(10,0);
    declare variable refs DECIMAL(9,0);
begin
    delete from graph_tc where parent = :p_parent and child = :p_parent and refcount <= 1;
    update graph_tc set refcount = refcount - 1 where parent = :p_parent and child = :p_parent and refcount > 1;
    delete from graph_tc where parent = :c_child and child = :c_child and refcount <= 1;
    update graph_tc set refcount = refcount - 1 where parent = :c_child and child = :c_child and refcount > 1;
    delete from graph_tc where parent = :p_parent and child = :c_child and refcount <= 1;
    update graph_tc set refcount = refcount - 1 where parent = :p_parent and child = :c_child and refcount > 1;
    for select distinct :p_parent,  b.child, refcount from graph_tc b where b.parent = :c_child and not b.parent = b.child into :tp_parent, :tc_child, :refs do begin
        delete from graph_tc where parent = :tp_parent and child = :tc_child and refcount <= :refs;
        update graph_tc set refcount = refcount - :refs where parent = :tp_parent and child = :tc_child and refcount > :refs;
    end
    for select distinct :c_child,  b.parent, refcount from graph_tc b where b.child = :p_parent and not b.parent = b.child into :tc_child, :tp_parent, :refs do begin
        delete from graph_tc where child = :tc_child and parent = :tp_parent and refcount <= :refs;
        update graph_tc set refcount = refcount - :refs where child = :tc_child and parent = :tp_parent and refcount > :refs;
    end
    for select distinct a.parent, b.child, a.refcount*b.refcount from graph_tc a, graph_tc b where not a.parent = a.child and not b.parent = b.child and a.child = :p_parent and b.parent = :c_child into :tp_parent, :tc_child, :refs do begin
        delete from graph_tc where parent = :tp_parent and child = :tc_child and refcount <= :refs;
        update graph_tc set refcount = refcount - :refs where parent = :tp_parent and child = :tc_child and refcount > :refs;
    end
end ^

CREATE TRIGGER GRAPH_TC_AFTER_INSERT FOR EDGE AFTER INSERT as
begin
    execute procedure graph_tc_create(new.parent,new.child);
end ^

CREATE TRIGGER GRAPH_TC_AFTER_UPDATE FOR EDGE AFTER UPDATE as
begin
    if ((new.parent <> old.parent) or (new.child <> old.child)) then begin
    execute procedure graph_tc_delete(old.parent,old.child);
    execute procedure graph_tc_create(new.parent,new.child);
    end
end ^

CREATE TRIGGER GRAPH_TC_AFTER_DELETE FOR EDGE AFTER DELETE as
begin
    execute procedure graph_tc_delete(old.parent,old.child);
end ^

这是我自己的想法，但我认为其他人已经实施了 TC。他们在做同样的事情吗？

我有一些测试用例，但我不确定是否会与更大的图表不一致。

并发怎么样，我认为当两个同时的事务想要更新图表时，这种方法会失败，对吧？

编辑

我在我的代码中发现了一些错误，我想与你分享修复的版本。

我发现了一篇很棒的文章：http://www.codeproject.com/Articles/22824/A-Model-to-Represent-Directed-Acyclic-Graphs-DAG-o。是否有更多有趣的文章或科学论文，采用不同的方法？

【问题讨论】：

您能否包括（相关部分）DDL 和触发器定义？ @MarkRotteveel 查看我的编辑考虑使用GTT (global temporary table) 而不是GRAPH_TC_TEMP 的普通表您想递归查询表格，还是查询特定的图表和连接？ @ChuckCottrill 我想递归查询所有连接。例如：节点 X 的所有（间接）父节点或子节点是什么？哪些节点位于节点 X 和 Y 之间？ 【参考方案1】：

我刚刚通过扩展到此处描述的传递自反闭包表模型修复了一个缓慢的删除操作： http://www.dba-oracle.com/t_sql_patterns_incremental_eval.htm。完全维护其中的路径计数需要更多的工作，但是当删除从每个单独的删除操作的 6 秒变为可忽略不计时，它得到了很大的回报（我现在可以删除图中的每个关系，然后将它们全部添加回来总共 4,000 条关系在 14 秒内完成）。

【讨论】：

另外，最短路径长度可以保持类似于路径总数tjhsst.edu/~rlatimer/acm/DatabaseSystems/…【参考方案2】：

SQL 不是处理图形的正确工具。使用其中之一：

http://en.wikipedia.org/wiki/Graph_database

我非常喜欢 ArangoDB，它的语法接近 mongodb。

【讨论】：

我知道graph db 将是理想的解决方案；但是对于两个具有 【参考方案3】：

尝试为相关的 where 子句创建索引（例如：child、parent）。

我不熟悉 Firebird，但看看“firebird describe”是如何在它上面工作的，并检查它是否正确使用了索引以加快您在程序中的选择。

即使在更新/删除/插入的性能上损失了索引，在您的情况下，它也可以改善结果。

【讨论】：

真正的实现有索引；我只是没有将CREATE INDEX 语句复制到上面的代码中。