如何优化我的 Firebird SQL 查询？

Posted 2023-05-08

技术标签:

【中文标题】如何优化我的 Firebird SQL 查询？【英文标题】：How can I optimize my Firebird SQL query? 【发布时间】：2018-03-23 18:57:27 【问题描述】：

数据库：火鸟 3.0

此查询用于选择符合特定条件的记录，以检测哪些DEVICEID 的LAST_TICK 比DEVICEID=INTEGER VALUE 知道的其他DEVICEID 的DEVICEID 大

这是 SQL 查询：

select 
    a.TABELA, 
    a.DEVICEID,
    (select LAST_TICK from KNOWLEDGE b where b.DEVICEID=INTEGER VALUE and b.CROSSID=a.DEVICEID and b.TABELA=a.TABELA) as MEU_TICK,
    max(LAST_TICK) as MAX_TICK 
from 
    KNOWLEDGE a 
where 
    a.DEVICEID=a.CROSSID 
group by
    a.TABELA, a.DEVICEID 
having 
    max(LAST_TICK) > (select LAST_TICK from KNOWLEDGE b where b.DEVICEID=INTEGER VALUE and b.CROSSID=a.DEVICEID and b.TABELA=a.TABELA) 
order by 
    a.TABELA

只有 11 条记录的性能统计；

Plan
PLAN (B INDEX (PK_KNOWLEDGE))
PLAN (B INDEX (PK_KNOWLEDGE))
PLAN SORT (A NATURAL)

------ Performance info ------
Prepare time = 0ms
Execute time = 0ms
Current memory = 36.473.632
Max memory = 36.729.200
Memory buffers = 2.048
Reads from disk to cache = 0
Writes from cache to disk = 0
Fetches from cache = 65

Select Expression
-> Singularity Check
    -> Filter
        -> Table "KNOWLEDGE" as "B" Access By ID
            -> Bitmap
                -> Index "PK_KNOWLEDGE" Unique Scan
Select Expression
-> Singularity Check
    -> Filter
        -> Table "KNOWLEDGE" as "B" Access By ID
            -> Bitmap
                -> Index "PK_KNOWLEDGE" Unique Scan
Select Expression
-> Filter
    -> Aggregate
        -> Sort (record length: 214, key length: 176)
            -> Filter
                -> Table "KNOWLEDGE" as "A" Full Scan

*** 编辑 1： DDL：

CREATE TABLE KNOWLEDGE (
DEVICEID   "INTEGER" NOT NULL /* "INTEGER" = INTEGER */,
TABELA     VARCHAR40 NOT NULL /* VARCHAR40 = VARCHAR(40) */,
CROSSID    "INTEGER" NOT NULL /* "INTEGER" = INTEGER */,
LAST_TICK  "INTEGER" NOT NULL /* "INTEGER" = INTEGER */
);
ALTER TABLE KNOWLEDGE ADD CONSTRAINT PK_KNOWLEDGE PRIMARY KEY (DEVICEID, TABELA, CROSSID);
CREATE INDEX KNOWLEDGE_IDX1 ON KNOWLEDGE (TABELA);
CREATE INDEX KNOWLEDGE_IDX2 ON KNOWLEDGE (DEVICEID, CROSSID);

样本数据：

INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 4, 11);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 5, 47);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 6, 118);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (4, 'PESSOAS', 12, 5);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 4, 11);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 5, 47);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 6, 118);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (5, 'PESSOAS', 12, 5);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 4, 11);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 5, 47);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 6, 118);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (6, 'PESSOAS', 12, 5);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 4, 11);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 5, 47);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 6, 118);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (7, 'PESSOAS', 12, 5);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 4, 11);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 5, 47);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 6, 118);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (8, 'PESSOAS', 12, 5);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 4, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 5, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 6, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (9, 'PESSOAS', 12, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 4, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 5, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 6, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (10, 'PESSOAS', 12, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 4, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 5, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 6, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (11, 'PESSOAS', 12, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 4, 11);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 5, 47);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 6, 118);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 7, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 8, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 9, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 10, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 11, 0);
INSERT INTO KNOWLEDGE (DEVICEID, TABELA, CROSSID, LAST_TICK)
           VALUES (12, 'PESSOAS', 12, 10);

COMMIT WORK;

详细逻辑：

用于数据同步算法。该知识表包含来自系统中注册的每个设备的知识（Last Update Tick），并将其存储在服务器中。客户端设备连接到服务器并发送更新的记录，服务器更新其知识表以反映这些更改。然后其他客户端设备连接到服务器以获取这些更改并更新其本地数据库。

知识表的逻辑是这样的：

D-Hey 服务器，我是 4 号设备，我想知道有什么新东西适合我吗？

S-K，等一下，让我检查一下。嘿数据库服务器，从知识表中获取 DEVICEID 的 LAST_TICK 高于我自己的知识 (CROSSID=DEVICEID) 的所有记录，并将设备自己的知识放入该查询中，以便服务器更容易构建查询用于这些记录。

DB- 当然，这是列表

S- 很好，现在从这些表中获取所有那些 update_tick 高于设备 4 知识的记录。

Db- 是的，就是这样。

S-嘿设备，获取这些记录并更新它

d-np，完成。谢谢大佬。。

使用上面提供的测试数据，无论我们在 INTEGER VALUE 中提供什么设备 ID，结果都应该是包含值的单行： TABELA: "PESSOAS", DEVICEID: 12, MEU_TICK: 5, MAX_TICK: 12

【问题讨论】：

问题是什么？请提供DDL（包括索引）和样本数据，这个查询会产生一条记录，还是有多个结果？糟糕，抱歉，将 DDL 和示例记录添加到问题中。此外，查询会产生多条记录结果。当我运行查询时，结果是 MAX_TICK=10，而不是 12，同样使用 deviceid 9、10、11 结果为 4 行，而不仅仅是 1。是的，是的，我又犯了错。它的 10 .. 关于设备 9,10 和 11 4 行的结果，这是因为在我从表中提取数据之前它们尚未同步。 .. 【参考方案1】：

一个可能会提高性能的非常简单的更改正在更改

group by
    a.TABELA, a.DEVICEID

到

group by
    a.DEVICEID, a.TABELA

这将允许优化器使用PK_KNOWLEDGE 进行聚合所需的排序。

可以通过将查询重写为：

with max_tick as (
    select 
       TABELA,
       DEVICEID,
       MAX(LAST_TICK) as MAX_TICK
    from KNOWLEDGE
    where DEVICEID = CROSSID
    group by DEVICEID, TABELA
)
select 
   a.TABELA,
   a.DEVICEID,
   a.MAX_TICK,
   b.LAST_TICK AS MEU_TICK
from max_tick a
inner join KNOWLEDGE b 
   on a.DEVICEID = b.CROSSID AND a.TABELA = b.TABELA and a.MAX_TICK > b.LAST_TICK
where b.DEVICEID = INTEGER VALUE
order by a.TABELA

但我不能 100% 确定此查询是否等效，因为您的示例数据没有提供足够的变化来测试它。

【讨论】：

感谢您的帮助，我将测试您的建议，稍后我会告诉您它是否正确并提高了性能好的。您的查询结果在我的测试中似乎是正确的。我已经在我的软件中的 asp.net core api 上对您的查询进行了实时测试，结果如下（每个查询运行 10 次）：我的查询平均 22.9 毫秒，按顺序更改组，平均 25.1 毫秒，您的完整查询平均 24.7 毫秒。但是，根据您的建议，我得到了 0 次非索引读取，随着知识表的扩展，这应该胜过我的查询！我会将其标记为已接受，因为它具有潜力【参考方案2】：

你可以像这样创建一个 cte：

With cte_base AS (
select distinct LAST_TICK 
from KNOWLEDGE b 
where b.DEVICEID=INTEGER VALUE and b.CROSSID=a.DEVICEID 
and b.TABELA=a.TABELA
)

select 
    a.TABELA, 
    a.DEVICEID,
    cte_base.LAST_TICK as MEU_TICK,
    max(LAST_TICK) as MAX_TICK 
from 
    KNOWLEDGE a 
where 
    a.DEVICEID=a.CROSSID 
group by
    a.TABELA, a.DEVICEID 
having 
    max(LAST_TICK) > (cte_base.LAST_TICK) 
order by 
    a.TABELA

【讨论】：

不适用于 firebird 3.0.. 列未知。 CTE_BASE.LAST_TICK 将 cte_base.last_tick 替换为 (select LAST_TICK from cte_base) 并且成功了好的，测试了一下，结果完全一样，在索引，非索引读取和提取中。

以上是关于如何优化我的 Firebird SQL 查询？的主要内容，如果未能解决你的问题，请参考以下文章

Firebird（如何使用 SQL 查找父类别）

Firebird hash join

向 Firebird SQL 查询中的列添加小计

Firebird/Lazarus SQL 视图与选择 iif？

使用 Firebird 的“SUBSTR”函数进行查询 [关闭]

如何使用 C# Firebird 对数据库执行选择查询并将其显示在 shell 中？