如何优化这个复杂的查询？
Posted
技术标签:
【中文标题】如何优化这个复杂的查询？【英文标题】：How can I optimize this complex query? 【发布时间】：2020-11-23 09:39:33 【问题描述】：我正在运行具有很多条件的复杂查询。在大型数据库上，查询需要 2 分钟以上。我正在为我的项目使用 Django。我可以优化这个查询吗？或者我应该使用 Django ORM 查询？哪个更快？ LN_ACCOUNT 表的行数超过 300 万行。任何建议和帮助将不胜感激。非常感谢。
SELECT COUNT(*), SUM((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = L.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT in ('1','4','5','8')
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '20.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (
select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1
) = 'Y'
)))/-100
FROM LN_CARD L
and
L.LOAN_ID IN (
select distinct (LOAN_ID) LID
from (
select (
select LOAN_ID
from LN_ACCOUNT s
where s.ACCOUNT_CODE = ac.CODE
and s.DATE_NEXT > '21.11.2020'
and s.DATE_VALIDATE <= '21.11.2020'
and rownum = 1) as LOAN_ID
from ACCOUNTS ac,
V_REP_LN_BAL vr
where vr.BAL_ACC = ac.CODE_COA
and vr.TYPE_BAL in (1, 2, 3, 4)
) lnIDS
where LOAN_ID is not null)
and (
select sum(abs(acc.SALDO_EQUIVAL_OUT))
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT in ('1','4','5','8')
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (
select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1
) = 'Y'
)
)
+ abs(
NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 3
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1) = 'Y'
)),0) +
NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 7
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1) = 'Y'
)),0) +
NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 79
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1) = 'Y'
)),0)
)
+ abs(NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 46
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1) = 'Y'
)),0))
+ abs(
NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 6
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1) = 'Y'
)),0) +
NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 52
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1) = 'Y'
)),0) +
NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 53
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1) = 'Y'
)),0) +
NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 54
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y' from
V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1) = 'Y'
)),0)
) <> 0;
【问题讨论】：
语法错误... 您在SELECT 子句和WHERE 子句中使用了很多子查询。尝试将其转换为某种 JOINS。
【参考方案1】：
您正在执行多个嵌套子查询：
+ abs(
NVL((select sum(acc.SALDO_EQUIVAL_OUT)
from LN_ACCOUNT lna, ACCOUNTS acc
where lna.LOAN_ID = l.LOAN_ID
and lna.LOAN_TYPE_ACCOUNT = 3
and lna.DATE_NEXT > '21.11.2020'
and lna.DATE_VALIDATE <= '21.11.2020'
and acc.CODE = lna.ACCOUNT_CODE
and ( substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
or (select 'Y'
from V_REP_LN_BAL v
where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
and ROWNUM = 1
) = 'Y'
)
),
0
)
唯一的区别似乎是您正在过滤的lna.LOAN_TYPE_ACCOUNT。将所有这些连接起来，这样您只需要查询一次表：
+ COALESCE(
(
SELECT SUM( t.abs_total )
FROM (
SELECT lna.LOAN_ID,
lna.LOAN_TYPE_ACCOUNT,
ABS( SUM( acc.SALDO_EQUIVAL_OUT ) ) AS abs_total
FROM LN_ACCOUNT lna
INNER JOIN ACCOUNTS acc
ON ( acc.CODE = lna.ACCOUNT_CODE )
WHERE lna.LOAN_TYPE_ACCOUNT IN ( 3, 7, 79, 46, 6, 52, 53, 54 )
AND lna.DATE_NEXT > DATE '2020-11-21'
AND lna.DATE_VALIDATE <= DATE '2020-11-21'
AND ( substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
OR EXISTS( SELECT 1
FROM V_REP_LN_BAL v
WHERE v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
)
)
GROUP BY
lna.LOAN_ID,
lna.LOAN_ACCOUNT_TYPE
) t
WHERE t.LOAN_ID = l.LOAN_ID
),
0
)
在使用ABS 之前，您需要检查以确保您仍然获得与聚合多个LOAN_TYPE_ACCOUNT 相同的结果；但是，如果是这种情况，那么您可以使用：
CASE
WHEN LOAN_TYPE_ACCOUNT IN ( 3, 7, 79 ) THEN 'Group1'
WHEN LOAN_TYPE_ACCOUNT IN ( 46 ) THEN 'Group2'
WHEN LOAN_TYPE_ACCOUNT IN ( 6, 52, 53, 54 ) THEN 'Group3'
END
【讨论】：
我还能做些什么来减少执行时间？ @Mika 我不知道。要获得更好的答案，您需要在edit 您的问题中包含一个minimal reproducible example，其中包含：表的 DDL 语句；一些样本数据的 DML 语句；解释代码试图实现的目标；您对该样本数据的预期输出；以及当前查询的执行计划。 ... 否则，我们将继续关注您的查询，并会挑选出容易获得性能的胜利，但可能会错过一些基本的东西（即可能需要完全重写但我们不知道的完全不同的方法因为我们没有您的表格或数据，也不知道您的目标是什么）。【参考方案2】：您需要显示您的查询计划。
为此，您可以使用 SQL Developper 并单击顶部工具栏中的“执行计划”图标（快捷键 F10），也可以直接使用 SQL PLus：
解释计划：
explain plan
for
select * from table_name where ...;
显示计划：
select * from table(dbms_xplan.display);
这里看看FULL TABLE SCAN 行。那些您查询完整阅读的显示表。所以如果一个表有超过 3M 行，这可能需要很长时间。为避免 FULL TABLE SCAN 在您在 where 子句中指定的列上创建索引。
【讨论】：
以上是关于如何优化这个复杂的查询？的主要内容，如果未能解决你的问题，请参考以下文章