当筛选日期范围超过 4 年时，Oracle 查询失去并行性
Posted
技术标签:
【中文标题】当筛选日期范围超过 4 年时，Oracle 查询失去并行性【英文标题】：Oracle query loses parallelism when filter date ranges spans more than 4 years 【发布时间】：2015-08-26 12:21:17 【问题描述】：我在做什么： 使用事实表和日期维度选择每年支付的美元总和。
我看到发生的事情： 当查询运行 1 到 4 年的数据时，优化器充分利用并行性和布隆过滤器，查询在 3 分钟内完成。 如果查询运行 5 年或更长时间，或者根本没有日期过滤器，则不使用并行性并且查询需要 90 分钟才能运行。
我的问题： 为什么在谓词中添加第 5 年或删除日期谓词会导致并行性丢失？ 我该如何解决这个问题？
事实表： 8000 万行 平行度 = 8 日期键 (INCUR_DTE_S_KEY) 是从 to_char(INCUR_DATE,'yyyymmdd') 派生的数字 表在 INCUR_DTE_S_KEY 上进行了范围分区： 分区“MED_2010”值小于 (20100101) ... 分区“MED_2016”值小于 (20160101) 分区“MED_MAXV”值小于 (MAXVALUE) INCUR_DTE_S_KEY 是一个分区的 BITMAP 索引
日期维度： 平行度 = 1 2.7 万行 无分区 DATE_S_KEY 是一个 BITMAP 索引
场景：
-
在谓词中使用 1 到 4 年的查询。运行 8 个并行进程，应用布隆过滤器，查询在 3 分钟内完成。
谓词中包含 5 年或更长时间的查询。丢失所有并行进程，丢失布隆过滤器，运行 1 个进程，查询在 90 分钟内完成。
查询无谓词。丢失所有并行进程，运行 1 个进程，丢失布隆过滤器，查询在 90 分钟内完成。
将所有表的维度索引更改为唯一并将并行度提高到 8。现在获得 4 度并行度，但没有布隆过滤器/分区识别。
-- 场景 1
EXPLAIN PLAN FOR
SELECT D."CAL_YEAR" AS "Incur_Year",
SUM(F."PAID") AS "Paid"
FROM "F_PAID" F
INNER JOIN "D_DATE" D
ON F."INCUR_DTE_S_KEY" = D."DATE_S_KEY"
WHERE D."CAL_YEAR" BETWEEN 2011 AND 2015
GROUP BY D."CAL_YEAR" ;
SELECT PLAN_TABLE_OUTPUT FROM TABLE(DBMS_XPLAN.DISPLAY());
Plan hash value: 1889948660
-------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop | TQ |IN-OUT| PQ Distrib |
-------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 5 | 110 | 117K (2)| 00:23:31 | | | | | |
| 1 | PX COORDINATOR | | | | | | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10002 | 5 | 110 | 117K (2)| 00:23:31 | | | Q1,02 | P->S | QC (RAND) |
| 3 | HASH GROUP BY | | 5 | 110 | 117K (2)| 00:23:31 | | | Q1,02 | PCWP | |
| 4 | PX RECEIVE | | 5 | 110 | 117K (2)| 00:23:31 | | | Q1,02 | PCWP | |
| 5 | PX SEND HASH | :TQ10001 | 5 | 110 | 117K (2)| 00:23:31 | | | Q1,01 | P->P | HASH |
| 6 | HASH GROUP BY | | 5 | 110 | 117K (2)| 00:23:31 | | | Q1,01 | PCWP | |
|* 7 | HASH JOIN | | 69M| 1454M| 116K (1)| 00:23:23 | | | Q1,01 | PCWP | |
| 8 | BUFFER SORT | | | | | | | | Q1,01 | PCWC | |
| 9 | PART JOIN FILTER CREATE| :BF0000 | 1826 | 20086 | 241 (1)| 00:00:03 | | | Q1,01 | PCWP | |
| 10 | PX RECEIVE | | 1826 | 20086 | 241 (1)| 00:00:03 | | | Q1,01 | PCWP | |
| 11 | PX SEND BROADCAST | :TQ10000 | 1826 | 20086 | 241 (1)| 00:00:03 | | | | S->P | BROADCAST |
|* 12 | TABLE ACCESS FULL | D_DATE | 1826 | 20086 | 241 (1)| 00:00:03 | | | | | |
| 13 | PX BLOCK ITERATOR | | 80M| 847M| 116K (1)| 00:23:19 |:BF0000|:BF0000| Q1,01 | PCWC | |
| 14 | TABLE ACCESS FULL | F_PAID | 80M| 847M| 116K (1)| 00:23:19 |:BF0000|:BF0000| Q1,01 | PCWP | |
-------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
7 - access("F"."INCUR_DTE_S_KEY"="D"."DATE_S_KEY")
12 - filter("D"."CAL_YEAR"<=2015 AND "D"."CAL_YEAR">=2011)
-- 场景 2
EXPLAIN PLAN FOR
SELECT D."CAL_YEAR" AS "Incur_Year",
SUM(F."PAID") AS "Paid"
FROM "F_PAID" F
INNER JOIN "D_DATE" D
ON F."INCUR_DTE_S_KEY" = D."DATE_S_KEY"
WHERE D."CAL_YEAR" BETWEEN 2010 AND 2015
GROUP BY D."CAL_YEAR" ;
SELECT PLAN_TABLE_OUTPUT FROM TABLE(DBMS_XPLAN.DISPLAY());
Plan hash value: 433333781
---------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
---------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 3 | 72 | 3640 (1)| 00:00:44 | | |
| 1 | HASH GROUP BY | | 3 | 72 | 3640 (1)| 00:00:44 | | |
| 2 | NESTED LOOPS | | 170 | 4080 | 3639 (1)| 00:00:44 | | |
|* 3 | TABLE ACCESS FULL | D_DATE | 2191 | 24101 | 241 (1)| 00:00:03 | | |
| 4 | VIEW PUSHED PREDICATE | VW_GBC_5 | 1 | 13 | | | | |
|* 5 | FILTER | | | | | | | |
| 6 | SORT AGGREGATE | | 1 | 11 | | | | |
| 7 | PARTITION RANGE SINGLE | | 37958 | 407K| 1206 (1)| 00:00:15 | KEY | KEY |
| 8 | TABLE ACCESS BY LOCAL INDEX ROWID| F_PAID | 37958 | 407K| 1206 (1)| 00:00:15 | KEY | KEY |
| 9 | BITMAP CONVERSION TO ROWIDS | | | | | | | |
|* 10 | BITMAP INDEX SINGLE VALUE | IX2F_PAID | | | | | KEY | KEY |
---------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - filter("D"."CAL_YEAR"<=2015 AND "D"."CAL_YEAR">=2010)
5 - filter(COUNT(*)>0 AND 2010<=2015)
10 - access("F"."INCUR_DTE_S_KEY"="D"."DATE_S_KEY")
-- 场景 3
EXPLAIN PLAN FOR
SELECT D."CAL_YEAR" AS "Incur_Year",
SUM(F."PAID") AS "Paid"
FROM "F_PAID" F
INNER JOIN "D_DATE" D
ON F."INCUR_DTE_S_KEY" = D."DATE_S_KEY"
GROUP BY D."CAL_YEAR" ;
SELECT PLAN_TABLE_OUTPUT FROM TABLE(DBMS_XPLAN.DISPLAY());
Plan hash value: 433333781
---------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
---------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 77 | 1848 | 28859 (1)| 00:05:47 | | |
| 1 | HASH GROUP BY | | 77 | 1848 | 28859 (1)| 00:05:47 | | |
| 2 | NESTED LOOPS | | 2129 | 51096 | 28858 (1)| 00:05:47 | | |
| 3 | TABLE ACCESS FULL | D_DATE | 27397 | 294K| 241 (1)| 00:00:03 | | |
| 4 | VIEW PUSHED PREDICATE | VW_GBC_5 | 1 | 13 | | | | |
|* 5 | FILTER | | | | | | | |
| 6 | SORT AGGREGATE | | 1 | 11 | | | | |
| 7 | PARTITION RANGE SINGLE | | 37958 | 407K| 1206 (1)| 00:00:15 | KEY | KEY |
| 8 | TABLE ACCESS BY LOCAL INDEX ROWID| F_PAID | 37958 | 407K| 1206 (1)| 00:00:15 | KEY | KEY |
| 9 | BITMAP CONVERSION TO ROWIDS | | | | | | | |
|* 10 | BITMAP INDEX SINGLE VALUE | IX2F_PAID | | | | | KEY | KEY |
---------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
5 - filter(COUNT(*)>0)
10 - access("F"."INCUR_DTE_S_KEY"="D"."DATE_S_KEY")
经过一些反馈后，我做了一些更改。我现在获得了并行性，但分区仍然被忽略。
-- 场景 4 -- -- 变化： -- 将 D.DATE_S_KEY 设为唯一索引 -- 将 F.INCUR_DTE_S_KEY 索引从并行 2 更改为并行 8 -- 将日期维度从平行 1 更改为平行 8 -- -- 结果并行 4 被使用但仍然丢失布隆过滤器
EXPLAIN PLAN FOR
SELECT D."CAL_YEAR" AS "Incur_Year",
SUM(F."PAID") AS "Paid"
FROM "F_PAID" F
INNER JOIN "D_DATE" D
ON F."INCUR_DTE_S_KEY" = D."DATE_S_KEY"
where D.CAL_YEAR between 2010 and 2015
GROUP BY D."CAL_YEAR" ;
SELECT PLAN_TABLE_OUTPUT FROM TABLE(DBMS_XPLAN.DISPLAY());
Plan hash value: 881547170
--------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop | TQ |IN-OUT| PQ Distrib |
--------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 4 | 96 | 1515 (1)| 00:00:19 | | | | | |
| 1 | PX COORDINATOR | | | | | | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10001 | 4 | 96 | 1515 (1)| 00:00:19 | | | Q1,01 | P->S | QC (RAND) |
| 3 | HASH GROUP BY | | 4 | 96 | 1515 (1)| 00:00:19 | | | Q1,01 | PCWP | |
| 4 | PX RECEIVE | | 4 | 96 | 1515 (1)| 00:00:19 | | | Q1,01 | PCWP | |
| 5 | PX SEND HASH | :TQ10000 | 4 | 96 | 1515 (1)| 00:00:19 | | | Q1,00 | P->P | HASH |
| 6 | HASH GROUP BY | | 4 | 96 | 1515 (1)| 00:00:19 | | | Q1,00 | PCWP | |
| 7 | NESTED LOOPS | | 170 | 4080 | 1514 (1)| 00:00:19 | | | Q1,00 | PCWP | |
| 8 | PX BLOCK ITERATOR | | | | | | | | Q1,00 | PCWC | |
|* 9 | TABLE ACCESS FULL | D_DATE | 2191 | 24101 | 33 (0)| 00:00:01 | | | Q1,00 | PCWP | |
| 10 | VIEW PUSHED PREDICATE | VW_GBC_5 | 1 | 13 | | | | | Q1,00 | PCWP | |
|* 11 | FILTER | | | | | | | | Q1,00 | PCWP | |
| 12 | SORT AGGREGATE | | 1 | 11 | | | | | Q1,00 | PCWP | |
| 13 | PARTITION RANGE SINGLE | | 37958 | 407K| 1206 (1)| 00:00:15 | KEY | KEY | Q1,00 | PCWP | |
| 14 | TABLE ACCESS BY LOCAL INDEX ROWID| F_PAID | 37958 | 407K| 1206 (1)| 00:00:15 | KEY | KEY | Q1,00 | PCWP | |
| 15 | BITMAP CONVERSION TO ROWIDS | | | | | | | | Q1,00 | PCWP | |
|* 16 | BITMAP INDEX SINGLE VALUE | IX2F_PAID | | | | | KEY | KEY | Q1,00 | PCWP | |
--------------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
9 - filter("D"."CAL_YEAR"<=2015 AND "D"."CAL_YEAR">=2010)
11 - filter(COUNT(*)>0 AND 2010<=2015)
16 - access("F"."INCUR_DTE_S_KEY"="D"."DATE_S_KEY")
【问题讨论】：
“DATE_S_KEY 是一个 BITMAP 索引”——这不是唯一值吗？如果是这样，BITMAP 索引听起来像是一个古怪的选择。 "INCUR_DTE_S_KEY" -- 是否在该索引上设置了 parallelsim？ Date_S_Key 应该是唯一的。我可以/将解决这个问题。 是的，“INCUR_DTE_S_KEY”（以及所有索引）是并行的 2. 我会在将该索引重新创建为 b-tree 后重新测试。有点长镜头，但该索引的存在和唯一/主键的缺乏可能会让优化器有点失望。 【参考方案1】：根据 Dani Schnider https://danischnider.wordpress.com/2015/03/13/time-dimension-keys-and-partitioning/ 的说法，Oracle 在使用日期作为分区键时，似乎确实希望数据类型为 DATE。我使用基于 to_char(date, 'yyyymmdd') 的整数作为拉尔夫·金博尔 (Ralph Kimball) 的代理键。
我将所有日期键切换为实际日期，使索引位图连接，并且执行计划适用于所有日期范围。
Plan hash value: 4263234925
----------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop | TQ |IN-OUT| PQ Distrib |
----------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 4 | 100 | 5921 (1)| 00:01:12 | | | | | |
| 1 | PX COORDINATOR | | | | | | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ20001 | 4 | 100 | 5921 (1)| 00:01:12 | | | Q2,01 | P->S | QC (RAND) |
| 3 | HASH GROUP BY | | 4 | 100 | 5921 (1)| 00:01:12 | | | Q2,01 | PCWP | |
| 4 | PX RECEIVE | | 4 | 100 | 5921 (1)| 00:01:12 | | | Q2,01 | PCWP | |
| 5 | PX SEND HASH | :TQ20000 | 4 | 100 | 5921 (1)| 00:01:12 | | | Q2,00 | P->P | HASH |
| 6 | HASH GROUP BY | | 4 | 100 | 5921 (1)| 00:01:12 | | | Q2,00 | PCWP | |
| 7 | NESTED LOOPS | | 195 | 4875 | 5920 (1)| 00:01:12 | | | Q2,00 | PCWP | |
| 8 | PX BLOCK ITERATOR | | 2505 | 30060 | 157 (0)| 00:00:02 | 1 | 15 | Q2,00 | PCWC | |
|* 9 | TABLE ACCESS FULL | D_DATE | 2505 | 30060 | 157 (0)| 00:00:02 | 1 | 15 | Q2,00 | PCWP | |
| 10 | VIEW PUSHED PREDICATE | VW_GBC_5 | 1 | 13 | | | | | Q2,00 | PCWP | |
|* 11 | FILTER | | | | | | | | Q2,00 | PCWP | |
| 12 | SORT AGGREGATE | | 1 | 177 | | | | | Q2,00 | PCWP | |
| 13 | PX COORDINATOR | | | | | | | | | | |
| 14 | PX SEND QC (RANDOM) | :TQ10000 | 1 | 177 | | | | | Q1,00 | P->S | QC (RAND) |
| 15 | SORT AGGREGATE | | 1 | 177 | | | | | Q1,00 | PCWP | |
| 16 | PX BLOCK ITERATOR | | 37958 | 6561K| 5554 (1)| 00:01:07 | KEY | KEY | Q1,00 | PCWC | |
|* 17 | TABLE ACCESS FULL| F_PAID | 37958 | 6561K| 5554 (1)| 00:01:07 | KEY | KEY | Q1,00 | PCWP | |
----------------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
9 - filter("D_DATE_INCUR"."CAL_YEAR"<=2015 AND "D_DATE_INCUR"."CAL_YEAR">=2009)
11 - filter(COUNT(SYS_OP_CSR(SYS_OP_MSR(COUNT(*),SUM("F_ACTIVE_MED_CLAIMS_PART"."PAID")),0))>0 AND 2009<=2015)
17 - filter("F_ACTIVE_MED_CLAIMS_PART"."INCUR_DTE_S_KEY"="D_DATE_INCUR"."CAL_DATE")
【讨论】：
哦，我早该发现的。有趣的是，我在 10 多年前提出了这样一个案例，即当您使用 DATE 数据类型作为键值时，无论如何您实际上是在使用合成键oraclesponge.wordpress.com/2005/06/28/…——这引发了一场激烈的辩论！ @Gerard，您也可以接受您的回答。看来您正确地发现了问题的核心问题。以上是关于当筛选日期范围超过 4 年时，Oracle 查询失去并行性的主要内容，如果未能解决你的问题，请参考以下文章
My97DatePicker选择两个日期范围不超过30天的demo