PostgreSQL对or exists的优化

Posted 2021-08-17 robinson1988

在Oracle数据库中，where条件中有or exists会走Filter，这种情况一般都需要改写SQL

先创建2个测试表a,b

create table a as select * from dba_objects;
create table b as select * from a;

下面SQL在Oracle中会产生filter，跑得慢

select count(*) from a
where owner='SCOTT' or exists (select null from b where a.object_id=b.data_object_id);

SQL> select count(*) from a
  2  where owner='SCOTT' or exists (select null from b where a.object_id=b.data_object_id);

已用时间:  00: 01: 33.95

执行计划
----------------------------------------------------------
Plan hash value: 1064440332

----------------------------------------------------------------------------
| Id  | Operation           | Name | Rows  | Bytes | Cost (%CPU)| Time     |
----------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |      |     1 |    30 |   290   (1)| 00:00:04 |
|   1 |  SORT AGGREGATE     |      |     1 |    30 |            |          |
|*  2 |   FILTER            |      |       |       |            |          |
|   3 |    TABLE ACCESS FULL| A    | 74940 |  2195K|   290   (1)| 00:00:04 |
|*  4 |    TABLE ACCESS FULL| B    |   800 | 10400 |     3   (0)| 00:00:01 |
----------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - filter("OWNER"='SCOTT' OR  EXISTS (SELECT 0 FROM "B" "B" WHERE
              "B"."DATA_OBJECT_ID"=:B1))
   4 - filter("B"."DATA_OBJECT_ID"=:B1)

Note
-----
   - dynamic sampling used for this statement (level=2)


统计信息
----------------------------------------------------------
         13  recursive calls
          0  db block gets
   72687576  consistent gets
          0  physical reads
          0  redo size
        424  bytes sent via SQL*Net to client
        416  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

需要将其改写，秒杀

select count(*)
  from a left join (select data_object_id from b group by data_object_id) b
 on a.object_id = b.data_object_id where a.owner = 'SCOTT' or b.data_object_id is not null;

SQL> select count(*)
  2    from a left join (select data_object_id from b group by data_object_id) b
  3   on a.object_id = b.data_object_id where a.owner = 'SCOTT' or b.data_object_id is not null;

已用时间:  00: 00: 00.02

执行计划
----------------------------------------------------------
Plan hash value: 2430901245

----------------------------------------------------------------------------------------
| Id  | Operation               | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
----------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT        |      |     1 |    43 |       |   831   (1)| 00:00:10 |
|   1 |  SORT AGGREGATE         |      |     1 |    43 |       |            |          |
|*  2 |   FILTER                |      |       |       |       |            |          |
|*  3 |    HASH JOIN RIGHT OUTER|      | 80038 |  3360K|  1960K|   831   (1)| 00:00:10 |
|   4 |     VIEW                |      | 80038 |  1016K|       |   294   (2)| 00:00:04 |
|   5 |      HASH GROUP BY      |      | 80038 |  1016K|       |   294   (2)| 00:00:04 |
|   6 |       TABLE ACCESS FULL | B    | 80038 |  1016K|       |   290   (1)| 00:00:04 |
|   7 |     TABLE ACCESS FULL   | A    | 74940 |  2195K|       |   290   (1)| 00:00:04 |
----------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - filter("A"."OWNER"='SCOTT' OR "B"."DATA_OBJECT_ID" IS NOT NULL)
   3 - access("A"."OBJECT_ID"="B"."DATA_OBJECT_ID"(+))

Note
-----
   - dynamic sampling used for this statement (level=2)


统计信息
----------------------------------------------------------
         28  recursive calls
          0  db block gets
       2714  consistent gets
          0  physical reads
          0  redo size
        424  bytes sent via SQL*Net to client
        416  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

现在将a，b迁移到PostgreSQL12中

orcl=> select * from version();
                                                 version
---------------------------------------------------------------------------------------------------------
 PostgreSQL 12.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit
(1 row)

orcl=> show work_mem;
 work_mem
----------
 64MB

orcl=> explain analyze select count(*) from a
where owner='SCOTT' or exists (select null from b where a.object_id=b.data_object_id);
                                                       QUERY PLAN
------------------------------------------------------------------------------------------------------------------------
 Aggregate  (cost=158042495.87..158042495.88 rows=1 width=8) (actual time=20.602..20.603 rows=1 loops=1)
   ->  Seq Scan on a  (cost=0.00..158042405.13 rows=36296 width=0) (actual time=8.705..20.384 rows=7144 loops=1)
         Filter: (((owner)::text = 'SCOTT'::text) OR (alternatives: SubPlan 1 or hashed SubPlan 2))
         Rows Removed by Filter: 65441
         SubPlan 1
           ->  Seq Scan on b  (cost=0.00..2177.31 rows=1 width=0) (never executed)
                 Filter: (a.object_id = data_object_id)
         SubPlan 2
           ->  Seq Scan on b b_1  (cost=0.00..1995.85 rows=72585 width=6) (actual time=0.004..5.449 rows=72585 loops=1)
 Planning Time: 0.058 ms
 Execution Time: 20.642 ms
(11 rows)

在PostgreSQL12中，上面SQL秒杀了，而在Oracle中要跑1分33秒

从PG的执行计划中可以看到 alternatives: SubPlan 1 or hashed SubPlan 2 ，意思就是在SubPlan 1 和 hashed SubPlan 2中 二选一

SubPlan 1 有个 never executed，说明没有选择SubPlan 1，选的是SubPlan 2，也就是说没有执行SubPlan 1，执行的是SubPlan 2

SubPlan 2 loops=1，说明只执行了一次

现在调小work_mem，设置为64kB

orcl=> set work_mem='64kB';
SET
orcl=> explain analyze select count(*) from a
where owner='SCOTT' or exists (select null from b where a.object_id=b.data_object_id);
                                                     QUERY PLAN
---------------------------------------------------------------------------------------------------------------------
 Aggregate  (cost=158042495.87..158042495.88 rows=1 width=8) (actual time=222323.048..222323.049 rows=1 loops=1)
   ->  Seq Scan on a  (cost=0.00..158042405.13 rows=36296 width=0) (actual time=4.212..222321.298 rows=7144 loops=1)
         Filter: (((owner)::text = 'SCOTT'::text) OR (SubPlan 1))
         Rows Removed by Filter: 65441
         SubPlan 1
           ->  Seq Scan on b  (cost=0.00..2177.31 rows=1 width=0) (actual time=3.062..3.062 rows=0 loops=72574)
                 Filter: (a.object_id = data_object_id)
                 Rows Removed by Filter: 69980
 Planning Time: 0.057 ms
 Execution Time: 222323.070 ms

将work_mem设置为64kB之后，PG的执行计划其实就和Oracle基本上一模一样了

结论：当work_mem足够大时，PG会对filter进行优化，将子查询的表进行HASH，让它只扫描一次，如果work_mem不够大，就不针对filter进行优化