SQL优化 - 避免使用 IN 和 NOT IN

Posted 2023-05-06

参考技术A 1、效率低

2、容易出现问题，或查询结果有误 （不能更严重的缺点）

1、用 EXISTS 或 NOT EXISTS 代替

select * from test1
where EXISTS (select * from test2 where id2 = id1 )
select * FROM test1
where NOT EXISTS (select * from test2 where id2 = id1 )

2、用JOIN 代替

select id1 from test1
INNER JOIN test2 ON id2 = id1
select id1 from test1
LEFT JOIN test2 ON id2 = id1
where id2 IS NULL
妥妥的没有问题了！

PS：那我们死活都不能用 IN 和 NOT IN 了么？并没有，一位大神曾经说过，如果是确定且有限的集合时，可以使用。如 IN （0，1，2）。

优化案例1-尽量避免使用自定义函数进行大量运算

案例说明
在月底进行代码优化检查过程中。在SQL检查过程之执行次数最多的SQL。发现SQL_ID为grk7dk5amf5m7和gzzzkzbfg8j2m 在半个小时内产生大约分别15亿次执行。逻辑读也有15G
其实SQL本身很简单；是一个自定义的分割函数。

原SQL

  select to_char(a.logintime, \'yyyymmdd\'),
           to_char(a.logintime, \'HH24\'),
           2552,
           substr(a.qn, 5, 4) ad,
           regexp_substr(a.qn, \'[^_]+\', 1, 2) channelid,
           a.account,
           sysdate
      from ssdk_acc_login a
     where to_char(a.logintime, \'yyyymmdd\') = \'20170728\'
       and splitstr(a.qn, 2, \'_\') in
           (select cid from tbl_channel where channel_tag like \'广点通%\');

该SQL的执行计划

Execution Plan
----------------------------------------------------------
Plan hash value: 2543353618

-----------------------------------------------------------------------------------------------------
| Id  | Operation                      | Name               | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT               |                    |  7236 |   558K|  2445   (1)| 00:00:30 |
|   1 |  VIEW                          | VM_NWVW_2          |  7236 |   558K|  2445   (1)| 00:00:30 |
|   2 |   HASH UNIQUE                  |                    |  7236 |   607K|  2445   (1)| 00:00:30 |
|*  3 |    HASH JOIN                   |                    |  7236 |   607K|  2444   (1)| 00:00:30 |
|*  4 |     TABLE ACCESS FULL          | TBL_CHANNEL        |   154 |  2772 |     5   (0)| 00:00:01 |
|   5 |     TABLE ACCESS BY INDEX ROWID| SSDK_ACC_LOGIN     | 95232 |  6324K|  2438   (1)| 00:00:30 |
|*  6 |      INDEX RANGE SCAN          | IND_SDK_LOGIN_TIME | 95232 |       |   670   (1)| 00:00:09 |
-----------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - access("CID"="SPLITSTR"(INTERNAL_FUNCTION("A"."QN"),2,\'_\'))
   4 - filter("CHANNEL_TAG" LIKE \'广点通%\')
   6 - access(TO_CHAR(INTERNAL_FUNCTION("LOGINTIME"),\'yyyymmdd\')=\'20170728\')


Statistics
----------------------------------------------------------
     152770  recursive calls
    1833240  db block gets
     307000  consistent gets
          0  physical reads
          0  redo size
    1465885  bytes sent via SQL*Net to client
      20225  bytes received via SQL*Net from client
       1793  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      26867  rows processed    

从执行计划上来看。存在有152770递归调用和约200M的逻辑读。但是返回的行数只有2万条。逻辑读跟返回的行数严重失调。本身表ssdk_acc_login是一个大表。
对有经验的人来说；产生大量的递归条用；很容易看出问题跟自定义函数SPLITSTR有关。

通过等价改下SQL

 

    select to_char(a.logintime, \'yyyymmdd\'),
           to_char(a.logintime, \'HH24\'),
           2552,
           substr(a.qn, 5, 4) ad,
           regexp_substr(a.qn, \'[^_]+\', 1, 2) channelid,
           a.account,
           sysdate
      from ssdk_acc_login a
     where to_char(a.logintime, \'yyyymmdd\') = \'20170728\'
       and regexp_substr(a.qn,\'[^_]+\',1,2) in
           (select cid from tbl_channel where channel_tag like \'广点通%\');

 

该SQL 的执行计划

 

Execution Plan
----------------------------------------------------------
Plan hash value: 2829062945

---------------------------------------------------------------------------------------------------
| Id  | Operation                    | Name               | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |                    |    47 |  3478 |  2443   (1)| 00:00:30 |
|   1 |  NESTED LOOPS                |                    |    47 |  3478 |  2443   (1)| 00:00:30 |
|   2 |   SORT UNIQUE                |                    |   154 |  2772 |     5   (0)| 00:00:01 |
|*  3 |    TABLE ACCESS FULL         | TBL_CHANNEL        |   154 |  2772 |     5   (0)| 00:00:01 |
|*  4 |   TABLE ACCESS BY INDEX ROWID| SSDK_ACC_LOGIN     |    47 |  2632 |  2437   (1)| 00:00:30 |
|*  5 |    INDEX RANGE SCAN          | IND_SDK_LOGIN_TIME | 95232 |       |   669   (1)| 00:00:09 |
---------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - filter("CHANNEL_TAG" LIKE \'广点通%\')
   4 - filter("CID"= REGEXP_SUBSTR ("A"."QN",\'[^_]+\',1,2))
   5 - access(TO_CHAR(INTERNAL_FUNCTION("LOGINTIME"),\'yyyymmdd\')=\'20170728\')


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     214489  consistent gets
          0  physical reads
        556  redo size
    1345632  bytes sent via SQL*Net to client
      20324  bytes received via SQL*Net from client
       1802  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
      27004  rows processed     

 

执行计划
与之前的SQL的执行计划相比；代价相差无几。连接方式由hash连接改为NESTED LOOPS
统计信息：
递归调用：前面是152770次。现在是0次。
逻辑读： 前面是约200M。现在是约20M。相差10倍。
redo日志。前面不产生。现在产生556kb。影响不大。
排序： 前面不产生；现在产生排序。此服务器专用于oracle。内存48G。影响不大。

综上所述：用正则函数regexp_substr来替换 自定义函数splitstr更好。尽量避免使用自定义函数进行大量运算