where 中使用 case when 无法使用索引

Posted 2022-06-22 渔夫数据库笔记

1.版本

1)操作系统

#cat /proc/version
Linux version 4.14.76-1.el7.rx.x86_64 (root@8dfc8db5e7914181ad0cf0484d9be73a) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)) #1 SMP Thu Oct 18 09:52:06 +06 2018

2)mysql数据库版本

> select version();
+-----------+
| version() |
+-----------+
| 8.0.19    |
+-----------+
1 row in set (0.00 sec)

 

2.问题描述

    一个朋友的数据库经常出现cpu使用率彪高的现象(正常时他的数据库cpu使用率在20%左右，异常时上升到 50%-60%)，通过排查发现一类可疑的sql(在 where 条件中使用了 case when)，他的原始sql我就不发了。我在测试环境写了一个更能看出问题的sql如下

select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end;

1. 表结构及统计信息如下:
[root@127.0.0.1][test_shao]> show create table test_user;
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table     | Create Table                                                                                                                                                                                                                                                                                                                                                                                                           |
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| test_user | CREATE TABLE `test_user` (
  `id` int NOT NULL AUTO_INCREMENT,
  `user_name` varchar(80) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `email` varchar(200) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `create_time` datetime DEFAULT NULL,
  `tel` varchar(30) COLLATE utf8mb4_general_ci DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=100000001 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci |
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

[root@127.0.0.1][test_shao]> show table status like 'test_user';
+-----------+--------+---------+------------+----------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
| Name      | Engine | Version | Row_format | Rows     | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time | Check_time | Collation          | Checksum | Create_options | Comment |
+-----------+--------+---------+------------+----------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
| test_user | InnoDB |      10 | Dynamic    | 99363553 |             99 |  9852420096 |               0 |            0 |   7340032 |      100000001 | 2021-02-02 15:53:23 | NULL        | NULL       | utf8mb4_general_ci |     NULL |                |         |
+-----------+--------+---------+------------+----------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
1 row in set (0.07 sec)


2.第一类 select 测试
[root@127.0.0.1][test_shao]> explain select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end;
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
| id | select_type | table     | partitions | type | possible_keys | key  | key_len | ref  | rows     | filtered | Extra       |
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
|  1 | SIMPLE      | test_user | NULL       | ALL  | NULL          | NULL | NULL    | NULL | 99363553 |   100.00 | Using where |
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
#我们看到查询并不会走 id 列的主键索引，走的是全表扫描

[root@127.0.0.1][test_shao]> select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end;
+----+-------------+--------------------------+---------------------+-------------+
| id | user_name   | email                    | create_time         | tel         |
+----+-------------+--------------------------+---------------------+-------------+
|  1 | user_test_1 | [e-mail] | 2021-02-02 15:54:56 | 15955827468 |
|  2 | user_test_2 | [e-mail] | 2021-02-02 15:54:56 | 15957465968 |
|  3 | user_test_3 | [e-mail] | 2021-02-02 15:54:56 | 15919847440 |
|  4 | user_test_4 | [e-mail] | 2021-02-02 15:54:56 | 15926839297 |
|  5 | user_test_5 | [e-mail] | 2021-02-02 15:54:56 | 15974654170 |
|  6 | user_test_6 | [e-mail] | 2021-02-02 15:54:56 | 15992752962 |
|  7 | user_test_7 | [e-mail] | 2021-02-02 15:54:56 | 15939802303 |
|  8 | user_test_8 | [e-mail] | 2021-02-02 15:54:56 | 15920752632 |
|  9 | user_test_9 | [e-mail] | 2021-02-02 15:54:56 | 15984356256 |
+----+-------------+--------------------------+---------------------+-------------+
9 rows in set (1 min 58.28 sec)

该查询的慢日志如下：
# User@Host: root[root] @  [127.0.0.1]  Id:    17
# Query_time: 118.227921  Lock_time: 0.001371 Rows_sent: 9  Rows_examined: 100000000
SET timestamp=1618278221;
select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end;

#从慢日志中我们可以看出来该sql扫描了表中所有的记录(Rows_examined: 100000000)，最终返回9条记录(id<10)



3.第二类 select 测试 
[root@127.0.0.1][test_shao]> explain select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end limit 20;
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
| id | select_type | table     | partitions | type | possible_keys | key  | key_len | ref  | rows     | filtered | Extra       |
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
|  1 | SIMPLE      | test_user | NULL       | ALL  | NULL          | NULL | NULL    | NULL | 99363553 |   100.00 | Using where |
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
1 row in set, 1 warning (0.01 sec)
#该sql依然是全表扫描

[root@127.0.0.1][test_shao]> select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end limit 20;
+----+-------------+--------------------------+---------------------+-------------+
| id | user_name   | email                    | create_time         | tel         |
+----+-------------+--------------------------+---------------------+-------------+
|  1 | user_test_1 | [e-mail] | 2021-02-02 15:54:56 | 15955827468 |
|  2 | user_test_2 | [e-mail] | 2021-02-02 15:54:56 | 15957465968 |
|  3 | user_test_3 | [e-mail] | 2021-02-02 15:54:56 | 15919847440 |
|  4 | user_test_4 | [e-mail] | 2021-02-02 15:54:56 | 15926839297 |
|  5 | user_test_5 | [e-mail] | 2021-02-02 15:54:56 | 15974654170 |
|  6 | user_test_6 | [e-mail] | 2021-02-02 15:54:56 | 15992752962 |
|  7 | user_test_7 | [e-mail] | 2021-02-02 15:54:56 | 15939802303 |
|  8 | user_test_8 | [e-mail] | 2021-02-02 15:54:56 | 15920752632 |
|  9 | user_test_9 | [e-mail] | 2021-02-02 15:54:56 | 15984356256 |
+----+-------------+--------------------------+---------------------+-------------+
9 rows in set (1 min 58.02 sec)
#执行时间同未加 limit 限制条件的sql 也差不多

该sql的慢日志如下：
# User@Host: root[root] @  [127.0.0.1]  Id:    19
# Query_time: 118.015081  Lock_time: 0.000325 Rows_sent: 9  Rows_examined: 100000000
SET timestamp=1618280076;

#从慢日志中我们可以看出，该sql也是扫描了整张表中的记录，最后返回了9条记录


3.第三类 select 测试(修改 limit限制的值)
[root@127.0.0.1][test_shao]> explain select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end limit 9;
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
| id | select_type | table     | partitions | type | possible_keys | key  | key_len | ref  | rows     | filtered | Extra       |
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
|  1 | SIMPLE      | test_user | NULL       | ALL  | NULL          | NULL | NULL    | NULL | 99363553 |   100.00 | Using where |
+----+-------------+-----------+------------+------+---------------+------+---------+------+----------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
#执行计划我们看到同上面两个sql一致

[root@127.0.0.1][test_shao]> select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end limit 9;
+----+-------------+--------------------------+---------------------+-------------+
| id | user_name   | email                    | create_time         | tel         |
+----+-------------+--------------------------+---------------------+-------------+
|  1 | user_test_1 | [e-mail] | 2021-02-02 15:54:56 | 15955827468 |
|  2 | user_test_2 | [e-mail] | 2021-02-02 15:54:56 | 15957465968 |
|  3 | user_test_3 | [e-mail] | 2021-02-02 15:54:56 | 15919847440 |
|  4 | user_test_4 | [e-mail] | 2021-02-02 15:54:56 | 15926839297 |
|  5 | user_test_5 | [e-mail] | 2021-02-02 15:54:56 | 15974654170 |
|  6 | user_test_6 | [e-mail] | 2021-02-02 15:54:56 | 15992752962 |
|  7 | user_test_7 | [e-mail] | 2021-02-02 15:54:56 | 15939802303 |
|  8 | user_test_8 | [e-mail] | 2021-02-02 15:54:56 | 15920752632 |
|  9 | user_test_9 | [e-mail] | 2021-02-02 15:54:56 | 15984356256 |
+----+-------------+--------------------------+---------------------+-------------+
9 rows in set (0.00 sec)
#但是sql执行起来却很快

该sql的慢查询日志如下：
# User@Host: root[root] @  [127.0.0.1]  Id:    20
# Query_time: 0.000758  Lock_time: 0.000294 Rows_sent: 9  Rows_examined: 9
SET timestamp=1618281582;
select * from test_user where case when '01' between '00' and '11' then id<10 else id<5 end limit 9;

#从该sql的慢查日志中我们看到该sql最终只扫描了9条记录(limit 9)

NOTE1:从上面的例子中我们看到，对于上面的sql 不加limit 限制，或者加的limit限制 大于9时，sql都是扫描整张表的数据。为什么没有使用主键索引索引？因为我们对索引列使用了函数，所以导致索引列失效(case when 函数)

NOTE2:全表扫描走索引链表的最小端往最大端扫描时扫描到id>=10之后还在继续往后扫描，并未终止扫描(上面测试中未加limit以及limit 大于10的情况)，这是为什么？确实在一直往下扫描，没搞清楚为什么

NOTE3:这时使用 use index() 或者 force index() 都无法让如下sql走主键索引